We have done some "playing" with this platform using Centos, and are
slowly moving towards first live deployment .... just wondering if
anyone is doing this now and would like to share experience (I'm not
expecting any "gottcha's", but just checking bases)

On Sep 7, 8:21*am, Ross Ferris <ro... (AT) stamina (DOT) com.au> wrote:

Hey Ross! Nope, not yet and we don't have short term plans to move in
general. We just got situated on new hardware for 7.5 after hardware
problems and have to move again from a HDD test bed to a replacement
FusionIO board. We are going to be playing with DRBD soon, though, for
replication. May be interesting. We have a major IT/network center
physical relocation to deal with in the interim so upgrading is off
for a while. Wanna pop over to the USA and help patch & jump
wiring?

Cheers,

Glen

On Sep 13, 1:16*am, GlenB <batch... (AT) bellsouth (DOT) net> wrote:
Hi Glen,

If you wanna pick up the tab for the flights & accomodation ...
sure!!

Do you have the FusionIO deployed live? We are looking at upgrading
some telehoused gear at some stage in the next 12 months, so SSD
alternatives are on the horizon. We have Intel X25's deployed, which
clients LOVE, but I'm thinking of playing with something a little bit
more hardcore at the Datacentre (probably paying an IBM Premium to get
their support if needed), front ending a SAS array for 1st level backup

On Sep 12, 8:12*pm, Ross Ferris <ro... (AT) stamina (DOT) com.au> wrote:
Ross,

I ran on a FusionIO IODrive 80GB for many months before moving to
different core hardware to resolve random instability problems. It's
like putting your DB on steroids, coffee and Guarana at the same time.
Do you still have the THRASH benchmarks I sent you? Those were from
one board under an 8GB 8-core Xeon machine. You can mdadm stripe a
bunch of the boards (drives) to get even more throughput, provided you
can actually get data to the drive system fast enough. I found that
I barely consume even a 1/4 of the I/O capacity during heavy loads on
our 70-license system. Did you read the pigz vs gzip thread on the
TigerLogic D3 form? If not, check it out. The file-save took ~7
minutes when running on a FusionIO 80GB board. It now takes about 45
minutes on an 8-drive RAID-10 SAS 10K array. I can't wait to move
back!

Of course, price is always an issue with memory hardware like this.
You can't buy 80GB boards any more unless you happen to find old
inventory. 160GB is the smallest and they run ~7000USD each. For most
medium-to-low use D3-only situations I don't recommend the memory
boards unless you have cash to throw at new tech. However, if you're
planning a large SAS array for performance you may want to consider
reducing the size in favor of faster performance with a FusionIO
device. IBM and Dell are installing FusionIO solutions as OEM products
in high-end servers. You'll end up paying considerably more for it,
but it'll be in a thoroughly tested configuration that you can
severely assault with IO. Ashwood helped us acquire the boards
directly and Nathan Woods @ FusionIO was helpful with general tech
support and some fringe tech questions and benchmarks dealing with
DRBD->FusionIO iomemory device possibilities. It's kind of cutting
edge stuff at this point so we'll see what happens.

Glen

Thanks Glen,

Yes, I remember your THRASH benchmarks --> set a high bar that I'm
looking forward to overtaking :-)

Cost is ALWAYS a factor -> only reason we are looking at using IBM
version is that we will not have to keep our own spare (JUST cheaper).
Whilst price for these top end devices will remain high, cheaper
options exist that can be run off a 6Gb SAS controller that
approximate results for "real" workloads (and there are "interesting"
alternatives you can throw off a USB-3 port for demos !

Assume you were looking at DRDB to a SAS array, or FusionIO in another
box?

On Sep 13, 10:08*pm, Ross Ferris <ro... (AT) stamina (DOT) com.au> wrote:

From my experience (which could be considered limited) I doubt
you'll get 700Mbps throughput on a SAS 6Gb stack unless it is quite
large. You'll be lucky to see 400 sustained. I have an 8-drive 6Gbps
stack now and I'm lucky to see 300 when doing large file dumps.
Regardless, the block writes from D3 are many and varied so it's more
about IOPS than overall channel bandwidth. I can get the FusionIO
performance monitor to spike at 120K IOPS if I try to. I can also get
it to spike 900Mbps of data throughput if I try to. Still, the CPU and
software (D3 mostly) limits what I can physically send down the wire
in a specific period of time. The main benefit for us is read
performance since we serve so much live web content from D3. The write
speed is an additional growth benefit. What are you trying to
accomplish? You could do fine with EasyCo's MFT solution and use hot-
swap bays. You could also take a look at OCZ's less-expensive drive
card offerings.

Yup, the plan is to put DRBD on top of the iomemory-vsl device and
mirror it over to a SAS array of the same size on an environmentally &
fire protected server. D3 will use the DRBD block device. It's all
disaster planning as apposed to fail-over/redundancy. From the
benchmarks Nathan sent me the latency over 1Ge is negligible unless
you abuse the device. I'm planning on either 10Gb fiber or 20/40Gb
Infiniband to avoid any potential latency caused issues and have DRBD
work in master/slave mode with 'write validation'. I can't decide
which way, though. The custom-made Infiniband cable won't be cheap or
expandable but it's *drool* 20/40Gb.

Regards,

Glen