I wanted to tryOpenSolaris on the new Atom board on a seperate disk before the machine got settled into normal use (normal use covered in this post).

First impression, the LiveCD booted and worked correctly including graphics in VESA mode and the networking. The install was simple and painless, and afterwards came right up with graphics in VESA mode and the networking working on the on board RealTek chip.

Bonnie++ on the 3-4 year old local disk said that for block operations it would be able to do about 50 MB/s on reads and 35 MB/s on writes.

My main area of testing was to create a new file system and share it via NFS to an Ubuntu 9.10 workstation. Initial results were 42MB/s reads and 3-8 MB/s writes (measured by timing copies). Not so good.

I tried tuning wsize and rsize.  It turns out that those are more or less set to something reasonable. I tried noatime. It seems that any of the stuff that turns up for “NFS tuning” in Google don’t do much for reasonably modern systems on a generic network.  They may be worth revisiting for people trying to get a bit more performance out, but I want an order of magnitude more.

I disabled ZIL (obviously I wouldn’t do that for production, but I figured it was fair to do it now and assume that a sensible flash drive would give reasonably similar performance with ZIL on in the future) and tried again, and things got better. I tried running bonnie++ remotely over NFS, and OpenSolaris lost it’s networking. No amount of ifconfig up/down or un-plumbing and plumbing the interface would bring it back, so I resorted to rebooting the system.

At that point I did some research. It looks like many people have problems with the gre driver. I found a gani driver, but I also saw many people try that then end up adding a separate network interface. I didn’t bother with the gani drive.

I didn’t think I would easily be able to add a good ethernet card since most cheap PCI ones seem to be Realtek and most good Intel or Broadcom GigE cards seem to be 64bit and I didn’t think that such a card would fit. Still, I grabbed my unused Broadcom PCI-X card, and found that they left enough room on either side of the PCI slot to fit a 64bit card. Nice.

With the broadcom card, it delivered NFS writes of 32 MB/s and reads of 45 MB/s. I feel that this is reasonable evidence to suggest that the SuperMicro D510 server board will do nicely as a ZFS storage server. That SuperMicro board comes with dual Intel GigE ports, not Realtek. And it also offers 6x SATA ports.

Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
hp-xw            8G 45167  68 44605  11 14707   7 32389  57 32344   7  65.8   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16   111   0  3489  10   108   0   111   0  3999   9   110   0

For low demand home applications (or small business ones I suppose) I think it would be interesting to try Intel D510MO board with mirrored SATA disk drives, a good Intel or Broadcom GigE NIC and a pro quality USB drive, such as this: http://www.logicsupply.com/products/af4gssgh
While USB isn’t all that fast, the linked drive claims to do writes of 25 MB/s, and that means that the ZFS server is limited to 25 MB/s, that probably isn’t too bad for storing videos and photos for a lot of home users.    What would be really exciting would be if someone would make a MiniPCIe flash or SATA card (or both).  A person can dream I suppose.