ZFS vs. Hardware RAID (RAID 10)

, by
Simon Krenger

As I am currently fiddling around with Oracle Solaris and the related technologies, I wanted to see how the ZFS file system compares to a hardware RAID Controller. The ZFS file system allows you to configure different RAID levels such as RAID 0, 1, 10, 5, 6. In this post, I want to test the RAID 10 performance of ZFS against the performance with the HP RAID Controller (also in a RAID 10 configuration) over 4 disks.


Preparation

For this test, I arranged two (quite old) HP DL380 G2 (2x 1.4GHz) with 6x 10k SAS Disks. These servers feature a Smart Array 5i Controller from HP. On both servers, I configured two disks in a RAID 1 configuration as system disk. The other four disks are used for this performance test. Both servers are physically identical apart from the RAID configuration.

Host 1 was configured using the setup utility of the internal RAID controller. The disk was then formatted with ZFS and mounted as /storage.

The RAID controller of Host 2 was configured to provide a JBOD to the Operating System. In Solaris, I then used the following command to create a ZFS RAID 10 storage pool:

host2:~# zpool create storage mirror c0t1d0 c0t2d0 mirror c0t3d0 c0t4d0

Benchmark

For the benchmark, I am using IOzone (Version 3.217). I ran IOzone with the following parameters:

iozone -a -f /storage/testfile

The output of IOzone can be found in the following spreadsheet: Hardware RAID vs. ZFS Output

When processing the averages of the output and putting it into a diagram, the following diagram is drawn:

Chart of the Performance Test

Hardware RAID vs. ZFS Result Chart

Conclusion

As we can clearly see, the performance of the Hardware RAID controller is a little better than the software RAID. But you shouldn’t jump to a conclusion just seeing this diagram. First of, the servers used in this benchmark are quite old and don’t have fast processors (Intel Pentium!). This can harm the performance of the software solution. Also note that both systems are using their standard settings and were not tweaked or optimised. There are a lot of parameters that can be set in ZFS to optimise performance.

From my own point of view I was surprised that ZFS performed so well in these tests, as I was expecting much worse results. The added flexibility and security of ZFS might be more important in your environment than pure performance. Overall, ZFS is 3-13% slower than a Hardware RAID, depending what load you apply to the file system.

[…] of curiosity, I decided to run the IOzone tests I performed with a RAID 10 (see this post) on a RAID-Z and compare it to the RAID 5 of the hardware RAID controller. For this test, I am […]

Henry (April 3, 2011):

Nice findings, i’m about to build a 8 x 2TB 5400rpm drives – ZFS setup, still learning FreeBSD @ the moment.. hopefully it’ll be running up soon..

I guess you can say here, save on the RAID card and buy more drives =D

Simon Krenger (April 4, 2011):

Hei Henry, thank you for your comment.
FreeBSD offers a great ZFS implementation, featuring many of the ZFS features on Solaris.
For a NAS or another network-based storage, performance should not be the problem at all and therefore ZFS is a great choice for this kind of appliance!
Can I track your progress somewhere? I would be interested to see what hardware and software you are using…

[…] different loads on it. For this, I am using the IOzone tool I already used for my ZFS tests (1) (2) and my Amazon EC2 IO […]

Daniel (August 16, 2013):

Would whole disk or individual file encryption affect these performance figures (i.e. would it affect one RAID system more than another)?

Simon Krenger (August 16, 2013):

Hello Daniel,
Yes, an encryption always adds a performance overhead. From experience, I can tell that hardware based encryption is usually faster than software based encryption. However, modern processors have extensive features to support such an encryption and the gap between software- and hardware-based encryption has closed quite a bit. I would still assume that encryption adds a performance overhead of around 2%-10%.

Post a comment

Please enter a name, an e-mail address and your comment (all fields are required).
Your e-mail address will never be published or made available to any third party, I promise.