Saturday, May 20, 2006


ZFS I/O reordering benchmark

I've posted about the write performance of ZFS compared to UFS, ext3fs and reiserfs (different OS's but exactly identical hardware (the same server) on the same disk cylinders.) That wasn't the extent of the benchmarking I performed, however. I'll detail another one that I did, although it's not as much a benchmark as a demonstration of the I/O reordering that ZFS does to give reads preference over writes.

The basic idea is to read a file while there's a heavy write load to the filesystem. The write load I was applying to the filesystem was the same one I detailed in this earlier entry. Again, I started out with a load that was serious overkill for the filesystems and then reran the test with a more reasonable workload.

I first created a data file to be read, 2GB for the overkill load, 512MB for the reasonable load. (The sizes were chosen to guarantee that the read of the file always took less than the time to generate the write load. I could have generated a longer write load so that I could reasonably compare the two different tests, but the tests were meant to be compared across filesystems with a similar write load, not across loads for the same filesystem.)

Instead of using cat or dd to read the file, I used md5sum. Why? Well, for one, 'cat file > /dev/null' has interesting behavior under Solaris. The cat utility actually mmaps the file and then relies on demand paging to read the file as needed. So 'cat file > /dev/null' essentially translates into a no-op. 'cat file | cat > /dev/null' achieves the goal, but it seems silly. I could have used dd, but I actually used md5sum in order to slightly handicap ZFS. Given that ZFS involves a lot of computation (checksums and whatnot), I wanted to give the CPU some more work to do so as to intentionally interfere with that. I also did it in order to create a more realistic test. In general, you don't read data from disk just to throw it away, you do something with it.

To quote from my earlier post for completeness here: The server I was using was a 2 x 2.8GHz Dell 1850 with a single 73GB SCSI disk and 2GB RAM. I ran the tests using both UFS and ZFS under Solaris x86 and both ext3fs and reiserfs under Linux. To avoid differences in performance between the inside and the outside of the disk, I used the same cylinders on the disk for all tests (plus or minus a cylinder or two.)

So here are the results for the overkill write load:

FilesystemTime (min:sec, unloaded)Time (min:sec,loaded)Ratio loaded:unloaded

Wow. So while ZFS performed the read under load in close to the same time it took with no load, ext3fs took 90 times as long, and reiserfs took 125 times as long. Again, all I can say is, "Wow." But I also have to emphasize that this write load was too heavy for the filesystems. (Although one could argue that the write load wasn't too heavy, given that ZFS could handle it gracefully. But it's certainly not a real-world workload, so while the data is interesting, it would be hard to argue that it's useful.)

And the results for the reasonable write load. (Note that I didn't run the test for UFS, mostly due to time constraints when I was doing this. Remember also that this was a 512MB file, not 2GB.):

FilesystemTime (min:sec, unloaded)Time (min:sec,loaded)Ratio loaded:unloaded

Okay, so these numbers aren't quite as ludicrous as the first. They're still impressive, though. The ZFS engineers appear to have done a very good job.

Hallo I absolutely adore your site. You have beautiful graphics I have ever seen.
This site is one of the best I have ever seen, wish I had one like this.
I find some information here.
The Linux numbers are incomprehensible unless you quote kernel version and which IO scheduler was used.

Linux performs the read-vs-write balancing at the IO scheduler level. It matters. A lot.
If you are eq2 plat looking buy ffxi gil as well as FFXI Gil WOW Power final fantasy gil Leveling lotro gold and World Of eve isk When you aoc gold need lineage 2 adena someone FFXI Gil to Maple Story mesos listen´╝îworld of warcraft gold I'll be 2moon dil there.lord of the rings gold When world of warcraft power leveling you world of warcraft power leveling need Guild Wars Gold a hug,final fantasy gil I'll be age of conan gold there.
This is a great article, supplies the useful information for me, thanks in this
soft sofa
Long span shelving
baby stroller
oil paintings
outdoor ceramic tiles
casual shoes
oil paintings
To introduce the latest nike.adidas soccer shoes. Hope you will enjoy it Nike mercurial soccer cleats Prefer the new nike soccer shoes.
Thanks. I always enjoy reading your posts - they are always humorous and intelligent.I am a china tour lover,You can learn more: China vacation packages | China city tours | China Travel Agency
Thanks. I habitually relish reading your blog - they are habitually funny and intelligent.I am a ceramic tour lover,You can discover more .
Please visit for escort girls-
Visit for select you dating and escort girl in Hong Kong.
The best place to learn mandarin Chinese is in China. However, we understand that it isn't always possible to move here to study Chinese language. The next best thing is to study with our experienced teachers in a virtual classroom. Online students enjoy the same excellent way of Chinese Online Courses and custom designed courseware that we provide for our face to face clients.
Post a Comment

<< Home

This page is powered by Blogger. Isn't yours?