<ARCHIVE> Wilmer's stuff

I have the Pandaboard running as my home server for a while now. Until last weekend, I was using a Microdrive as its root filesystem. Sadly, the drive seems to be broken. :-( That means I finally had a chance to try bootstrapping a server very quickly using Puppet. This worked fairly well, which means the time investment is paying off already.

Since all the storage I had at home was the 32GB SD card I bought for this thing anyway, I decided to give it another chance. At some point I was reminded already that alignment really matters with these things. Some Bonnie++ runs do seem to confirm this. I removed the second partition on the SD, and recreated it on a 4MB barrier. (The trick to do this is to use the "u" command in fdisk to switch units to sector instead of cylinders, and make sure the start sector is a multiple of 8192.)

To be honest, I did run most of these benchmarks with the SD card reader/writer in my desktop machine. Only the last test was done on my Pandaboard, but as you can see the results are very similar.

Version 1.96		Sequential Output						Sequential Input				Random Seeks			Sequential Create						Random Create
	Size	Per Char		Block		Rewrite		Per Char		Block				Num Files	Create		Read		Delete		Create		Read		Delete
		K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	K/sec	% CPU	/sec	% CPU		/sec	% CPU	/sec	% CPU	/sec	% CPU	/sec	% CPU	/sec	% CPU	/sec	% CPU
ext3-noalign	4G	189	41	3514	1	4878	1	2646	98	21147	2	20.9	0	16	463	0	+++++	+++	2832	3	3214	4	+++++	+++	3221	4
	Latency	6643ms		28833ms		19841ms		6668us		483ms		47888ms		Latency	1335us		655us		933us		531us		85us		60814us
ext3	4G	437	93	3727	1	5631	1	2725	99	21484	2	21.5	0	16	938	1	+++++	+++	4303	5	3743	6	+++++	+++	4304	5
	Latency	412ms		32024ms		5622ms		6422us		469ms		52114ms		Latency	1574us		760us		713us		1644us		13us		585us
logfs	4G	1149	75	13670	5	3530	4	1067	81	5920	7	6.5	3	16	2033	9	+++++	+++	4731	22	1755	8	+++++	+++	1516	11
	Latency	10944us		1810ms		4562ms		29505us		54279us		2401ms		Latency	568ms		9300us		7881us		2783ms		1483us		570ms
nilfs2	4G	797	95	7118	2	5186	1	2711	98	24112	2	617.3	29	16	2102	36	+++++	+++	+++++	+++	4592	71	+++++	+++	10886	56
	Latency	20454us		2767ms		2788ms		25974us		21810us		3432ms		Latency	4928ms		1288us		1020us		1339us		358us		294us
ext4	4G	443	91	18315	2	9299	2	2602	99	25041	2	22.4	0	16	3567	8	+++++	+++	4369	7	3792	7	+++++	+++	4393	6
	Latency	32264us		920ms		926ms		12851us		14137us		5042ms		Latency	530us		1451us		1316us		393us		401us		771us
ext4-panda	1496M	106	96	16838	8	8865	7	663	99	22243	12	30.1	1	16	3479	26	+++++	+++	4035	23	8760	65	+++++	+++	10388	63
	Latency	78219us		3925ms		961ms		13732us		104ms		1251ms		Latency	1190us		1801us		1892us		762us		61us		671us

Click here for a table not f*cked up by my blog software.

Although the throughput numbers for ext3 are pretty similar for non-aligned and aligned access, look at the latency numbers. Unfortunately I haven't got a clue how Bonnie++ calculates these and can't find very good documentation on it. Throughput may be average and latency worst-case? Either way, as you can see a misaligned partition can cause some slowdowns.

What surprised me more is that a switch to ext4fs sped up things a lot more, up to the point that the performance is perfectly reasonable! I'm running with this SD as my root filesystem now and everything just works. (While before a simple apt-get install run could take several minutes.)

While I was at it, I also tried out logfs and nilfs2, which are officially optimised for flash media. However, AFAIK they're more meant for raw NAND storage, not for block devices with all the NAND logic abstracted away (like anything you buy in stores these days). Not worth it for these SDs.

Obviously this test is far from scientific. Only in the case of ext4-panda have I run the test five times to then pick a decent result (there were some outliers in all areas). All other tests were done on a freshly formatted filesystem, which I'm sure also doesn't make the result that reliable.

Just my 2 cents! But my Pandaboard's definitely happier now. Here's hoping that wear leveling works well..

If you're interested, here is a more thorough overview of SD card performance. The LWN article about flash storage it links to is interesting too. The Flash card I used here is a 32GB class 10 Transcend card.