Slow backups with LTO4 on SuSE Linux

Question

We have a PowerEdge 2900 III that we're working on putting into service with SuSE Linux Enterprise Server 10. The problem is that in testing we're finding that the LTO4 tape drive is very slow to back up. Although LTO4 drives are capable of 120 MB/s, we're typically getting less than 20 MB/s. I've done tests with both tar and the backup program that I'm evaluating, Arkeia Network Backup.

The drive is a PowerVault LTO-4-120HH on a Dell SAS 5 (LSI) controller. The hard drives are 15K RPM SAS drives on a PERC 6/i controller. Two drives are configured RAID-1 and hold the OS. The other four drives are RAID-5 and will be for our data. The server has 8 GB of RAM, and we've installed SuSE 10 64-bit. The OS is current with patches, although we have not installed SLES 10 SP2 and cannot at this time. As far as I can tell from Dell's support site there are no newer drivers or firmware available for anything in the system.

I have had an open case with Dell support for a couple of weeks now, but so far they have come up with no explanation for the problem. I would appreciate any suggestions.

mot99 · Answer

Have you tried changing the block size ? Maybe the current blocksize is

fixed at something like 16k. You can check this using a command like mt -f /dev/st0 status.

Try upping the block size to 64k or higher, or setting it to variable block size - "mt setblk 0" .

Here's some useful commands for testing performance on a tape drive:

http://www.adsm.org/lists/html/Veritas-bu/2004-12/msg00172.html

techinmo · Answer

Thanks for your reply. I did the tar tests on that page as best as I could with my limited knowledge of Linux. I had to make some changes to get the commands to run, so hopefully the tests gave accurate results. Here's what I found using the methodology described on that page.

tar Block factor / resulting speed (MB/s)

128 38.9

512 66.6

128 43.8

(rebooted)

128 19.7

128 54.9

512 67.0

(omitted block factor, used default) 26.6

So it certainly look like the block size is one factor. Notice, though, the test after the reboot. The first was 19.7 MB/s, but doing the same tar test again immediately resulted in 54.9. Is this the result of file caching?

I've been wondering about the transfer rate from the RAID arrays. I rebooted and ran some additional tests, letting tar report its own performance using this command: tar --totals -c -b 512 -f /dev/st0 /usr. (I've used /usr for all of the tests.) Tar reported 23 MiB/s. Then I immediately ran the command again, and the results were 134 MiB/s. Here I'm finally seeing a speed that I like!

To take the tape drive out of the equation I again rebooted and ran this: tar --totals -c -b 512 -f /dev/nul /usr. The first run reported 25 MiB/s, and the second run right afterward was 613 MiB/s.

It looks to me like the transfer rate from the hard drives is a limiting factor. I wouldn't think that this RAID configuration should be holding us back, though. Any thoughts on this?

mot99 · Answer

File system or raid array caching would have an effect on the speed at which data is read from

disk. Also reading large files will be much faster than reading lots of small ones. This link on

HP's website explains what can affect backup performance:

http://h10025.www1.hp.com/ewfrf/wc/document?dlc=de&lc=en&product=60729&cc=ch&docname=lpg50460#N417

techinmo · Answer

Those are some great resources you provided, thanks. I used a couple of the HP performance testing tools. Our LTO4 tape drive write tested at 110 MB/s - right about what I would expect. Our RAID-1 array, on the other hand, read tested at only 31.5 MB/s, and our RAID-5 array as low as 15.4 MB/s. I've shared this data with Dell support. Maybe they will come up with something.

PowerVault

Slow backups with LTO4 on SuSE Linux

Was this post helpful?