Exploring Backup and Restore Performance Using a Tape Library
Ward Wolfram and Steve Feibus (November 2002)
The DellTM PowerVaultTM 136T library is designed to adapt to the demands of extreme data growth and shrinking backup and restore time frames. This article explores how the performance of a PowerVault 136T tape library, as well as the architecture in which it is implemented, influences the total throughput of a backup and restore solution.
The explosive increase of data and users in today's computing environments requires organizations to implement efficient backup and restore strategies that consider future expansion. Understanding how tape drive performance and tape backup architectures affect the total throughput of a backup and restore system can help IT professionals to implement new strategies that enhance performance and improve existing backup and restore solutions.
A well-designed, well-implemented backup and restore strategy should compress the amount of time required for a full backup or restore job. To demonstrate how tape drive performance and tape backup architectures affect backup and restore performance, Dell engineers tested a DellTM PowerVaultTM 136T LTOTM tape library in different architectures.
Common backup architectures that transfer data between server storage and tape storage devices include direct attach SCSI, over-the-LAN, and storage area network- (SAN-) based configurations. The PowerVault 136T is available as a direct attach Ultra2 SCSI backup library that is directly connected to a server for local and over-the-LAN backups. The PowerVault 136T also allows installation of an optional, embedded 2 Gbps Fibre Channel router, which permits the PowerVault 136T to operate in FC-1 (100 MB/sec data transfer rate) or FC-2 (200 MB/sec) SANs. The Dell PowerVault 136T accommodates up to six Linear Tape-OpenTM (LTO) or Super Digital Linear Tape (SDLT or Super DLTtapeTM ) tape drives and 72 LTO or 60 SDLT tape cartridges.
Test environments for tape backup architectures
Dell engineers tested the PowerVault 136T in a direct attach SCSI configuration and in 100 Mbps and 1 Gbps LANs (see Figure 1 ). Testing was also done in a SAN configuration (see Figure 2 ). The Dell PowerVault 136T tape library contained six LTO drives.
Figure 1. LAN test environment
Figure 2. SAN test environment
The tests used Dell PowerEdgeTM 2550 servers installed with dual Intel® Pentium® III processors at 1.26 GHz, 2 GB of 133 MHz SDRAM, 512 KB level 2 (L2) cache, and the Microsoft® Windows® 2000 operating system with Service Pack 2. Each server had a PowerEdge Expandable RAID Controller 3, Dual Channel Integrated (PERC 3/Di), and Dell configured one 36 GB, three-spindle RAID-5 array in each server to contain all data used in the tests. The data had a 30 percent data compression ratio, as confirmed by the WinZip® file compression utility.
Every test configuration used a one-to-one mapping between servers and tape drives. For example, during a three-server backup test, each server backed up to a unique LTO drive within the PowerVault 136T. The test environments did not queue any backup or restore jobs during testing. In addition, no network client or application interference other than backup and restore activity existed within the test environments.
Testing the direct attach SCSI device
As a direct attach SCSI backup device, the PowerVault 136T received data through both channels of an Adaptec® 39160 dual-channel SCSI Peripheral Component Interconnect (PCI) adapter card installed in a PowerEdge 2550 server designated as the backup server.
Testing the over-the-LAN architecture
The 100 Mbps LAN included a Dell PowerConnectTM 2016 10/100BaseT Ethernet switch, and an embedded 100 Mbps Ethernet adapter connected six Dell PowerEdge 2550 test servers to the LAN. The 1 Gbps LAN used a Dell PowerConnect 5012 Gigabit Ethernet switch, and an embedded 1 Gbps Ethernet adapter connected each server to the LAN.
For LAN tests, one copy of the VERITAS Backup ExecTM 8.6 backup and restore application was installed on a dedicated backup server (S1), and the VERITAS Backup Exec Remote Agent for Windows NT/Windows 2000 was installed on the other five servers in the LAN (S2 through S6).
Testing the SAN-based architecture
The SAN testing used FC-1 (100 MB/sec) Fibre Channel components. The PowerVault 136T tape library, which included the embedded FC-2 router option, connected to a Dell PowerVault 56F Fibre Channel switch. Each server connected to the PowerVault 56F through a QLogic® SANbladeTM 2000 Series QLA2200/66 model host bus adapter (HBA).
The Dell team issued and managed backup and restore jobs using VERITAS Backup Exec with the SAN Shared Storage Option and Library Expansion Option modules installed on each server (S1 through S6).
Backup and restore performance: Direct attach SCSI device
In a direct attach SCSI configuration, backup and restore traffic does not flow through the LAN, but directly through the SCSI adapter card of the backup server to the PowerVault 136T. The test results, which were measured from the server, indicated that the direct attach SCSI environment provides excellent backup performance with up to 74 GB/hr per LTO drive when backing up data that had a 30 percent compression ratio.
A direct attach SCSI solution provides an efficient backup and restore approach. When internal server storage exceeds the capacity of one tape, multiple tapes are required to complete a backup or restore. A tape library, such as the PowerVault 136T, can eliminate the inconvenience of manually managing tapes. However, direct attach SCSI tape configurations typically require one tape device attached to each server to achieve high performance backup and restore results, which can quickly make this approach expensive and hard to manage.
Backup and restore performance: Over the LAN
A LAN-based backup architecture enables efficient sharing of a tape library by designating one server as the backup server (S1). The backup server manages backup and restore jobs and transfers all data traffic to an attached tape library. The backup server can be any server on the LAN or it can be a dedicated server that manages only backup and restore requests.2
Backup and restore jobs must travel across the LAN to reach the shared tape device. Every LAN component in the path of the data flow can become a limiting factor that slows the feed speed to each tape drive, because data flows only as fast as the slowest component in the data path. LAN components that affect backup and restore performance include the server network interface cards (NICs), the TCP/IP protocol, the LAN medium (typically Category 5 cabling), the LAN switch, the LAN backup server, and the Adaptec 39160 SCSI adapter card (if a direct attach SCSI server is used).
LAN bandwidth limits backup performance
In the 100 Mbps test, the slowest component was the bandwidth of the LAN. A switched 100 Mbps LAN has a segment bandwidth of 100 Mbps, which is theoretically equivalent to 45 GB/hr. Because the connection from the backup server to the LAN was the bottleneck, the 100 Mbps LAN test reached the 45 GB/hr limit after one backup job. Therefore, the 45 GB/hr limit of a 100 Mbps LAN cannot effectively feed one LTO drive to take advantage of the drive's 54 GB/hr performance.
In comparison, the 1 Gbps LAN bottleneck did not appear until the fourth concurrent backup job was initiated. A 1 Gbps LAN provides the same advantages as a 100 Mbps LAN, but its theoretical bandwidth increases tenfold. Backup results shown in Figure 3 suggest that the actual throughput of the 1 Gbps LAN backup environment was approximately six times that of the 100 Mbps LAN, or approximately 260 GB/hr.
Figure 3. Backup server (S1) performance at different LAN speeds
Figure 4 compares each server's backup performance for a five-server LAN backup. No server had priority over the LAN, and the 260 GB/hr maximum throughput for the 1 Gbps LAN was evenly distributed among all servers.
Figure 4. Throughput for individual servers during a five-server, LAN-based backup
Restore performance slower than backup performance
Regardless of the LAN speed or tape backup architecture, restore times are typically slower than backup times. Slower restore times usually result from extra file system overhead and the write penalty incurred from the RAID-5 array on which the data is being restored. Virus protection software can also impose additional overhead on restore jobs. In the Dell test, the 1 Gbps LAN restore times were approximately 50 percent slower than backup times (see Figure 5), primarily because VERITAS Backup Exec performs a file-by-file check after a file has been sent over the LAN for the restore process.
Figure 5. Backup and restore for individual servers on 1 Gbps LAN
Backup server can become bottleneck
The backup server itself can also become a bottleneck, because all LAN backup and restore traffic must flow to a single backup server. When multiple servers are backed up simultaneously over the LAN, the backup server's CPU utilization increases. Adding LAN-based backup jobs increased CPU utilization slightly in the 100 Mbps LAN and significantly in the 1 Gbps LAN. The greater increase occurred in the 1 Gbps LAN because the backup server handled more I/O requests per second. Tests show that the backup server did not reach 80 percent CPU utilization until five simultaneous LAN backups were issued on the 1 Gbps LAN.
CPU utilizations above 80 percent can result in slow performance. Thus, the Dell PowerEdge 2550 configured in this test environment, or any higher performing server, can effectively serve as a 1 Gbps LAN backup server using a PowerVault 136T.
LANs present other potential bottlenecks
Adding more servers within a LAN environment does not result in improved backup throughput, because the backup server's LAN segment can quickly reach its maximum theoretical throughput. In addition, a backup server's available memory can cause it to become a bottleneck. However, in the Dell tests of 100 Mbps LAN, 1 Gbps LAN, and 100 MB/sec SAN configurations, the available memory on both the backup server and over-the-LAN servers showed no significant variances.
Backup and restore performance: SAN
A SAN combines the high performance of a direct attach SCSI approach with the central management benefits of a LAN-based backup architecture. Unlike direct attach SCSI configurations, SANs are easily scalable and provide excellent levels of data availability. A SAN-based architecture overcomes the bottleneck issue of a LAN-based configuration, because it does not require backup and restore traffic from each server to flow through a single backup server to the tape library.
The efficiency of a SAN-based backup configuration results from data traffic flowing through a minimal number of components. The bandwidth of a single FC-1 SAN link is 100 MB/sec, which means that a single SAN link can feed a backup tape library at a theoretical throughput rate of 360 GB/hr for uncompressed data. The Fibre Channel router embedded in the PowerVault 136T is equipped with two Fibre Channel ports that feed the PowerVault 136T with up to 720 GB/hr.
The limiting components of the SAN-based architecture tested in this article were the tape drives of the Dell PowerVault 136T, due to their maximum theoretical transfer rates of 54 GB/hr. When configured with six LTO tape drives, the PowerVault 136T can back up 324 GB/hr of uncompressed data.
SAN backup and restore outperforms LAN-based architecture
In a comparison of the SAN- and LAN-based architectures, the five-server backup performance of a SAN was approximately 20 percent better than that of a 1 Gbps LAN (see Figure 6).
Figure 6. Five concurrent backup jobs on SAN, 1 Gbps LAN, and 100 Mbps LAN
When varying numbers of servers are actively backing up over the SAN, each server's backup throughput is generally the same (see Figure 7 ), because the bandwidth of a SAN can efficiently feed the PowerVault 136T with enough data to take advantage of all six tape drives. The ability of the SAN to move data at a theoretical rate of 360 GB/hr per Fibre Channel port eliminates the bandwidth bottleneck observed in the LAN environment. This performance makes the PowerVault 136T a good choice for an FC-1 or faster SAN and demonstrates that a SAN architecture is a quality foundation for future, faster tape technologies.
Figure 7. SAN backup performance results with six servers
The SAN environment was also 50 percent more effective when restoring data than the 1 Gbps LAN (see Figure 8 ). Unlike a LAN, a SAN does not use VERITAS Backup Exec Remote Agent to perform file-by-file checks on each server. Administrators must install full versions of VERITAS Backup Exec and the SAN Shared Storage Option and Library Expansion Option modules on each server to speed the restore process significantly.
Figure 8. SAN versus 1 Gbps LAN restore for individual servers
Performance is not the only benefit of a SAN backup and restore solution. Because backup and restore data travels on a SAN and not on an organization's LAN, backup and restore traffic does not slow LAN performance and response times, nor are backups affected by other LAN traffic. The Fibre Channel protocol is more efficient than TCP/IP and therefore a better solution for efficient movement of raw data.
Backup architecture options for the PowerVault 136T
Direct attach SCSI devices offer an efficient, high-performance backup and restore method for one- or two-server computing environments. As the number of servers within a network grows, and multiple tapes are needed to complete backup and restore jobs, direct attach SCSI backup configurations can quickly become unmanageable.
A high-performance tape library can eliminate the manageability problems associated with multiple tapes, but direct attach SCSI devices do not permit efficient sharing of tape devices. Organizations must incur the cost of attaching a tape library to every server or take a different approach to their backup and restore needs.
LANs offer a familiar and affordable backup and restore method, but administrators must consider network speed. A 100 Mbps LAN backup environment cannot move data faster than its 45 GB/hr theoretical limit, so it seriously underutilizes a PowerVault 136T, which has a potential backup bandwidth of 324 GB/hr of uncompressed data if implemented with six LTO tape drives. For a 100 Mbps LAN, a one- or two-tape drive solution, such as the PowerVault 128T SDLT Mini-Library, is more appropriate.
A fast network, in combination with a high-performance backup server and the PowerVault 136T tape library, can make a LAN a good backup and restore solution for some companies. Organizations that require the backup of large or ever-increasing amounts of data, however, may find the limitations imposed by the LAN backup server bottleneck unacceptable.
A SAN architecture is a proven technology that permits backup strategies to embrace central management and high bandwidth over long distances. Extremely scalable, SANs enable the use of multiple tape libraries, such as the PowerVault 136T, and highly available storage arrays, such as the Dell|EMC CX600. SAN configurations also allow for redundant paths and components, which significantly enhance data availability. Because a SAN is its own network and can be difficult to access using traditional TCP/IP snooping tools, administrators can implement highly secure SAN environments.
A drawback to a SAN backup solution can be cost. SANs require a hardware investment and full copies of backup software on each server, which can make the price of a SAN notably higher than that of a LAN. To justify and enhance the investment of a SAN solution, many computing environments also provide external storage capacity to servers on the SAN, which means higher availability and better return on investment (ROI) than direct attach SCSI storage.
For computing environments requiring superior backup and restore performance and data protection, the ROI benefits of a SAN far outweigh its additional costs. A SAN backup architecture with a PowerVault 136T and Fibre Channel router provides a scalable, first-rate backup and restore solution.
Ward Wolfram (ward_wolfram@dell.com) is a storage performance/solution engineer on the Dell Solution Enablement Lab and Technology Showcase team. His responsibilities include performance and best practice analysis for SAN, network attached storage (NAS), and tape backup solutions. He received an M.S. in Computer Science from the University of Nebraska at Lincoln and a B.S. in mathematics from Concordia University in Seward, Nebraska.
Steve Feibus (steve_feibus@dell.com) is a storage enterprise technologist in the Advanced Systems Group at Dell. Steve has a B.S. in Electrical Engineering from the University of Florida and has spent many years solving customer backup issues using the latest technologies and products.
Factors affecting tape drive performance
The potential throughput of any given tape drive is just that—potential. Actual performance of a tape drive depends on the data's compression ratio. The tape drives that the PowerVault 136T supports—LTO and SDLT—have theoretical transfer rates for uncompressed data of 54 GB/hr for LTO and 40 GB/hr for SDLT.
The more compressible the data, the better the tape throughput. To determine a tape device's throughput potential, administrators can identify the data compression ratios of their computing environments by applying different file compression utilities, such as WinZip, to a well-defined data set.
Another critical factor affecting performance on many tape drives, including LTO and SDLT, is feed speed . Because tape is a streaming medium, the drive can achieve its theoretical maximums only when the tape is moving forward, and this can happen only if data is fed fast enough to the tape drive's buffer to keep it from running out of data. If the drive outruns the feed speed (called starving ), the tape must be stopped and repositioned (commonly called shoe shining ) to start streaming once the buffer has sufficient data. Shoe shining significantly reduces the performance of any tape drive. Feeding the tape drive to keep the buffer full is a key objective of a good backup and restore solution.
