• Home
• Articles Archive
• Features List

• Media Pack
• Contact Us
• Subscribe

Maximising disk I/O performance - the benefits of SAS

From STORAGE Magazine Vol 6, Issue 3 - April 2006

SAS and SATA have revolutionised the speed of storage interconnectivity compared with previous generation systems. Even SATA - which is considered an entry-level standard - compares very favourably with earlier enterprise-class protocols. here, Charles E Gimarc, Storage Components Group, LSI Logic, considers the data throughput capability of SAS and sata with Ultra 320 SCSI, and describes how the technologies are likely to develop for the future

A large proportion of existing storage systems employ Ultra 320 parallel SCSI for boot drives, local data storage or external arrays of modest size. Over the next two years, such systems will increasingly begin to employ SAS disks instead, even for larger external arrays. For some applications, SATA makes use of the same SAS infrastructure and offers the capability to add vast storage capacity at relatively low cost.

Two primary metrics of storage subsystem performance are data throughput and I/O rate. Data throughput is typically stated in terms of MB/s and measures the maximum sustained data rate. Usually, maximum data rates are seen with sequential data streams that are either entirely Read or entirely Write operations, and employ data block sizes of 64KB or larger. I/O rate is the maximum number of I/Os the system can complete per second. Maximum I/O rates are usually seen with sequential data streams of either Read or Write operations, and have data block sizes of 512B, a single disk sector.

Many of the I/O operations from user applications are random, in which requests for data may jump around to different locations on the disk. Random I/O will never occur at the same throughput levels as sequential I/O, since a random I/O involves moving the disk head (seek time), waiting on the disk platter to spin to the right location (rotational latency) in addition to the time required to move the data. Data caches on disk drives, in RAID controllers, and in the file system can mitigate the effects of random addressing, but cannot completely eliminate them. Spreading a data volume across multiple disks and using only a portion of the disk for storage is a common technique for increasing random I/O throughput. SAS and SATA, with their increased connectivity, permit the use of volumes that are larger than with parallel SCSI and potentially have higher random throughput performance.

The I/O solution with the highest sequential data throughput will offer the best chance of supporting the highest overall I/O throughput when random and sequential I/O are both considered. For performance planning, it is therefore preferable to ensure that the storage subsystem has sufficient sequential bandwidth and connectivity before considering methods to improve random throughput.

SAS and SATA interfaces and disk drives have been in development laboratories for long enough that system developers have acquired a good understanding of their capability in a variety of storage configurations. Throughput values used in this article are measured from best-in-class solutions. These values can be used as a basis for planning storage configurations to meet specific application requirements. We cannot anticipate all the requirements, but we can provide the building blocks upon which you can base your own performance calculations.

Parallel vs. serial disk performance
For parallel Ultra320 solutions, connectivity and performance are tightly linked. Controllers typically have one or two SCSI channels, each supporting a maximum of 270 MB/s. Each channel can connect from 1 to 15 devices, sharing the 270 MB/s bandwidth. Performance variation in an Ultra320 SCSI implementation is limited and can result from any of the following factors:

• Number of disks (1 to 15 per channel)
• Mode of operation (Ultra 320, Ultra160, Ultra 2, etc.)
• PCI-X bus width (32-bit or 64-bit) and frequency (64MHz, 100MHz or 133MHz)

With the serial point-to-point implementation of SAS and SATA, connectivity and performance are independent, and can be individually optimised for the needs of each system. Performance variation can be a result of:

• Number of disks (up to 126 per controller, up to 16K per domain)
• Mode of operation (SAS 3Gb/s, SATA 1.5Gb/s, SATA 3Gb/s)
• Width of the link to system memory for PCI-Express, or for PCI-X bus width and frequency
• Width of the link from controller to disks if expanders are used

Connectivity variation may be managed through topologies of controllers, expanders and disks, up to the limit of number of addresses supported by the active components.

Availability comparison
Highly-available parallel SCSI, SAS and SATA storage subsystems are built using various RAID controllers and protection levels, as well as redundant paths and failover of controllers. SAS enhances availability by supporting redundant paths to the disk drive. SAS disks are typically tested to higher MTBF levels and lower bit error rates than SATA disks.

Throughput comparison
Figure 1, below, shows the maximum achievable throughput of Ultra320, SAS and SATA technologies. Each vertical bar indicates the highest large-IO Read throughput that can be supported by current controllers, host computer systems, and disks.
Separate bars are given for 1 and 2 channel Ultra 320, since controllers exist in either configuration. Separate bars are given for SAS and SATA with narrow (x1) links and wide links of widths 2, 4, and 8 (x2, x4, x8). The horizontal lines in the chart show bandwidth limits of various host side links used in these controllers. The Ultra 320 solutions employ native PCI-X, which is available in widths of 32-bit and 64-bit, and for frequencies of 66MHz, 100MHz and 133MHz. Some SAS and SATA solutions are available with the same PCI-X host link. SAS and SATA controllers are also available with native PCI Express with link widths of x1, x2, x4, and x8.

In most cases, Write throughput is very close to the Read throughput limit for sequential I/Os. Of course, I/Os going to RAID volumes may be much slower for Write, if parity or other redundant data techniques are used. In this article, we focus on Read throughput since it typically sets a higher throughput limit.
Figure 1 shows that the maximum throughput of an Ultra320 controller is only a third of the maximum for a SAS controller. In addition, the new serial interfaces offer the opportunity to tune data throughput by selecting the width of the disk-side link. In using the chart, maximum data throughput is set by whichever of the following parameters is lower: the vertical bar for the disk-side link, or the horizontal line for the host-side link.

planned configurations
There are several examples that will illustrate how this chart can be used to plan a storage configuration to meet
a data throughput goal.

• A SATA x1 link is limited to 115MB/s regardless of the host-side bus. One can connect as many disks as desired through expanders or port multipliers to achieve connectivity or availability through a RAID volume. But the maximum throughput will not exceed 115MB/s with a x1 narrow link.
• Disks connected with a SAS x2 link can be limited to 195, 390, 480, or 540MB/s if connected to a PCI Express x1, PCI Express x2, PCI-X 66, or anything faster or wider than these three. Again, expanders can be used to increase the drive count, but throughput will be limited by either the host side link or 540MB/s, the limit of a x2 SAS wide link.
• Disks connected with a single Ultra320 channel are limited to 270MB/s aggregate throughput, regardless of the host-side link.

For a system with a goal of 780 MB/s throughput on SAS, one would need
to configure it with:

• at least x4 SAS 1.0 link to the disks
• a PCI Express x4, PCI-X 133 MHz, or better host side link
• at least thirteen 2.5" disks, or at least nine 3.5" disks, and those disks could be attached via one or more expanders in any of several different topologies.

Disk performance data
For Ultra 320 SCSI, bandwidth planning was much simpler, but also with much lower capabilities. The choices were basically 1 or 2 SCSI busses, with a maximum of 15 devices per SCSI bus. The host side connection was PCI or PCI-X at 133 MHz or slower.

There are, of course, some PCI Express Ultra320 adapters that are available, but they all use controllers with a native PCI-X connection. So
the speeds of PCI Express x8 are not available with a single Ultra320 controller.

In the example offered, the minimum number of disks needed to achieve a given throughput is given, based upon the per-disk values in Table 1.

Currently, enterprise-class disks are offered in two form factors. The form factor has a significant impact on maximum sustained data throughput. For Ultra320 and SAS, a 3.5" disk will typically support 90MB/s sustained throughput. There is some variation between disks from different manufacturers, and also depending upon which part of the disk surface is tested, but this is a typical value useful for planning. A 2.5" Ultra320 or SAS disk will support 60MB/s sustained throughput. When we look at SATA disks, the values are lower: a 3.5" SATA disk will support around 60MB/s sustained throughput, while 2.5" disks support up to 45MB/s. These throughput numbers assume a sequential data stream, and are not significantly affected by spindle speed. For random I/Os, the supported data rate is entirely different, and is driven by rotational speed, stroke of the disk (amount of the disk capacity that is configured for the volume), I/O size and number of queued I/Os. However, the sustained sequential throughput sets the upper bound of what can be expected from each disk.

Roadmap
Understanding where these interfaces are going to be in the future could have a large influence in selecting a solution for today. Ultra320 is the end of the
line for parallel SCSI. There are no plans for increasing the data through- put or connectivity of Ultra320. This interface has grown tremendously during the past 20 years and has been very successful.

The new serial SCSI interfaces are at the beginning of their development cycle, and will grow in terms of connectivity and performance over the next several years. Standards are already being developed for 6Gb/s SAS and there are plans for extending it to 12Gb/s. Some SATA solutions are emerging that implement SATA II with a 3Gb/s serial link rate. SATA configurations supporting more than 1.0GB/s are expected soon. Extending the capability of SAS and SATA with management utilities, expanders, port multipliers, and faster and higher capacity disks is also occurring. SAS and SATA will continue to develop, with increases in performance, connectivity and management capability.

Putting it all together
The performance benefits, along with the connectivity, scalability and future roadmap of SAS and SATA, make them the technology of choice for many new systems.

This article has sought to demonstrate the limitations of the best parallel SCSI storage interface, Ultra320, and how the emerging serial SCSI products are allowing system developers and users to increase performance, connectivity and availability well beyond the levels set by the Ultra320 standard. Those increases and the additional connectivity options allow optimisation of a storage subsystem, either to meet a specific current requirement or provide room for expansion to meet future performance and capacity goals.

The data presented here seeks to illustrate how the new SAS and SATA interfaces truly do offer unprecedented performance, connectivity and capacity flexibility.
www.lsilogic.com/sas