SSS Performance Test Specification Coming Soon

A new Performance Test Specification (PTS) for solid state storage is about to be released for Public Technical Review by the SNIA SSSI and SSS TWG.  The SNIA PTS is a device level performance specification for solid state storage testing that sets forth standard terminologies, metrics, methodologies, tests and reporting for NAND Flash based SSDs.  SNIA plans to release the final PTS v 1.0 later this year as a SNIA architecture tracking for INCITS and ANSI standards treatment.

Why do we need a Solid State Storage Performance Test Specification?

Lack of Industry Standards / Difficulty in Comparing SSD Performance

There has been no industry standard test methodology for measuring solid state storage (SSS) device performance. As a result, each SSS manufacturer has utilized different measurement methodologies to derive performance specifications for their solid state storage (SSS) products. This made it difficult for purchasers of SSS to fairly compare the performance specifications of SSS products from different manufacturers.

The SNIA Solid State Storage Technical Working Group (SSS TWG), working closely with the SNIA Solid State Storage Initiative (SSSI), has developed the Solid State Storage Performance Test Specification (SSS PTS) to address these issues. The SSS PTS defines a suite of tests and test methodologies that effectively measure the performance characteristics of SSS products. When executed in a specific hardware/software environment, SSS PTS provides measurements of performance that may be fairly compared to those of other SSS products measured in the same way in the same environment.

Key Concepts

Some of the key concepts of the PTS include proper pre test preparation, setting the appropriate test parameters, running the prescribed tests, and reporting results consistent with PTS protocol.  For all testing, the Device Under Test (DUT) must first be Purged (to ensure a repeatable test start point), preconditioned (by writing a prescribed access pattern of data to ensure measurements are taken when the DUT is in a steady state), and measurements taken in a prescribed steady state window (defined as a range of five rounds of data that stay within a prescribed excursion range for the data averages).

Standard Tests

The PTS sets forth three standard tests for client and enterprise SSDs:  IOPS, Throughput and Latency and measured in IOs per second, MB per second and average msec.  The test loop rounds consist of a Random data pattern stimulus in a matrix of R/W mixes and Block Sizes at a prescribed demand intensity (outstanding IOs – queue depth and thread count).  The user can extract performance measurements from this matrix that relate to workloads of interest.  For example, 4K RND W can equate to small block IO workloads typical in OLTP applications while 128K R can equate to large block sequential workloads typical in video on demand or media streaming applications.

Reference Test Environment

The SNIA PTS is hardware and software agnostic.  This means that the specification does not require any specific hardware, OS or test software to be used to run the PTS.  However, SSD performance is greatly affected by the system hardware, OS and test software (the test environment).  Because SSD performance is 100 to 1,000 times faster than HDDs, care must be taken not to introduce performance bottlenecks into the test measurements from the test environment.

The PTS addresses this by setting forth basic test environment requirements and lists a suggested Reference Test Platform in an informative annex.  This RTP was used by the TWG in developing the PTS.  Other hardware and software can be used and the TWG is actively seeking industry feedback using the RTP and other test environment results.

Standard Reporting

The PTS also sets forth an informative annex with a recommended test reporting format.  This sample test format reports all of the PTS required test and result information to aid in comparing test data for solid state storage performance.

Facilitate Market Adoption of Solid State Storage

The SSS PTS will facilitate broader market adoption of Solid State Storage technology within both the client and enterprise computing environments.

SSS PTS version 0.9 will be posted very shortly at http://www.snia.org/publicreview for public review. The public review phase is a 60-day period during which the proposed specification is publicly available and feedback is gathered (via http://www.snia.org/tech_activities/feedback/) across the worldwide storage industry. Upon completion of the public review phase, the SSS TWG will remove the SSS PTS from the web site, consider all submitted feedback, make modifications, and ultimately publish version 1.0 of the ratified SSS PTS.

PTS Press Release……

Watch for the press release on or about July 12, and keep an eye on http://www.snia.org/forums/sssi for updates.

SDDs have tried to displace HDDs in computers for a few years but the higher cost of flash memory has been a major barrier to wide-spread adoption.  Lower flash memory prices will help adoption but HDDs decrease in $/GB at about the same rate as SSDs so the relative ratio of prices doesn’t improve much in SSDs’ favor.  At the same time there are serious issues in performance for many computers with HDDs, associated with the slower access time of HDDs. 

There have attempts to combine the advantages of HDDs and flasy memory in the past such as Intel’s Turbo Memory and the hybrid hard disk alliance but these were mostly  dependent upon the operating system to provide performance advantages.  The latest initiatives to combine flash memory and hard disk drives to create tiered storage systems in computers are known as Blink Boot, hyperHDD and the solid state hybrid hard drive.

Most of these approaches (hyperHDD and solid state hybrid hard drive) don’t depend upon the operating system to manage the use of the flash memory and the HDDs.  In the case of the recent solid state hybrid hard drive from Seagate the 4 GB of  flash memory  on the PCB board of the hard drive is used to store the most recently accessed data that the computer is using.  This is done internally by the hard drive providing a boost in access speed for this content without any special requirements on the computer operating system.

By adding a little flash memory to a hard disk drive for frequently accessed data or even for OS and application booting while still keeping the HDD for inexpensive mass storage makes a lot of sense.  Computer storage tiering with flash memory and HDDs could finally help flash memory become mainstream in computers.