The SNIA Solid State Storage Initiative (SNIA SSSI) announces a testing service where interested parties may submit their SSD products for testing to the SSS Performance Test Specification.

Drive Requirements

Any mSATA, SATA, SAS and PCIe SSDs can be tested. The tested device must be recognized as a logical device by CentOS 6.3 and must support Purge (via Security Erase, Format Unit, or equivalent proprietary method of Purge).

Available Tests

Testing is based on the SSS PTS version 1.1.  Visit the SSS Performance Test Service page for more information on the tests.

Testing Process

Testing will be conducted by Calypso Systems, a certified SSS PTS testing facility.  Participants must submit two (2) samples of the SSD to be tested and provide prepaid return express shipment bills (FedEx, DHL or UPS). Testing will take approximately 3-4 weeks to complete.

Any failed test, or test that will not complete, will be tested twice and error logs will be provided.  All product test result data will be kept confidential.

Test results are provided in standard SNIA Report Format as specified in the SSS Performance Test Specification.

For more details, contact ptstest@snia.org

There’s a new page on the SSSI website which describes the wide range of SSD form-factors (physical formats) on the market today.   SSSI defines three major categories – Solid State Drive, Solid State Card, and Solid State Module – and the new page provides descriptions and examples of each.

Take a look.

Just posted to the SSSI site is the latest version of the SSSI Glossary.  New to this edition is a complete set of terms from the SSS Performance Test Specification.

The new glossary can be downloaded here.

Today, SSSI released the Client PTS.  Client refers to a single user / few tasks environment, as opposed to Enterprise, which implies multiple users / many tasks.  What are the differences between the Client and Enterprise PTS?

The Enterprise PTS calls out a Write Saturation test, where the SSD is written to continuously over the entire drive capacity 4 times or for 24 hours, whichever comes first.  This test provides a good idea of the robustness of the drive in an enterprise environment. This test is not applicable to Client environments, and was not included in the Client PTS.

The other three main types of tests measure IOPS, throughput (MB/sec), and Latency (how quickly a drive responds to commands) and are included in both Enterprise and Client PTS.  Here the Client PTS differs in that that tests may be performed on smaller segments of the drive, not all of the portions of the drive being tested need to be preconditioned, and different types of test stimulus are applied. These changes were based on the testing of literally dozens of different SSDs, as well as data provided by manufacturers of client SSDs.

The Client and Enterprise PTS documents can be downloaded at www.snia.org/pts.

The SNIA SSSI site has a new page entitled Solid State Storage Standards Explained.   It provides an overview of standards on drivers, interfaces, connectors, form factors, security, and testing that must be considered when designing or evaluating an SSD.  There are links to each standards site to get further details. 

The page will be updated regularly and inputs are welcome.  Questions and comments may be sent to asksssi@snia.org.

This week, the SSSI formally released the Enterprise Solid State Storage Performance Test Specification.   The SSS PTS provides standard test methodology for measuring the performance of Enterprise-class SSDs, so that the performance of products from different manufacturers may be fairly compared.  The press release can be found here. 

The Enterprise SSS PTS may be downloaded from here.

A Client (single user) SSS PTS is planned to be released in Q3 2011.

The Enterprise Solid State Storage Performance Test Specification v1.0 draft has been posted at http://www.snia.org/publicreview/ and is available for download.  [Update: released specification is now available here.]

The Enterprise SSS PTS provides a standard way of measuring the performance of SSDs for Enterprise applications, thus enabling fair comparisons of SSDs from different suppliers.

It is anticipated that final release of the Enterprise SSS PTS will occur in April 2011.

Work is ongoing in the  SSS Technical Work Group on development of a Client SSS PTS.

More information about the SSS PTS can be found at http://www.snia.org/forums/sssi/knowledge/education/.

Input from the public review of the v0.9 Performance Test Specification led to a decision to split it into two documents – one focused on enterprise and one focused on client performance tests.

The Enterprise PTS is due to be released in March 2011, with the Client PTS coming later in the year.

See the press release here.

Calypso recently presented an Industry Blind Survey of SSD Performance at the Flash Memory Summit.

The survey compared (9) MLC, (8) SLC, and (1) 15K RPM SAS HDD.  The Chart shows all sample drives at RND 4K IOPS x Block Size for 65:35 R/W mix.  Small Blocks are in the back, large Blocks are in the front, IOPS are the Y axis.  This Chart clearly shows the general Steady State performance of SLC and MLC SSDs while referencing a 15K RPM SAS HDD.

Take aways?  There is a lot of variance in performance between SSDs, but it is nice to see an apples to apples comparison on a Device Level.  RND 4K IOPS at a 65:35 R/W mix is a good corner case benchmark.  All  numbers are Steady State and comply with the recently released SNIA SSS Performance Test Specification.   All measurements were taken on the SNIA compliant Calypso Reference Test Platform.

SSS Performance Test Specification Coming Soon

A new Performance Test Specification (PTS) for solid state storage is about to be released for Public Technical Review by the SNIA SSSI and SSS TWG.  The SNIA PTS is a device level performance specification for solid state storage testing that sets forth standard terminologies, metrics, methodologies, tests and reporting for NAND Flash based SSDs.  SNIA plans to release the final PTS v 1.0 later this year as a SNIA architecture tracking for INCITS and ANSI standards treatment.

Why do we need a Solid State Storage Performance Test Specification?

Lack of Industry Standards / Difficulty in Comparing SSD Performance

There has been no industry standard test methodology for measuring solid state storage (SSS) device performance. As a result, each SSS manufacturer has utilized different measurement methodologies to derive performance specifications for their solid state storage (SSS) products. This made it difficult for purchasers of SSS to fairly compare the performance specifications of SSS products from different manufacturers.

The SNIA Solid State Storage Technical Working Group (SSS TWG), working closely with the SNIA Solid State Storage Initiative (SSSI), has developed the Solid State Storage Performance Test Specification (SSS PTS) to address these issues. The SSS PTS defines a suite of tests and test methodologies that effectively measure the performance characteristics of SSS products. When executed in a specific hardware/software environment, SSS PTS provides measurements of performance that may be fairly compared to those of other SSS products measured in the same way in the same environment.

Key Concepts

Some of the key concepts of the PTS include proper pre test preparation, setting the appropriate test parameters, running the prescribed tests, and reporting results consistent with PTS protocol.  For all testing, the Device Under Test (DUT) must first be Purged (to ensure a repeatable test start point), preconditioned (by writing a prescribed access pattern of data to ensure measurements are taken when the DUT is in a steady state), and measurements taken in a prescribed steady state window (defined as a range of five rounds of data that stay within a prescribed excursion range for the data averages).

Standard Tests

The PTS sets forth three standard tests for client and enterprise SSDs:  IOPS, Throughput and Latency and measured in IOs per second, MB per second and average msec.  The test loop rounds consist of a Random data pattern stimulus in a matrix of R/W mixes and Block Sizes at a prescribed demand intensity (outstanding IOs – queue depth and thread count).  The user can extract performance measurements from this matrix that relate to workloads of interest.  For example, 4K RND W can equate to small block IO workloads typical in OLTP applications while 128K R can equate to large block sequential workloads typical in video on demand or media streaming applications.

Reference Test Environment

The SNIA PTS is hardware and software agnostic.  This means that the specification does not require any specific hardware, OS or test software to be used to run the PTS.  However, SSD performance is greatly affected by the system hardware, OS and test software (the test environment).  Because SSD performance is 100 to 1,000 times faster than HDDs, care must be taken not to introduce performance bottlenecks into the test measurements from the test environment.

The PTS addresses this by setting forth basic test environment requirements and lists a suggested Reference Test Platform in an informative annex.  This RTP was used by the TWG in developing the PTS.  Other hardware and software can be used and the TWG is actively seeking industry feedback using the RTP and other test environment results.

Standard Reporting

The PTS also sets forth an informative annex with a recommended test reporting format.  This sample test format reports all of the PTS required test and result information to aid in comparing test data for solid state storage performance.

Facilitate Market Adoption of Solid State Storage

The SSS PTS will facilitate broader market adoption of Solid State Storage technology within both the client and enterprise computing environments.

SSS PTS version 0.9 will be posted very shortly at http://www.snia.org/publicreview for public review. The public review phase is a 60-day period during which the proposed specification is publicly available and feedback is gathered (via http://www.snia.org/tech_activities/feedback/) across the worldwide storage industry. Upon completion of the public review phase, the SSS TWG will remove the SSS PTS from the web site, consider all submitted feedback, make modifications, and ultimately publish version 1.0 of the ratified SSS PTS.

PTS Press Release……

Watch for the press release on or about July 12, and keep an eye on http://www.snia.org/forums/sssi for updates.