• Home
  • About
  •  

    Updated Client Solid State Performance Test Specification Now Available

    June 26th, 2012

    SNIA’s Solid State Storage Initiative has just released a revised Client SSS Performance Test Specification (PTS-Client) which adds a new write saturation test and refines existing tests.

    The Solid State Storage Performance Test Specification (PTS) is a device-level performance test suite for benchmarking and comparing performance among SAS, SATA and PCI Express SSDs

    Revision 1.1 of the PTS-Client updates tests for IOPS, throughput and latency to more accurately reflect the workload conditions under which Client SSDs are used.  The PTS-Client v1.1 also adds a Write Saturation test that measures the initial Fresh-Out-of-Box state of SSDs and their performance evolution as data is randomly written to the device.

    Eden Kim, Chair of SNIA’s SSS Technical Working Group, describes the primary updates to PTS-Client v1.1 as adjustments to preconditioning ranges and test boundaries.   Taken together, these parameters create a repeatable test stimulus that more accurately reflects the workload characteristics of SSDs used in a single user environment The PTS-Client v1.1 also adds an easily understandable description of each test, which helps the user to understand the purpose of the test, the test flow, and guidance on how to interpret the test results.

    Sample test results using the PTS-Client v1.1 have been posted to the SNIA SSSI Understanding PTS Performance webpage.

     


    Recommended Reading List on SSDs and Performance

    February 3rd, 2012

    SSSI has developed an extensive library of educational materials about SSD performance and how to use the SSS Performance Test Specifications to measure it.  If you’re new to SSDs or simply want to become more knowledgeable on the subject, we can help.

    Below is a list of white papers, presentations, webcasts, and even a video that discuss SSDs, SSD performance and how it should be measured.  The list is in the recommended order of reading / viewing, and ranges from basic overviews to technical details.  Hope you find this useful.

    1. What more logical place to start than Solid State Storage 101?  This white paper talks about SSDs, how they work and how they fit into system architectures.
    2. Another white paper, NAND Flash Solid State Storage for the Enterprise, looks at Flash memory in more detail and how SSD controllers work.
    3. Facing an SSS Decision? Here is How SNIA is Helping Users Evaluate SSS Performance is a presentation that starts to delve into SSD performance and the basic principles of the SSS Performance Test Specification.
    4. The presentation Validating SSS Performance also introduces the SSS PTS, but in additional detail.
    5. The Solid State Storage Performance Test Specification (SSS PTS) White Paper provides an easily understandable introduction to the SSS PTS.
    6. Here’s a video of our own Eden Kim Describing the SSS PTS at Storage Visions 2012.
    7. SNIA Solid State Storage Test Specification is a more technical description of the contents of the SSS PTS.
    8. Now that you’ve read all about them, the actual SSS PTS documents can be downloaded here.
    9. And finally, SSSI has put together a webpage on Understanding SSD Performance, which explains the test results generated from the SSS PTS and what they mean to users.

    You can find a lot of other informative material related to SSDs on the SSSI Education page.

    If you have any questions, comments or requests, please comment on this post or send a message to asksssi@snia.org.


    Understand SSD Performance Project

    January 17th, 2012

    At last week’s Storage Vision conference, SSSI announced the Understanding SSD Performance project, which is intended to educate users about how to use the SSS PTS (Performance Test Specification) to make intelligent decisions about SSD performance.  You can find the press release here.

    The project outcomes so far include a new webpage at www.snia.org/forums/sssi/pts, a white paper (www.snia.org/forums/sssi/knowledge/education), and a webcast.

    Join us for the webcast on January 19 at 11AM Pacific Time by going to www.brighttalk.com/webcast/663/40549.

     


    Quick PTS Implementation

    November 11th, 2011

    PTS ProcedureNeed an abbreviated version of the SNIA SSD Performance Test Specification (PTS) in a hurry?  Jamon Bowen of Texas Memory Systems (TMS) whipped up a simple implementation of certain key parts of the PTS that can be run on a Linux system and interpreted in Excel.

    It’s a free download on his Storage Tuning blog.

    This is a boon for anyone that might want to run a internal preliminary test before pursuing a more formal route.

    The bash script uses the Flexible I/O utility (FIO) to run through part of the SSSI PTS.  FIO does the heavy lifting, and the script manages it.  The script outputs comma separated (CSV) data and the download includes an Excel pivot table that helps format the results and select the measurement window.

    Since this is a bare-bones implementation the SSD must be initialized manually before the test script is run.

    The test runs the IOPS Test from the PTS.  This test covers a range of block sizes, read/write ratios and iterates until the steady state for the device is reached (with a maximum of 25 iterations).  Altogether the test takes over a day to run.

    Once the test is complete, the downloadable pivot tables allow users to select the steady-state measurement window and report the data in a recommended format.

    See Mr. Bowen’s blog at http://storagetuning.wordpress.com/2011/11/07/sssi-performance-test-specification/ for details on this valuable download.


    Client Performance Test Specification Released

    August 9th, 2011

    Today, SSSI released the Client PTS.  Client refers to a single user / few tasks environment, as opposed to Enterprise, which implies multiple users / many tasks.  What are the differences between the Client and Enterprise PTS?

    The Enterprise PTS calls out a Write Saturation test, where the SSD is written to continuously over the entire drive capacity 4 times or for 24 hours, whichever comes first.  This test provides a good idea of the robustness of the drive in an enterprise environment. This test is not applicable to Client environments, and was not included in the Client PTS.

    The other three main types of tests measure IOPS, throughput (MB/sec), and Latency (how quickly a drive responds to commands) and are included in both Enterprise and Client PTS.  Here the Client PTS differs in that that tests may be performed on smaller segments of the drive, not all of the portions of the drive being tested need to be preconditioned, and different types of test stimulus are applied. These changes were based on the testing of literally dozens of different SSDs, as well as data provided by manufacturers of client SSDs.

    The Client and Enterprise PTS documents can be downloaded at www.snia.org/pts.


    Enterprise SSS Test Specification Released

    May 26th, 2011

    This week, the SSSI formally released the Enterprise Solid State Storage Performance Test Specification.   The SSS PTS provides standard test methodology for measuring the performance of Enterprise-class SSDs, so that the performance of products from different manufacturers may be fairly compared.  The press release can be found here

    The Enterprise SSS PTS may be downloaded from here.

    A Client (single user) SSS PTS is planned to be released in Q3 2011.


    Enterprise Solid State Storage Performance Test Specification v1.0 Draft is Available

    February 2nd, 2011

    The Enterprise Solid State Storage Performance Test Specification v1.0 draft has been posted at http://www.snia.org/publicreview/ and is available for download.  [Update: released specification is now available here.]

    The Enterprise SSS PTS provides a standard way of measuring the performance of SSDs for Enterprise applications, thus enabling fair comparisons of SSDs from different suppliers.

    It is anticipated that final release of the Enterprise SSS PTS will occur in April 2011.

    Work is ongoing in the  SSS Technical Work Group on development of a Client SSS PTS.

    More information about the SSS PTS can be found at http://www.snia.org/forums/sssi/knowledge/education/.


    SSSI Announces Enterprise & Client Performance Test Specifications

    December 17th, 2010

    Input from the public review of the v0.9 Performance Test Specification led to a decision to split it into two documents – one focused on enterprise and one focused on client performance tests.

    The Enterprise PTS is due to be released in March 2011, with the Client PTS coming later in the year.

    See the press release here.


    SSD Blind Survey at Flash Memory Summit

    August 27th, 2010

    Calypso recently presented an Industry Blind Survey of SSD Performance at the Flash Memory Summit.

    The survey compared (9) MLC, (8) SLC, and (1) 15K RPM SAS HDD.  The Chart shows all sample drives at RND 4K IOPS x Block Size for 65:35 R/W mix.  Small Blocks are in the back, large Blocks are in the front, IOPS are the Y axis.  This Chart clearly shows the general Steady State performance of SLC and MLC SSDs while referencing a 15K RPM SAS HDD.

    Take aways?  There is a lot of variance in performance between SSDs, but it is nice to see an apples to apples comparison on a Device Level.  RND 4K IOPS at a 65:35 R/W mix is a good corner case benchmark.  All  numbers are Steady State and comply with the recently released SNIA SSS Performance Test Specification.   All measurements were taken on the SNIA compliant Calypso Reference Test Platform.