• Home
  • About
  •  

    Your Questions Answered on Non-Volatile DIMMs

    April 3rd, 2017

     

    by Arthur Sainio, SNIA NVDIMM SIG Co-Chair, SMART Modular

    SNIA’s Non-Volatile DIMM (NVDIMM) Special Interest Group (SIG) had a tremendous response to their most recent webcast:  NVDIMM:  Applications are
    Here
    !  You can view the webcast on demand.

    Viewers had many questions during the webcast.  In this blog, the NVDIMM SIG answers those questions and shares the SIG’s knowledge of NVDIMM technology.

    Have a question?  Send it to nvdimmsigchair@snia.org.

    1. What about 3DXpoint, how will this technology impact the market?

    3DXPoint DIMMs will likely have a significant impact on the market. They are fast enough to use as a slower tier of memory between NAND and DRAM.  It is still too early to tell though.

    2. What are good benchmark tools for DAX and what are the differences between NVML applications and DAX aware applications?

    For benchmark tools, please see the answer for (11).

    NVML applications are written specifically for NVM (Non-Volatile Memory). They may use the open source NVML libraries (http://pmem.io/nvml) for their usage.

    DAX is a File System feature that avoids the usage of Page Cache buffers.  DAX aware applications are aware that the writes and reads would go directly to the underlying NVM without being cached.

    3. On the slide talking about NUMA, there was a mention accessing NVDIMMs from a CPU on a different memory bus. The part about larger access times was clear enough. However, I came away with the impression that there is a correctness issue with handling of ADR signal as well. Please clarify.

    If this question is asking whether the NUMA remote CPU will successfully flush ADR-protected buffers to memory connected to the NUMA near CPU then yes there is the potential for a problem in this area. However ADR is an Intel feature that is not specified in the JEDEC NVDIMM standard, so this is an Intel specific implementation question. The question needs to be posed to Intel.

    4. How common is NVDIMM compatible BIOS? How would one check?

    They are becoming more common all the time. There are at least 8 server/storage systems from Intel and 22 from Supermicro that support NVDIMMs.  Several other motherboard vendors have systems that support NVDIMMs.  Most of the NVDIMM vendors have the lists posted on their websites.

    5. How does a system go in to save? How what exactly does the BIOS have to do to get a system before asserting save?

    The BIOS does the initial checking of making sure the NVDIMM has backup supply on power loss, before it ARMs it. Also, the BIOS makes sure that any RESTORE of the previously saved data is properly done. This involves a set of operations by setting appropriate registers in the NVDIMM module – all that happens during the boot up initialization. On A/C Power Loss, the PCH (Platform Control Hub) detects the condition and initiates what is called the ADR (Asynchronous DRAM Refresh) sequence, terminating in the assertion of SAVE signal by the CPLD. Without the BIOS ARM-ing the NVDIMM module, the NVDIMM module will not respond to the SAVE signal on power loss situation.

    6. Could you paint the picture of hardware costs at this point? How soon will NVDIMM-enabled systems be able to become “the rest of us”?

    The NVDIMM use DRAM, NAND Flash, a controller and well as many other parts in addition to what are used on standard RDIMMs. On that basis the cost of NVDIMM-N is higher that standard RDIMMs.  NVDIMM-enabled systems have been available for several years and are shipping now.

    7. Does RHEL 7.3 easily support Linux Kernel 4.4?

    RHEL 7.3 is still using the 3.10 version of the Linux Kernel. For RHEL related information, please, check with Red Hat.

    You can also refer to: https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/7.3_Release_Notes/index.html

    The distribution has drivers to support the persistent memory. They have also packaged the libraries for the persistent memory.

    8. What are the usual sizes for NVDIMMs available today?

    4GB, 8GB, 16GB, 32GB

    9. Are there any case studies of each of the NVDIMM-N applications mentioned?

    You can find some examples of case studies at these websites:  https://channel9.msdn.com/events/build/2016/p466 and https://msdn.com/events/build/2016/p470

    10. What is the difference between pmem lib/pmfs in Linux and an DAX enabled files system (like ext-DAX)?

    A DAX based File System avoids the usage of Kernel Page Cache Layer for caching its write data. This would make all its write operations go directly to the underlying storage unit. One important thing to understand is, a DAX File System can still use BLOCK DRIVERS for accessing its underlying storage.

    PMFS is a File System that is optimized to use Persistent Memory, by completely avoiding the Page Cache and the Block Drivers. It is designed to provide efficient access to Persistent Memory that would be directly accessible via CPU load/store instructions.

    Refer to this link: https://github.com/linux-pmfs/pmfs for more details. PMFS, as of now is only in experimental stages.

    11. What tool is used to measure the performance?

    The performance measurement depends on what kind of Application workload is to be characterized. This is a very complex topic. No single benchmarking tool is good for all the workload characteristics.

    For File System performance, SpecFS, Bonnie++, IOZone, FFSB, FileBench etc., are good tools.

    SysBench is good for a variety of performance measurements.

    Phoronix Test Suite (http://www.phoronix.com/scan.php?page=home) has a variety of tools for Linux based performance measurements.

    12. How similar do you expect the OS support for P to be to this support for –N? I don’t see a lot of need for differences at this level (though there certainly will be differences in the BIOS).

    As of now, the open source libraries (http://pmem.io) are designed to be agnostic about the underlying memory types. They are simply classified as Persistent Memory, meaning, it could be “-N” or “-P” or something else. The libraries are written for User Space, and they assume that any underlying Kernel support should be transparent.

    The “-P” type has been thought of supporting both the DRAM and the PERSISTENT access at the same time. This might need a separate set of drivers in the Kernel.

     

    13.  Does the PM-based file system appear to be block addressable from the Application?

    A File System creates a layer of virtualization to support the logical entities such as VOLUMES, DIRECTORIES and FILES. Typically, an Application that is running in the User Space has no knowledge of the underlying mechanisms used by a File System for accessing its storage units such as the Persistent Memory. The access provided by a File System to an Application is typically a POSIX File System interface such as open, close, read, write, seek, etc.,

     14. Is ADR a pin?

    ADR stands for Asynchronous DRAM Refresh. ADR is a feature supported on Intel chipsets that triggers a hardware interrupt to the memory controller which will flush the write-protected data buffers and place the DRAM in self-refresh. This process is critical during a power loss event or system crash to ensure the data is in a “safe” state when the NVDIMM takes control of the DRAM to backup to Flash. Note that ADR does not flush the processor cache. In order to do so, an NMI routine would need to be executed prior to ADR.


    How Many IOPS? Users Share Their 2017 Storage Performance Needs

    March 24th, 2017

    New on the Solid State Storage website is a whitepaper from analysts Tom Coughlin of Coughlin Associates and Jim Handy of Objective Analysis which details what IT manager requirements are for storage performance. The paper examines how requirements have changed over a four-year period for a range of applications, including databases, online transaction processing, cloud and storage services, and scientific and engineering computing.  Users disclose how many IOPS are needed, how much storage capacity is required,  and what system bottlenecks prevent them for getting the performance they need.

    You’ll want to read this report before signing up for a SNIA BrightTalk webcast at 2:00 pm ET/11:00 am PT on May 3, 2017 where Tom and Jim will discuss their research and provide answers to questions like:

    • Does a certain application really need the performance of an SSD?
    • How much should a performance SSD cost?
    • What have other IT managers found to be the right balance of performance and cost?

    Register for the “How Many IOPS?  Users Share Their 2017 Storage Performance Needs” at https://www.brighttalk.com/webcast/663/252723


    Add to your NVDIMM Knowledge – Attend the January 28 Summit

    January 15th, 2014

    Over 150 individuals participated in the BrightTALK Enterprise Storage Summit NVDIMM webcast.  If you are eager for more information on NVDIMM, you will want to attend an upcoming SNIA Event – the Storage Industry Summit on Non–Volatile Memory.

    This Summit will take place at the Sainte Claire Hotel in San Jose, CA on January 28th as part of the SNIA Annual Members’ Symposium, and will offer critical insights into NVM, including NVDIMMs, and the future of computing. This event is complimentary to attend and you can register here.

    The Summit will take place from 8:15 AM to 5:30 PM and speakers currently include:

    • Nigel Alvares, Senior Director of Marketing, Inphi
    • Bob Beauchamp, Distinguished Engineer and Director Hardware Technology and Architecture, EMC
    • Matt Bryson, ABR Investment Strategy, LLC, SVP-Research
    • Jeff Chang, Vice President, Marketing & Business Development, AgigA Tech
    • Tom Coughlin, Founder, Coughlin Associates
    • Mark Geenen, President, TrendFocus
    • Jim Handy, Analyst, Objective Analysis
    • Jay Kidd, CTO, NetApp
    • Eden Kim, CEO, Calypso
    • Tau Leng, VP/GM, Supermicro
    • Jeff Moyer, Principal Software Engineer, Red Hat
    • Wes Mukai, VP of Cloud Computing, System Engineering, SAP
    • Jim Pinkerton, Lead Partner Architect, Microsoft
    • Adrian Proctor, VP Marketing, Viking Technology
    • Andy Rudoff, Senior Software Engineer, Intel
    • Esther Spanjer, Director, Marketing Management, SanDisk
    • Garret Swart, Database Architect, Oracle
    • Nisha Talagala, Lead Architect, Fusion-IO
    • Doug Voigt, Distinguished Technologist in Storage, HP

    Visit http://www.snia.org/nvmsummit for more information and we hope you will join us in San Jose!


    Quick PTS Implementation

    November 11th, 2011

    PTS ProcedureNeed an abbreviated version of the SNIA SSD Performance Test Specification (PTS) in a hurry?  Jamon Bowen of Texas Memory Systems (TMS) whipped up a simple implementation of certain key parts of the PTS that can be run on a Linux system and interpreted in Excel.

    It’s a free download on his Storage Tuning blog.

    This is a boon for anyone that might want to run a internal preliminary test before pursuing a more formal route.

    The bash script uses the Flexible I/O utility (FIO) to run through part of the SSSI PTS.  FIO does the heavy lifting, and the script manages it.  The script outputs comma separated (CSV) data and the download includes an Excel pivot table that helps format the results and select the measurement window.

    Since this is a bare-bones implementation the SSD must be initialized manually before the test script is run.

    The test runs the IOPS Test from the PTS.  This test covers a range of block sizes, read/write ratios and iterates until the steady state for the device is reached (with a maximum of 25 iterations).  Altogether the test takes over a day to run.

    Once the test is complete, the downloadable pivot tables allow users to select the steady-state measurement window and report the data in a recommended format.

    See Mr. Bowen’s blog at http://storagetuning.wordpress.com/2011/11/07/sssi-performance-test-specification/ for details on this valuable download.