Our recent webcast on Composable Infrastructure and Computational Storage raised some interesting questions. No need to compose your own answers – my co-presenter Philip Kufeldt and I answer them here! You can find the entire webcast video along with the slide PDF in the SNIA Educational Library. We also invite you and your colleagues to take 10 and watch three short videos on Computational Storage topics.
Q (I’m) a little confused about moving data across, for example, NVMe-oF, as it consumes DDR bandwidth. Can you elaborate?
A: Any data moving in or out of server consumes DDR bandwidth by virtue of the DMAs done. Consider a simple NFS file server, where I as a client write a 1GiB file. That data arriving from the client in the server first appears as a series of TCP packets. These packets arrive first in the NIC and then are DMA-ed across the PCIe bus into main memory where the TCP/IP stack deciphers and then ultimately delivers them to waiting NFS server software. If you have a smart NFS implementation that copy from the PCIe NIC to main memory is the only copy in the process. But you have consumed 1GiB of DDR BW. Now the NFS server SW translates the request into a series of Block IO requests to an underlying storage device via a SATA/SAS/NVMe controller. This controller will again DMA the data from memory to the device. Another 1GiB of DDR BW is consumed. Traditionally this has not really been noticed because the devices consuming this data have been slow enough to throttle how quickly the DDR BW can be consumed. Now we have SSDs capable of consuming several GiB/s of bandwidth per device. You can easily design an unbalanced system where the DDR bus actually traps storage throughput.
Q: Some vendors are now running virtual machines within their arrays. Would these systems be considered a computational storage system?
A: Computational Storage at SNIA and other working groups are defining it at the storage device level, not the system level of sorts without at least some sort of computational storage processor (CSP). While these systems have intelligence, it is not at the storage device level, but addressed about that level before the user sees it (still at the CPU level).
Q: For Composable Infrastructure, you mentioned CXL as a more evolved PCIe fabric, when it will actually be released? How about using PCIe Gen4 as a fabric, as it’s available today?
A: PCIe 4 does not provide robust memory semantics, specifically the cache coherency needed by some of these devices. This is the exact purpose of CXL: to extend PCIe to better support load store operations, including cache coherency, needed by memory and memory like devices.
Q: Computational storage moves processing into storage. Isn’t that the opposite of disaggregation in composable infrastructure?
A: It is and it isn’t. As said in the presentation, the diagram of CI was quite simplistic. I doubt there will ever be just processors connected to a fabric. Just as processors have memory built into them, level 1-3 caches, you can envision CI processor elements having some amount of local RAM as part of the component, an external level 4 cache if you will. Imagine a small PCB with a processor and some small number of DIMMs. Other memory resources might be across the fabric to complete the memory requirements of a composed system. Storage devices already have processor and memory components within them for processing IO requests. Augmenting these resources to handle only portions of processing of the governing app. Allowing cycles to migrate to the data, not the entire app but some of the data centric portions of it. This is exactly how TPU or GPU processing would work as well, migrating the computational portions of the app to the component.