Introduced in 2011, non-volatile memory express (NVMe) lets companies access and use their data much faster and, in doing so, overcome the limitations of traditional data storage protocols designed for slower, spinning hard disk drives (HDDs).
NVMe was specifically designed for flash and next-generation solid-state drives (SSDs). Its faster data transfer speeds and reduced latency give it significant advantages over traditional data storage architectures, enabling businesses to reduce costs, optimise performance, improve the user experience, enhance data security, gain a competitive edge, and future-proof their data storage.
Read on to discover why NVMe storage is so important, how it works, its key features and benefits, and how Pure Storage lets you optimise the potential of your NVMe storage.
How NVMe Works
NVMe is a storage transfer protocol for accessing data quickly from flash memory storage devices such as SSDs. It enables flash memory to communicate directly with a computer via a high-speed peripheral component interconnect express (PCIe) bus, offering fast, high-throughput, and massively parallel data transfer.
Here’s how it works:
- The host writes I/O command submission queues and doorbell registers (i.e., ready signals).
- The NVMe controller fetches and executes the I/O command queues and returns I/O completion queues followed by an interrupt to the host.
- The host records the I/O completion queues and clears the door register.
The end result is significantly lower overhead when compared with traditional transfer protocols such as serial attached SCSI (SAS) and serial ATA (SATA).
Also, NVMe is optimised for non-uniform memory access (NUMA), meaning it was designed to allow for multiple CPU cores to manage queues.
Benefits of NVMe
NVMe’s main benefits include:
High Throughput and Low Latency
NVMe leverages the high-speed PCIe bus for latency, allowing for significantly faster data transfer rates compared to older interfaces. This results in substantially lower latency and higher input/output operations per second (IOPS).
Performance Enhancement
NVMe's ability to deliver high-speed data transfer and low latency significantly improves the performance of storage systems. NVMe uses parallel data paths through multiple queues, each capable of handling up to 64,000 commands, to support multi-core processors used for applications requiring fast data access, such as databases, virtualised environments, and high-performance computing (HPC).
Scalability
NVMe's architecture supports scalable performance as SSD technology evolves, ensuring compatibility with future advancements in storage.
Enterprise Applications
In enterprise environments, NVMe is essential for handling large-scale data operations and supporting demanding workloads, enabling faster data analytics, reduced processing times, and improved overall efficiency.
Cost Efficiency
Although NVMe may initially be more expensive than traditional HDDs, its superior performance and durability usually leads to long-term cost savings through increased productivity and reduced downtime.
Consumer Experience
For individuals, NVMe offers faster boot times, quicker file transfers, and a more responsive computing experience. This is particularly beneficial for gamers, content creators, and professionals working with large files.
Future-proofing
NVMe is very extendable and works well with all emerging persistent memory technologies. It’s also much more flexible than transmission control protocol (TCP), as it can be deployed on any TCP network without special hardware, making it attractive for both on-premises and cloud environments.
NVMe Advantages over Traditional Protocols
NVMe offers numerous advantages over traditional storage protocols like SAS and SATA, particularly in terms of performance, scalability, and efficiency.
Here are some specific metrics and examples illustrating these differences:
1. Command Queues and Depth
- NVMe supports up to 64,000 queues, with each queue handling up to 64,000 commands.
- SATA supports a single queue with a maximum of 32 commands (known as Native Command Queuing, or NCQ).
- SAS typically supports up to 256 queues, each with a depth of 256 commands.
The vast number of command queues and the depth supported by NVMe allows systems to minimize latency and maximise throughput. This is especially beneficial in multi-core systems where multiple processors can issue commands simultaneously.
2. Data Transfer Speeds
- NVMe uses the PCIe interface. PCIe 3.0 offers a theoretical maximum throughput of about 3.94GB/s per lane, with typical NVMe drives using four lanes, yielding up to approximately 15.75GB/s. PCIe 4.0 doubles the throughput of PCIe 3.0, offering up to 7.88GB/s per lane, with typical drives using four lanes, resulting in up to 31.5GB/s.
- SATA offers a maximum throughput of 600MB/s (SATA III).
- SAS-3 (12Gb/s) offers a theoretical maximum of 1.5GB/s per lane, often using multiple lanes for higher throughput.
An NVMe SSD connected via PCIe 4.0 can achieve transfer speeds up to 31.5GB/s, which is significantly faster than the 600MB/s limit of SATA III or the 1.5GB/s per lane of SAS-3. This makes NVMe particularly advantageous for data-intensive applications such as real-time data processing and large-scale database management.
3. Latency
- NVMe is designed for low latency, typically enabling latencies of around 20 microseconds or less.
- SATA has higher latencies due to the older AHCI protocol, typically around 100 microseconds.
- SAS latencies are generally better than SATA but still higher than NVMe, often around 50-100 microseconds.
Lower latency in NVMe drives means faster response times for applications, which is crucial for performance-critical tasks like high-frequency trading, real-time analytics, and interactive gaming.
4. Protocol Efficiency
- NVMe has an optimised command set for flash memory, reducing overhead and enabling more efficient data processing.
- SATA and SAS command sets are designed for spinning disks, resulting in higher overhead when used with SSDs.
The NVMe protocol reduces unnecessary command translations and utilizes fewer CPU cycles, which directly contributes to faster read and write operations. This efficiency is noticeable in enterprise environments where high transaction rates and minimal latency are critical.
5. Scalability
- NVMe easily scales with advancements in PCIe technology, supporting newer versions like PCIe 5.0 and beyond, ensuring compatibility with future storage innovations.
- SATA and SAS are limited by the aging architecture and slower evolution compared to PCIe.
Businesses adopting NVMe can future-proof their storage infrastructure, ensuring long-term performance gains and compatibility with cutting-edge storage technologies.
What Is NVMe over Fabrics (NVMe-oF)?
NVMe-oF is the practice of connecting NVMe storage systems with hosts over a network or data fabric. A data fabric simply refers to the network architecture, transfer protocol, and other technologies and services that allow data to be accessed and managed seamlessly across this network. It’s about extending the low latency and performance capabilities of NVMe over PCIe to storage area networks (SANs) through NVMe-friendly standards for popular transfer protocols such as Ethernet, Fibre Channel, and TCP.
The NVMe over Fabrics (NVMe-oF) specification was created and is currently maintained by NVM Express, an open collection of standards for non-volatile memory technologies. Let’s take a closer look at NVMe transfer protocols supported by this standard.
What Is NVMe over Fibre Channel (NVMe/FC)?
NVMe over Fibre Channel (also known as NVMe/FC or NVMe-FC) is a high-speed transfer protocol for connecting NVMe storage systems to host devices over fabrics. It supports the fast, in-order, lossless transfer of raw block data between NVMe storage devices in a network.
The original Fibre Channel Protocol (FCP) was designed to transport SCSI commands over Fibre Channel networks. It has become the dominant protocol used to connect servers with shared storage systems. While traditional FCP can be used to connect servers with NVMe storage devices, there’s an inherent performance penalty incurred when translating SCSI commands into NVMe commands for the NVMe array.
NVMe/FC supports the transfer of native NVMe commands, eliminating this translation bottleneck. This unlocks the full potential of FCP as a transport technology for end-to-end NVMe storage solutions, including parallelism, deep queues, multi-queueing, and high-speed data transfer.
What Is NVMe over TCP (NVMe/TCP)?
NVMe over TCP (NVMe/TCP) is a low-latency transfer protocol that allows you to use standard Ethernet TCP/IP networking equipment natively with NVMe storage.
TCP/IP is the default transfer protocol used by the internet by which messages are broken up into packets to avoid having to resend an entire message in the event of a disruption of service. As an extension of the NVMe-oF specification, NVMe/TCP allows you to send NVMe commands using the same TCP/IP protocol transfer packets you use to transmit other types of data.
The plug-and-play ease and lower cost of standard Ethernet makes it an economical solution for connecting your NVMe storage devices over a data fabric. Ethernet also provides a greater range of network speeds and queue paths for data transport when compared to traditional iSCSI.
NVMe Use Cases
NVMe technology is transforming many industries, including:
- Healthcare, where fast access to patient records is crucial for timely diagnosis and treatment. High-resolution imaging modalities such as MRI, CT scans, and digital pathology also require rapid storage and retrieval of large data sets.
- Finance, where high-frequency trading bots use ultra-low latency to execute trades in milliseconds and large-scale data analysis is used for risk assessment and decision-making.
- Cloud computing, where efficient management of virtual machines (VMs) and containers optimises resource utilization and reduces costs.
- Media and entertainment, where video editing and production requires rapid access to large video files and seamless playback.
- Gaming, which demands fast load times and smooth performance for a better user experience.
Basically, any industry that relies on quick data access and retrieval can benefit from NVMe.
Conclusion
Big data is no longer enough to maintain a competitive edge—it must also be fast.
How do you make big data fast?
You start in the server room. Transitioning from HDDs to SSDs is a good place to start, but it’s only one piece of the SAN puzzle. The transfer protocol, interconnects, and networking architecture also play important roles in the overall speed of your storage system. That means replacing legacy technologies like SAS and SATA with NVMe, which offers clear and significant advantages in throughput and latency.
Pure Storage leverages the increased transfer speeds of native NVMe transfer protocols to provide performant all-flash storage solutions.
The secret to Pure Storage performance is DirectFlash®:
- DirectFlash Fabric: Delivering performance close to DAS, DirectFlash Fabric offers enterprise-class reliability and data services.
All Pure Storage solutions leverage NVMe storage to unlock the full potential of flash memory. Pure Storage offers on-premises all-flash solutions for all your block, file, and object storage needs:
- FlashArray//X™: A performance-optimised all-flash array for Tier 0 and Tier 1 block storage applications
- FlashArray//XL™: Top tier performance at petabyte scale for your most demanding workloads
- FlashArray//C™: A capacity-optimised all-flash array for Tier 2 block storage applications
- FlashArray//E™: An always-on data repository that lowers your TCO by 40%
- FlashBlade//S™: An all-flash, scale-out, network attached storage (NAS) solution for file and object storage
- FlashBlade//E™: A unified file and object storage platform that delivers all-flash storage at a cost comparable to disk for everyday use
Additionally, Pure Storage offers a suite of software solutions that can simplify and unify storage management across your hybrid cloud:
- Evergreen®: Our portfolio of storage-as-a-service subscriptions allows you to non-disruptively upgrade your hardware with no downtime, no migrations, and no degradations in performance.
- Pure1®: Our AI-driven data services platform provides predictive analytics that help you catch bugs and address bottlenecks before they happen.
- Purity: Purity’s AI abstracts away the complexity of managing a data centre, providing you with a simple dashboard for complete control over your data.
- Portworx®: The most complete Kubernetes data services platform, Portworx provides a fully integrated solution for persistent storage, data protection, disaster recovery, and more for container workloads.
