Why did the Linux kernel replace the traditional single-queue block layer with blk-mq?

The legacy Linux block I/O layer (pre-3.13) was designed for mechanical hard drives (HDDs) operating at ~200 IOPS. It utilized a single global request_queue protected by a single global spinlock. Modern NVMe SSDs execute millions of IOPS and support up to 64,000 independent hardware queues. Under multi-core CPUs, threads spent up to 80% of their execution time spinning on the single block lock. The multi-queue block layer (blk-mq) introduced a two-tier queueing hierarchy that scales locklessly to millions of IOPS.

What is the two-tier architecture of blk-mq (Software Staging vs Hardware Dispatch)?

blk-mq divides block I/O into two tiers: 1) Software Staging Queues (struct blk_mq_ctx), instantiated per CPU core where bio requests are submitted, merged, and sorted without taking any cross-core locks; and 2) Hardware Dispatch Queues (struct blk_mq_hw_ctx), mapped to the actual underlying device submission queues (e.g. 1-to-1 or N-to-M mapping with NVMe hardware queues). Requests batch-dispatch from software queues directly to hardware queues.

How does sbitmap (scalable bitmap) eliminate tag allocation contention in blk-mq?

Every outstanding request in flight requires a unique hardware tag. Traditional tag allocators relied on atomic test-and-set loops over a single bitmap word, creating catastrophic cacheline bouncing across 64+ CPU cores. blk-mq uses sbitmap (scalable bitmap), which shards the bitmask across multiple cacheline-aligned words and per-CPU reservation caches, achieving O(1) lockless tag allocation.

What is the role of NVMe Doorbell registers in request submission and completion?

NVMe devices communicate through circular ring buffers in host RAM: Submission Queues (SQ) and Completion Queues (CQ). When the kernel writes a new request to an SQ, it notifies the NVMe controller by writing the new tail pointer index to the PCIe Memory-Mapped I/O (MMIO) Submission Queue Tail Doorbell register. When the device finishes the command, it writes a completion entry to the CQ and raises an MSI-X interrupt.

Linux Block Layer Multi-Queue (blk-mq) & NVMe Studio

Verified Specification | Standardized Formulas | Instant Precision

Secure & Private (Zero Data Retention) Free Access • No Sign-Up

Linux Block Layer Multi-Queue (blk-mq) Studio

Kernel storage architecture simulator modeling per-CPU software staging queues (blk_mq_ctx), hardware dispatch queues, lockless sbitmap tags, and NVMe doorbells.

Linux 3.13+ / 6.x

Block Layer Architecture

CPU Cores Generating I/O

Hardware NVMe Submission Queues

Target I/O Pressure (kIOPS)

100 kIOPS 1.20 MIOPS 3.0 MIOPS

Lock Contention Stalls

0.00 % (Lockless)

Per-CPU lockless staging queues

Hardware Tag Allocation

sbitmap (Scalable)

Per-CPU tag caches, O(1) bits

Achievable Throughput

1,200 kIOPS

Direct PCIe 4.0/5.0 line rate

Average I/O Latency

14.2 µs

Zero queue lock wait delay

Block Layer Queue Topology & NVMe Ring Dispatch ● Multi-Queue Lockless Pipeline

Kernel Queue Structure & Dispatch (C)

Performance Mathematics & Little's Law

Linux Block Layer Multi-Queue (blk-mq) Studio

🔍 Companion Workbenches & Verification Tools

Linux Memory Compaction & Transparent Huge Pages Studio: Buddy Allocator & Fragmentation

Kubernetes eBPF Service Mesh & Ambient Mesh Architecture Studio

Linux Kernel Slab Allocator & Free List Hardening Studio | Heap Exploitation Mitigations

AI Bot Blocker robots.txt Generator

Why Smart Users Are Switching to This Free Toolkit

The Browser Feature Most People Never Use

Stop Paying for Software You Can Get Free

File Conversion Tricks That Save Hours Every Week

This Free Tool Replaced 5 Paid Apps for Our Team

What Your Browser Can Do Without Any Extensions