NVMe frontend
This page is hw/femu/docs/design/nvme-frontend.md at FEMU d8b29b1e0 (2026-11-16), licensed GPL-2.0-or-later. Send corrections to the FEMU repository.
The NVMe frontend is the part of -device femu that a guest driver talks to:
the PCI function, the controller registers, the admin and I/O queues, the
poller threads that fetch and complete commands, and the interrupts. It turns
NVMe commands into calls to a mode (BBSSD, ZNS, KV and the others) and holds
each completion until the time the mode charged has passed.
This chapter describes the frontend as the code in this tree implements it. The design overview shows where it sits among the other components.
Place in the hierarchy
guest NVMe driver
| ^
MMIO writes | | CQEs and interrupts
DMA reads v |
+-------------------------------------------------------------------+
| NVMe FRONTEND (this chapter) |
| PCI function, BAR0 registers, doorbells, shadow doorbells |
| admin queue (vCPU thread) I/O queues (femu-poller threads) |
| dispatch, completion heap, interrupts, reset, teardown |
+-------------------------------------------------------------------+
| ns->ext_ops.io_cmd | to_ftl[i] ^ to_poller[i]
v v |
mode extensions FEMU-FTL-Thread (BBSSD, ZNS, CSD)
(zns.md, kvssd.md, ...) (ftl.md, nand-timing.md)
|
v
memory backend (data copies happen here, on the poller)
PCI function and BAR layout
The controller is a PCIe endpoint with class code 01 08 02 (NVMe). Its vendor
and device IDs come from vid and did (by default 1d1d:1f1f), and it
reports NVMe version 1.4. It has INTA, an MSI capability with up to 32
vectors at config offset 0x50, a PCIe capability at 0x80, and MSI-X with
queues + 1 vectors in a BAR of its own. nvme_init_pci() sets this up.
| BAR | Contents | When |
|---|---|---|
| 0-1 | controller registers and doorbells (64-bit) | always |
| 2-3 | Controller Memory Buffer (64-bit) | cmbsz non-zero; cmbloc must select BAR 2 |
| 4-5 | MSI-X table and pending bits | always |
BAR0, reg_size bytes
offset register FEMU behaviour
0x0000 CAP MQES = entries, CQR = cqr, AMS = 0, TO = 0xf, DSTRD = stride,
NSSRS = 0, CSS = NVM + I/O command sets by CSI,
MPSMIN = mpsmin, MPSMAX = mpsmax,
CRMS = CRWMS (not with OCSSD) read only
0x0008 VS 0x00020100 (2.1); 0x00010400 (1.4) with OCSSD read only
0x000c INTMS set interrupt mask bits (MSI and pin; ignored with MSI-X)
0x0010 INTMC clear interrupt mask bits; a held MSI is sent when unmasked
0x0014 CC EN, SHN, MPS, IOSQES, IOCQES, CSS; AMS must be 0 (round
robin); see the state machine below
0x001c CSTS RDY, CFS, SHST read only
0x0020 NSSR reads 0; writes ignored (no subsystem reset)
0x0024 AQA admin ASQS and ACQS, 1..4095 (0's based): 2..4096 entries
0x0028 ASQ admin SQ base, 64 bits, may be written as two dwords
0x0030 ACQ admin CQ base, 64 bits, may be written as two dwords
0x0038 CMBLOC cmbloc read only
0x003c CMBSZ cmbsz read only
0x0068 CRTO CRWMT = CAP.TO, CRIMT = 0 (reads 0 with OCSSD) read only
0x1000 doorbells: DB = 4 << stride bytes
0x1000 + (2 * qid) * DB SQ qid tail
0x1000 + (2 * qid + 1) * DB CQ qid head
qid 0 is the admin pair, 1 .. queues are I/O queues
reg_size = max(16 KiB, pow2ceil(0x1000 + 2 * (queues + 1) * DB))
BAR0 is at least 16 KiB because the PCIe transport makes bits 13:4 of the BAR
read only. Writes go to nvme_mmio_write(): offsets inside the register block
go to nvme_write_bar(), the admin doorbell pair to nvme_process_db_admin(),
and everything else to nvme_process_db_io(). All of them run on the vCPU
thread that made the access, holding the BQL.
The CMB is an I/O memory region (nvme_cmb_ops) backed by a host buffer of
the size cmbsz describes. PRP and SGL data pointers, and the entries of
PRP-list queues, that fall inside it are served from that buffer
(nvme_addr_is_cmb() in hw/femu/dma.c). A physically contiguous queue is
mapped directly and has no CMB case.
Controller states
realize
|
v
+-----------------------+ the host programs AQA, ASQ, ACQ
| DISABLED | and the CC fields while EN = 0
| CC.EN=0 CSTS.RDY=0 |<------------------------------------------+
+-----------+-----------+ |
| CC.EN 0 -> 1 |
v |
nvme_start_ctrl(): |
check MPS against CAP, IOSQES/IOCQES, AQA, ASQ/ACQ alignment |
check CC.CSS against CAP.CSS (000b and 110b are offered) |
map the admin CQ and SQ |
run each distinct mode's start_ctrl hook |
nvme_start_dataplane(): create pollers on the first enable, |
set dataplane_started |
| all ok | any check fails |
v v |
+-------------------+ +-------------------+ |
| READY RDY=1 | | FAILED CFS=1 |---- CC.EN 1 -> 0 --+
+----+---------+----+ +-------------------+ |
| | CC.SHN set (EN still 1) |
| v |
| nvme_clear_ctrl(shutdown), CSTS.SHST = complete |
| CC.EN 1 -> 0 |
v |
nvme_write_bar(), CC.EN 1 -> 0: |
nvme_clear_ctrl(): pause pollers and FTL thread; drop held AERs and |
queued events; drain and free every SQ and CQ; deassert the pin; |
reset features; unmap the shadow doorbell buffers |
then clear RDY, CFS and SHST, reset the Timestamp, and record a |
reset in the Persistent Event Log ------------------------------------+
A shutdown with CC.EN still 1 runs nvme_clear_ctrl() but leaves RDY set
and resets neither the Timestamp nor the Persistent Event Log context.
Pollers are created once and survive a reset; a reset only clears
dataplane_started until the next enable. With power_loss=on, a
shutdown of any kind (CC.SHN 01b, 10b or 11b) first flushes the BBSSD
write buffer: a completed shutdown leaves nothing the host was told is
written in the buffer. A failed BBSSD or KV flush sets CSTS.CFS and
leaves SHST at 00b.
The shutdown has its own state, apart from CC.SHN:
CC.SHN 00b -> nonzero, no shutdown yet --> flush, clear, SHST = 10b
(01b for shutdown_ms first)
CC.SHN changes again --> CC readback only
CC.SHN -> 00b --> SHST stays as it is
next CC.EN 0 -> 1, a controller reset, a power cut, a PCI or
system reset --> SHST = 00b
a power cut after SHST 10b: no unsafe shutdown, no new recovery
a failed flush: CFS, SHST 00b, and no shutdown until a reset or the
next enable, which also clears that CFS
CC.EN 0 -> 1 with a new SHN in the same write: no start, a shutdown
Queues
Data structures
| Structure | Fields that matter |
|---|---|
NvmeSQueue | sqid, cqid (the CQ it reports to), head, tail, size; dma_addr and dma_addr_hva (contiguous ring) or prp_list; io_req (one NvmeRequest per slot) and req_list (the free ones); db_addr/db_addr_hva and eventidx_addr/eventidx_addr_hva (shadow doorbell); is_active |
NvmeCQueue | cqid, head, tail, size, phase; vector, irq_enabled, virq and guest_notifier (irqfd route), irq_bh (fallback); sq_list; post_lock; the same four shadow doorbell fields; is_active |
NvmeRequest | cmd, cqe, sq, ns, status; slba, nlb, is_write, xfer_bytes; stime, expire_time, reqlat (ns); cxl_seq |
FemuCtrl | sq[], cq[] (index 0 is admin); nr_pollers, poller[]; to_ftl[], to_poller[], pq[], cpl_backlog[], should_isr, poller_in_sweep[], poller_ctr[] (all indexed by poller, 1-based); dataplane_started, ftl_in_sweep, use_ftl_thread |
A submission queue owns one request per slot, so it can have at most size
commands outstanding; when its free list is empty the poller leaves the rest
of the queue for a following sweep.
Admin queue
The admin queue pair is created from AQA, ASQ and ACQ when the controller is
enabled. It is processed synchronously: a write to the admin SQ tail doorbell
calls nvme_process_sq_admin() on the vCPU thread, which fetches each new
entry, runs nvme_admin_cmd(), writes the completion and notifies the admin
CQ's interrupt. It fetches only while the admin CQ has a free slot; a CQ head
doorbell write resumes it. The admin queue always uses its doorbell registers,
even after Doorbell Buffer Config.
Admin commands that free or replace state the I/O path reads (Delete SQ and
CQ, Format NVM, Namespace Management and Attachment, Sanitize, Streams, some
Set Features) first stop the pollers and the FTL thread with
nvme_pause_pollers() and restart them after (see
Quiescing the data plane).
I/O queues
The host creates I/O queues with Create I/O CQ and Create I/O SQ. Queue ids run
from 1 to queues, sizes from 2 to entries + 1. With cqr=1 (the default)
queues must be physically contiguous; with cqr=0 a queue may be a PRP list.
A completion queue's interrupt vector must be at most queues, and below the
number of MSI vectors the guest enabled when MSI is in use. Several SQs may
share one CQ.
A new SQ is published with a write barrier before is_active is set, and the
poller reads is_active with a read barrier before using the queue, so a
poller never sees a half-built queue. Delete I/O SQ pauses the data plane,
returns the queue's requests to it, and frees it. Delete I/O CQ is refused
while an SQ still reports to it.
Set Features Number of Queues reports queues for both SQs and CQs and is
accepted only before any I/O queue exists.
Doorbells and shadow doorbells
Doorbell registers
A doorbell write records the new value and returns. For an I/O SQ the vCPU
stores sq->tail; the poller notices it on its next sweep. For an I/O CQ it
stores cq->head, notifies the CQ again if entries remain, and on the pin
updates the interrupt level. A write to a doorbell that does not exist, or of
a value past the end of its queue, is ignored and, while the controller is
ready, raises an Error asynchronous event (event information 00h, write to an
invalid doorbell register, or 01h, invalid doorbell write value). An I/O
doorbell write that is not aligned to DB, and a register write to a queue
that has a shadow doorbell, are dropped without an event (for a shadowed CQ
the write only updates the pin level).
Doorbell Buffer Config
The controller always advertises Doorbell Buffer Config (OACS bit 8). The host
gives two page-aligned pages of its memory with this admin command: a shadow
doorbell page, where it writes tails and heads instead of the registers, and
an EventIdx page, where the controller writes back hints.
nvme_set_db_memory() maps both pages and points every existing I/O queue at
its two entries:
shadow doorbell page (host writes) EventIdx page (controller writes)
entry size DB = 4 << stride bytes same layout
offset +------------------+ +------------------+
0 * DB | SQ0 tail (unused)| | SQ0 EventIdx | admin pair: kept
1 * DB | CQ0 head (unused)| | CQ0 EventIdx | equal to the value
2 * DB | SQ1 tail | | SQ1 EventIdx | just written
3 * DB | CQ1 head | | CQ1 EventIdx |
... | ... | | ... |
2q * DB | SQq tail | | SQq EventIdx |
(2q + 1) * DB | CQq head | | CQq EventIdx |
+------------------+ +------------------+
Rules the code enforces:
- Both addresses must be non-zero and page aligned, and both pages must map completely; otherwise the command fails with Invalid Field and nothing is recorded.
- The pages are set once per enable. A second command before a reset fails.
- All entries must fit in one page: (2 x
queues+ 1) x DB bytes at most. With a 4 KiB page andstride=0that is 511 I/O queues. - Each queue that already exists is seeded with its current tail or head before it is switched to its entry, so a poller never reads a stale zero. A queue created afterwards starts at 0 and points at its entry at once, relying on the host's zeroed entry.
- A controller reset unmaps both pages and forgets them.
Once a queue has a shadow doorbell, its register doorbell no longer moves it.
The poller reads the SQ tail from the shadow at the start of every sweep
(nvme_update_sq_tail()) and the CQ head whenever it checks for space or
posts an interrupt (nvme_update_cq_head()). A value past the end of the
queue is ignored.
EventIdx
With shadow doorbells the host rings the real doorbell only when its new value passes the EventIdx the controller published. FEMU's pollers read the shadow values of every active I/O queue on every sweep, so I/O queues do not depend on doorbell writes. Two queues are handled differently:
- The admin pair keeps its EventIdx equal to the value just written, so a host that follows the EventIdx protocol still rings it (the admin queue is driven only by its registers).
- Pin interrupts. With neither MSI-X nor MSI enabled, the CQ EventIdx is the head itself. The pin is level triggered and can only drop after the host reports its new head, so the host must ring the CQ head doorbell after every batch; the register write then updates the pin level.
Pollers
How many and who owns what
nvme_init_poller() creates the pollers the first time the host enables the
controller. Their number depends on two properties:
multipoller_enabled | Pollers | Poller i of N owns |
|---|---|---|
0 (default) | 1 | every I/O SQ |
1 | ceil(queues / poller_ratio); poller_ratio=0 counts as 1 | SQs i, i + N, i + 2N, ... |
| anything else | realize fails |
multipoller_enabled=0 multipoller_enabled=1, queues=8, poller_ratio=3
N = ceil(8 / 3) = 3
femu-poller 1: SQ1 SQ2 ... SQ8 femu-poller 1: SQ1 SQ4 SQ7
femu-poller 2: SQ2 SQ5 SQ8
femu-poller 3: SQ3 SQ6
per poller i: to_ftl[i] ring, poller i -> FTL thread, holds 65535
to_poller[i] ring, FTL thread -> poller i, holds 65535
pq[i] min-heap on expire_time, starts at 65536, grows
cpl_backlog[i] completions waiting for CQ space
should_isr row which CQs to interrupt after this sweep
poller_in_sweep[i], poller_ctr[i]
-device femu,femu_mode=2,queues=8,multipoller_enabled=1,poller_ratio=3
Every SQ has exactly one owner because a poller fetches from it without a
lock. Values of multipoller_enabled greater than 1 used to start one poller
per shard but had each of them sweep every queue; two pollers then fetched,
executed and completed the same command. Realize now refuses them.
A CQ is not owned. Its SQs may belong to different pollers, so with more than
one poller a completion is posted under the CQ's post_lock, and each poller
keeps its own row of should_isr flags.
The sweep
nvme_poller(i), forever until poller_stopping:
if !dataplane_started: in_sweep = false, sleep 1 ms, retry
in_sweep = true; smp_mb; recheck dataplane_started
for each SQ this poller owns, active and with an active CQ:
nvme_process_sq_io(sq, i) fetch and execute new commands
nvme_process_cq_cpl(i) time and post completions
smp_mb; in_sweep = false
A poller never sleeps while the data plane runs: it spins on a host core. Plan one core per poller and one for the FTL thread (performance tuning).
Data copies and write atomicity
Identify reports AWUN 0, so a one-block write must reach the media whole
with respect to every other command. With atomic_write_lbas = N, each
namespace reports NAWUN and NABSN as N - 1. The same locks cover each
whole command, so a write of up to N blocks inside one N-block boundary
is atomic too. NAWUPF stays 0: a power loss can still tear a buffered
write, so power_loss refuses the property. With more than one poller, two commands
on the same blocks copy at the same time. Each access to the memory backend
therefore holds the locks of the 4 KiB granules it touches. The granules share
1024 locks: granule g takes lock g % 1024.
poller 1: Write LBA 0 ---+ +--> memcpy into backend
| lock granules of |
poller 2: Read LBA 0 ---+-> [off, off + len) --+--> memcpy out of backend
in index order (waits for the write)
- Every path that moves block data takes the locks:
- Read, Write and Zone Append (
backend_rw()); - the NoSSD inline path;
- Copy, for each source range and for the destination;
- Compare;
- Write Zeroes and deallocate;
- the protection information and metadata paths;
- the power-loss write cache;
- CSD memory reads.
- Read, Write and Zone Append (
- Read, Write and Compare hold the locks of their whole range, so a multi-block Read or Write is also atomic. OCSSD takes the locks per vector entry.
- Copy takes one source range at a time, then the destination. Its write to the destination is atomic, but a Write between two of its source reads can leave old data from one range and new data from the next. Deallocate and Write Zeroes also take one range at a time.
- A bbssd controller linked to a
femu-cxl-ssdmedium (cxl_ssd=) brings the locks to the medium's backend. CXL.mem loads and stores, and pages mapped straight into the guest, do not take them. - The data-remanence debug output (
FEMU_EXP_LOG,FEMU_SECRET,FEMU_DUMP_LPN) reads the backend without the locks. - The tests
rw-atomic-split-writeandrw-atomic-split-copystop a write halfway through its copy (x-copy-hold, qtest builds only) and read the block meanwhile. The write goes on oncex-range-waitsshows the read waiting for its lock. Without the locks the read returns half old, half new data. - Locks are taken in index order, so two commands never wait in a cycle. A
namespace with metadata takes its
mdata_lockfirst. - Two commands on different blocks wait for each other only when their granules share a lock. For random 4 KiB commands that happens once in 1024.
Command lifecycle
SQ slot in guest memory (or the CMB)
| nvme_process_sq_io(): copy the 64-byte SQE, take a free NvmeRequest,
| stime = expire_time = now (QEMU_CLOCK_REALTIME)
v
nvme_io_cmd()
| nsid -> nvme_ns(): allocated and attached, else Invalid Namespace
| Flush, Dataset Management, Compare, Write Zeroes, Copy, Verify,
| Write Uncorrectable, I/O Management Send/Receive: handled here
| any other opcode -> ns->ext_ops.io_cmd() of the namespace's mode
| NoSSD, BBSSD, CSD Read/Write: nvme_rw() maps PRP/SGL, copies data
| ZNS, OCSSD, KV: their own paths; OCSSD and KV add NAND time now
| status stored; on success the SMART host counters are updated
v
to_ftl[i] (every request, failed ones included)
|
+-- controller has an FTL thread: it dequeues, calls
| femu_ftl_process_req(), adds the latency to expire_time,
| enqueues on to_poller[i]
+-- no FTL thread: the poller reads to_ftl[i] itself
v
nvme_process_cq_cpl(i)
| add host-link time (Read, Write) and firmware-CPU time
| (Read, Write, Zone Append), insert into pq[i]
| retry cpl_backlog[i]
| while the heap's earliest request is due (now >= expire_time)
| and, for a CXL-linked request, the medium has dropped stale pages:
| CQ inactive -> request back to the free list
| CQ full -> cpl_backlog[i]
| otherwise -> post CQE
v
request back on sq->req_list; should_isr[cqid] = true
end of sweep: for each flagged CQ, raise its interrupt
Things to note:
- Data moves before time is charged. By the time a request reaches
to_ftl[i], a Read's data is already in guest memory and a Write's data is already in the backend. The FTL thread only computes time. (Withpower_loss=onthe poller skipsnvme_io_cmd()and the FTL thread executes every I/O command, data copy included, throughnvme_power_io().) - Failed commands take the same path. A request with an error status is still enqueued; the mode charges it nothing, and it completes as soon as the poller sees it again (in the same sweep when there is no FTL thread), plus any host-link or firmware-CPU time, which is charged regardless of status. Skipping the ring would leak the request.
- Order. Completions are posted in
expire_timeorder per poller, not in submission order. - Full completion queues. A due completion whose CQ has no free slot waits in the poller's backlog, and the poller retries it every sweep.
- Abort looks for the command in the SQ between head and tail with the
pollers paused. If it is there, the controller records it in its own state
(the SQ itself is never written) and the Abort completes with dword 0 bit 0
clear; the command then completes with Command Abort Requested when it is
fetched, without running. A command already fetched is not aborted, and the
Abort completes with bit 0 set. Admin commands run one at a time, so the
Aborts outstanding together are the one running and those still in the
admin SQ; with more than
aclof them waiting, the running one fails with Abort Command Limit Exceeded. A command queued behind an Abort on the admin queue can be aborted too. - Media errors carry DNR. Unrecovered Read Error (a block marked by Write Uncorrectable, or an injected read error) and Compare Failure always set Do Not Retry, on every command that can return them: a retry would fail again.
- Admin commands use PRPs. An admin command whose PSDT is not 00b fails with Invalid Field in Command.
Completion timing
nvme_process_cq_cpl() is where FEMU enforces latency. A request's
expire_time starts at its fetch time and grows as each model adds to it:
| Added by | When | Property |
|---|---|---|
| OCSSD, KV, CSD compute units | while the command executes on the poller | mode properties |
| FTL thread (BBSSD, ZNS, CSD NAND time) | before to_poller[i] | FTL and NAND properties |
| host link | in nvme_process_cq_cpl(); a per-direction next-free time | pcie_bandwidth_mbps, pcie_prop_delay_ns |
| firmware CPU | in nvme_process_cq_cpl(); one next-free time per firmware core (nvme_fw_cost() picks the time per opcode) | fw_cpu_ns, fw_read_ns, fw_write_ns, fw_other_ns, fw_cores |
The heap pq[i] is a binary heap (hw/femu/lib/pqueue.c) keyed on
expire_time. Each sweep pops every request that is due and stops at the
first that is not. Nothing sleeps: a request simply stays in the heap until a
sweep finds it due. The guest therefore sees at least the charged time, plus
however long the poller takes to come round.
The clock is QEMU_CLOCK_REALTIME. Only the qtest-only property
x-oc12-clock switches it to the virtual clock, for deterministic tests.
Interrupts
Each I/O CQ created with interrupts enabled gets its vector. After a sweep
that posted to a CQ through the heap, the poller calls nvme_isr_notify_io()
once for that CQ:
nvme_isr_notify_io(cq) (poller thread, no BQL)
|
+-- cq->virq > 0 (MSI-X under KVM, route added at Create I/O CQ):
| event_notifier_set() -> KVM irqfd injects the MSI-X vector
|
+-- otherwise: qemu_bh_schedule(cq->irq_bh) (main loop, BQL)
nvme_isr_notify_legacy():
MSI-X enabled -> msix_notify(vector)
MSI enabled -> msi_notify(vector), or hold the bit in
irq_status while INTMS masks it
neither (pin) -> nvme_irq_update(): assert INTA while any
enabled CQ has head != tail and INTMS allows
The admin CQ is notified on the vCPU thread through the same legacy path.
MSI-X mask, unmask and pending-bit polling are handled by vector notifiers
(nvme_vector_mask(), nvme_vector_unmask(), nvme_vector_poll()) that move
the irqfd routes.
On the heap path a poller raises at most one interrupt per CQ per sweep.
Interrupt coalescing
Interrupt Coalescing (Feature 08h) holds the interrupt of an I/O completion queue's vector until enough entries wait, or until the oldest waits too long. The threshold THR (0's based entries) and the time TIME (100 us steps) both must be non-zero, or coalescing is off. The threshold counts per interrupt vector, so queues that share a vector share one count. Coalescing Disable in Feature 09h turns it off for one vector.
These are never held: the admin queue, completions with an error, and the
pin. The pin is a level the controller recomputes from the queues, and the
threshold is only a recommendation. intc_thresh, intc_time and intc
set the reset values.
entry posted: vector pending++; first entry: first_ns = now; error: urgent
end of a sweep, CQs it posted to: off, urgent, pending >= THR + 1,
and every sweep, vectors in its share: or now - first_ns >= TIME x 100 us
--> interrupt, pending = 0
Set Features 08h or 09h: pause the pollers, send what is held, apply
Each vector is checked for its time by one poller. A CQ head doorbell write that leaves entries behind still interrupts at once, which the spec allows, and it clears the vector's count. With Feature 08h at 0h nothing is counted.
A Set Features 08h or 09h, and the delete of a queue whose vector holds entries or carries other queues, signal the vector at once on the thread that holds the BQL. They do not use the queue's bottom half or eventfd, which a delete would close. A controller reset sends nothing. The rule is that an interrupt may come early but is never lost. One extra interrupt can follow when an admin completion on vector 0 races a poller that has just posted an I/O entry; the host then finds no new entry.
Quiescing the data plane
Admin commands, controller reset and some features must change state the
pollers and the FTL thread read without locks. nvme_pause_pollers() waits
until neither is inside a sweep:
caller (vCPU, BQL) poller i FTL thread
dataplane_started = false in_sweep[i] = true ftl_in_sweep = true
smp_mb smp_mb smp_mb
wait while any in_sweep[i] dataplane_started? no: dataplane_started? no:
or ftl_in_sweep is true in_sweep[i] = false, back off ftl_in_sweep = false, skip
... change queues or namespaces ... ... sweep ... ... one request ...
nvme_resume_pollers(): smp_mb ftl_in_sweep = false
dataplane_started = true in_sweep[i] = false
Either the caller sees a sweep in progress and waits for it, or the thread
sees the cleared flag and backs off. With shared namespaces
(femu-subsys,ns_mgmt=on) the pause covers every controller in the
subsystem, and a nested pause does not resume its caller's work.
Teardown
femu_exit() runs when the device is unplugged (device_del). QEMU does not
call it when the process exits. It stops the threads before freeing anything
they read:
- Set
poller_stoppingand join every poller. Pollers go first because they feed the FTL thread. - Set
ftl_stoppingand join the FTL thread. - Close the Persistent Event Log, detach a linked CXL medium.
- Run each distinct mode's
exithook once (not for a controller on a shared subsystem's storage). nvme_clear_ctrl(shutdown), then free the rings, heaps and per-poller arrays, the AER bottom half and the memory backend (unless it is shared or lent by a CXL medium).
Commands, namespaces and modes
Dispatch into modes
A mode is a FemuExtCtrlOps table (hw/femu/nvme.h):
| Hook | Called from | For |
|---|---|---|
init | realize, once per namespace of that mode, and Namespace Management create | build the mode's state for the namespace |
init_ctrl_name | realize, only with ns_mgmt (otherwise each mode's init sets the name); only a namespace of the controller's femu_mode names it, and the controller's own table names it when no namespace does | Identify model number and serial |
start_ctrl | controller enable, once per distinct mode | refuse settings the mode cannot serve; derive values from CC |
io_cmd | poller, for opcodes the frontend does not handle | the mode's I/O commands |
admin_cmd, admin_cmd_cqe | vCPU, for admin opcodes the frontend does not handle | mode admin commands, such as the Open-Channel commands, BBSSD 0xEF and the CSD commands |
get_log | vCPU, for log ids the frontend does not handle | OC 2.0 chunk information, the ZNS Changed Zone List |
rw_check_req | not called through the table | OC 2.0 sets it and calls its checker directly |
exit, ns_exit | teardown, namespace deletion | free the mode's state |
nvme_register_extensions() installs the table for the controller's
femu_mode, and nvme_register_extensions_ns() gives each namespace the table
of its own mode, so I/O dispatches on the namespace, not the controller.
Admin commands go to the controller's table; Get Log Page tries the named
namespace's table first, then the controller's.
Capability registry
hw/femu/nvme-caps.c is the one place that says what the controller
handles. nvme_admin_effects() and nvme_io_effects() return the Commands
Supported and Effects entry of an admin opcode, or of an I/O opcode in one
command set, and zero for an opcode that is not handled.
nvme_log_support() returns the Supported Log Pages entry of a log id in one
command set.
An I/O opcode's entry for a command set is what the namespaces of that set
handle between them (nvme_ns_io_effects() for one namespace): Flush, the
optional commands and I/O Management, which nvme_io_cmd() serves itself,
and then the mode's own table, so log 05h lists the Open-Channel vector
commands, the CSD commands and the key value commands of the modes present,
and leaves Read and Write out for Open-Channel 1.2, which refuses them. A
command set with no namespace reports what a namespace of it would handle.
The admin entries add the commands of the controller's mode: BBSSD 0xEF, the
Open-Channel and CSD admin commands. nvme_caps_id_ctrl() fills OACS, ONCS, OCFS, LPA and SANICAP
from those three each time Identify Controller is answered. LPA bit 5 stays
clear: setting it would oblige a Command Scope in every log 05h entry, and
Linux warns about any I/O entry bit beyond CSUPP and LBCC as an unusual
effect.
Identify CNS 06h reports the version of each command set: NVM 1.1 in bytes 23:20 (CSI 00h) and ZNS 1.2 in bytes 35:32 (CSI 02h). The NVM structure also sets AOCS bit 0, as Get LBA Status serves action type 02h. KV reports 1.3, and SLM and CP 1.2. An Open-Channel controller keeps its 1.4 replies: the NVM and ZNS VER fields and AOCS stay 0, and CSI 01h still reports KV 1.3.
Logs 05h and 00h are built from the same functions, and nvme_io_cmd() runs
an optional NVM command (Compare, Dataset Management, Write Zeroes, Copy,
Verify, Write Uncorrectable) only where nvme_io_effects() lists it for the
namespace's command set. Features follow the same rule through
nvme_fid_supported(), which Get Features, Set Features and log 12h share.
The caps-* qtests check the registry against dispatch. Vendor command 0xEE
answers Invalid Command Opcode outside Open-Channel, as any unlisted admin
opcode does. I/O Management Send and Receive answer Invalid Command
Opcode where log 05h does not list them: without FDP, and on namespaces of
other command sets. Two differences are outside the test's reach. Get
Features serves the Select field while ONCS bit 4 is clear. CNS 1Ch always
lists the NVM, zoned and key value sets.
Namespace routing
nvme_ns() maps an NSID to a namespace that is both allocated and attached
to this controller. NSIDs run from 1 to namespaces (or to 256 with
Namespace Management). An unknown NSID fails with Invalid Namespace, and with
Namespace Management an unallocated or detached NSID within range fails with
Invalid Field. A Flush to every namespace (NSID FFFFFFFFh) is refused, which
Identify reports as Flush Behavior 10b.
The FTL thread routes each request by its namespace: a namespace borrowed
from a linked CXL medium goes to the medium's FTL; with power_loss=on the
command is first executed; then ZNS namespaces go to zns_ftl_process_req(),
BBSSD and CSD namespaces to bb_ftl_process_req(), and others charge
nothing. One FTL thread serves all of them, and it exists only
when a namespace needs it. With shared namespaces, the subsystem's ns_lock
is held around command execution and FTL processing.
namespaces.md covers sizing, per-namespace modes and
Namespace Management.
Log pages and features at a glance
| Log page | Id | Notes |
|---|---|---|
| Supported Log Pages | 00h | lists the ids the controller advertises (04h only with Namespace Management) |
| Error Information | 01h | newest elpe + 1 entries; each names the command set (byte 30) and opcode (byte 31) of its command, and every entry, an empty one too, has Log Page Version 1h (byte 63); Open-Channel entries keep the 1.4 layout |
| SMART / Health | 02h | host totals from the pollers, media wear, temperature |
| Firmware Slot | 03h | slot 1 read-only; with fw_slots above 1, AFI shows the running slot and the slot for the next reset, and FRS2 to FRS7 the committed images; a read without RAE clears the Firmware Activation Starting notice |
| Changed Namespace List | 04h | namespaces whose attributes changed |
| Commands Supported and Effects | 05h | per command set (NVM, zoned, KV) |
| Device Self-test | 06h | tests complete at once |
| Telemetry Host / Controller | 07h, 08h | 07h: header plus the C0h counters captured by the last Create; 08h: the counters captured when a SMART warning, the temperature condition or thermal protection begins, kept until a read without RAE (header only with ocp=on) |
| Endurance Group | 09h | with femu-subsys; Identify reports the group (CTRATT bit 4, ENDGIDMAX, ENDGID, and the Endurance Group List, CNS 19h) with it |
| Asymmetric Namespace Access | 0Ch | with ana=on: one group per namespace; the state comes from the runtime property ana-state; RGO leaves out the namespace lists |
| Persistent Event | 0Dh | kept in pel_file if set |
| OCP Hardware Component | C6h | with ocp=on: the emulated controller, NAND, DRAM, PMIC and board, then the serial number, the country of origin (not available), the hardware revision and the born-on date, the day the device was realized |
| OCP Latency Monitor | C3h | with ocp=on: 512 bytes, version 0005h, configured by Feature C5h; bytes 512 to 4095 read as zeros |
| OCP SMART / Health Extended, Error Recovery | C0h, C1h | with ocp=on: OCP 2.7 layouts at UUID index 0 and 1 (CNS 17h lists the OCP UUID, then the FEMU UUID 2ff92965-47be-4ce0-923d-649d30079203); the FEMU counters at C0h, UUID index 2. Fields FEMU does not model are 0, capacitor health is FFFFh (no PLP) |
| Rotational Media Information | 16h | with femu-subsys,rotational=on: one actuator, one spinup and one load per QEMU run; Spinup Control (Feature 1Ah) is fixed off |
| LBA Status Information | 0Eh | one element per attached namespace with tracked blocks; see LBA Status Information |
| Feature Identifiers Supported and Effects | 12h | built from the check Get and Set Features use; per command set (CSI) when CC.CSS is 110b |
| NVMe-MI Commands Supported and Effects | 13h | all zero: no NVMe-MI Send or Receive |
| Command and Feature Lockdown | 14h | with lockdown=on: what the Lockdown command (24h) can prohibit, and what the admin queue refuses now; no Management Endpoint, so Contents 10b is Invalid Field |
| FDP Configurations, RUH Usage, Statistics, Events | 20h-23h | with FDP |
| Reservation Notification | 80h | with reservations=on: the oldest notice of the queue (32 pages), which the read removes; an empty queue reads as zeroes |
| Sanitize Status | 81h | SPROG while a background sanitize (sanitize_ms) runs, estimated times, and a read without RAE clears Sanitize Operation Completed |
| Changed Zone List | BFh | ZNS, from the mode |
| Chunk Information | CAh | Open-Channel 2.0, from the mode |
| FEMU media counters | C0h | WAF and FTL counters |
nvme_log_csi() gives the command set of a Get Log Page. It is CDW14.CSI
when CC.CSS is 110b, and NVM when CC.CSS is 000b (Base 2.3, Figure 205).
Get Log Page refuses an id that log 00h does not list for that command
set with Invalid Log Page. So the Changed Zone List (BFh) answers only for
the zoned set, and the Open-Channel 2.0 Chunk Information (CAh) only for
NVM. When CC.CSS selects by CSI, logs 00h, 05h, 12h and BFh refuse an
unknown CSI with I/O Command Set Not Supported.
With FDP off there is no FDP capability, so the FDP pages are neither
listed nor answered.
Get and Set Features answer Arbitration (01h), Power Management (02h, one
power state, or four with power_states=on, which adds Autonomous Power
State Transition, 0Ch), LBA Range Type (03h), Temperature Threshold (04h), Error
Recovery (05h), Volatile Write Cache (06h, only with vwc=1), Number of
Queues (07h), Interrupt Coalescing (08h), Interrupt Vector Configuration
(09h), Write Atomicity (0Ah), Asynchronous Event Configuration (0Bh),
Timestamp (0Eh), Read Recovery Level (12h, only with rrl=on), LBA
Status Information Attributes (15h, NVM and ZNS), Host Behavior Support
(16h), Command Set Profile (19h), FDP (1Dh, 1Eh, only with FDP on), Key
Value Configuration (20h, only with a KV namespace) and Software Progress
Marker (80h). Any other identifier, or one of these without what it needs,
fails with Invalid Field in Command, and log 12h lists exactly the
identifiers that answer.
LBA Status Information Attributes (NVM 1.2, 4.1.3.5) return LSIPI in bits 31:16 and LSIRI in bits 15:0, both in 100 ms units. LSIPI is fixed at 10 (1 s). A Set takes LSIRI from bits 15:0; a value below 10 becomes 10, the closest supported value. The feature is not saveable, and a controller reset sets LSIRI back to 10.
LBA Status Information (log 0Eh)
The tracked blocks are the uncorrectable map of each namespace. Write Uncorrectable sets them. A write, a Write Zeroes, a Format or a Sanitize clears them.
The log has one element for each attached namespace with a tracked block. An element lists the runs as LBA Range Descriptors with RATYPE 11h. A run longer than 2^32 blocks takes several descriptors. More than 256 descriptors give NLRD FFFFFFFFh and no list. ESTULB is the sum of the tracked blocks, and saturates at FFFFFFFEh.
first read after a clear --> capture (pollers paused): page S, gen G_S
read OK, any RAE --> R = what S listed, G_R = G_S
RAE=1 --> S stays: later reads return the same bytes
RAE=0 --> S goes: the next read captures again
gen with no S = max(G_pub, G_R + 1 if a tracked block is
not in R) G_pub: highest gen given out
- LSGC (Figure 109) steps only when the page lists a block that the last host read did not list. A repair alone, or a repeat Write Uncorrectable of a listed block, does not step it.
- Take an element that the last read gave with FFFFFFFFh descriptors. The next page is new when it lists descriptors, or when a block was added since. Figure 109 compares descriptors only; this is the FEMU rule for an element with none.
- A failed transfer or a bad offset changes nothing. A controller reset drops the held page and keeps R, so the same blocks do not step LSGC again.
The alert is an AER of type Notice, information 05h, log 0Eh (AEC bit 13, OAES bit 13). FEMU sends it when the generation is above the last one it announced:
new block --> wait: LSIRI after the last clear --> scan (at most 1 per 100 ms)
--> queue Notice 05h --> AER --> capture S, check its gen --> post
component failure --> no LSIRI wait (also with no tracked blocks)
posted --> no more 05h until a read with RAE=0 clears it
- The page is captured when the alert is posted, so the alert and the page agree. A block added after that waits for the next alert.
- A full event queue refuses the alert. It goes in when an event leaves the queue. AEC bit 13 off drops a queued alert.
- A controller reset keeps the generation the host was told. An alert the host did not read is sent again, one LSIRI after the next enable; a cleared one is not.
persist_filekeeps the generations, the last alert, a pending component failure, and the last host read. A restart then acts as a reset: an alert the host did not read comes back one LSIRI after the enable.- Format and Sanitize clear the blocks with no LSGC step. After a Format to the same LBA size, the last host read still covers its blocks; after a new LBA size it covers none. A namespace created again with the same NSID is a new namespace for LSGC.
- LSGC is 16 bits and wraps from FFFFh to 0.
With power_states=on and power_limit=on, Power Limit (23h, Base 2.3
8.1.18.6) takes a limit in CDW13 (PLS 01b or 10b, PLV). The states whose
MP is above it are prohibited. MP falls with the state number, so these
are always the first states; the rest are renumbered from 0.
ps_max_watts 15: PS0 15 W PS1 9 W PS2 5.25 W PS3 0.05 W (non-op)
limit prohibited host PS0.. NPSS
none, >= 15 W none PS0 3
9 to 14.99 W PS0 PS1 2
5.25 to 8.99 W PS0, PS1 PS2 1
below 5.25 W Invalid Power Limit (0x13e); nothing changes
- Identify reports CTRATT bit 20, MUPS 10b and MUP (the MP of PS0 without a limit). PSD slots past NPSS read 0, and the relative ranks are renumbered with the states.
- The controller stays in the state it is in. A limit that prohibits it moves the controller to the new PS0. A wake from PS3 goes to the last operational state, or to the new PS0 if that state is prohibited.
- The APST table is renumbered too. When the limit first comes, FEMU keeps the table; a higher limit or none brings it back, cut again for the states that stay prohibited (8.1.18.6.2).
- 23h is saveable (ONCS bit 4 set). The default is PLS 10b, PLV 0 (no
limit). A reset and a power cut take the saved value; it lives in
memory, so
persist_fileis refused. - The MP figures are nominal. The limit restricts the states; it does not cap the energy that the thermal model and log 25h count.
- Not with
ocp=on: OCP 2.7 NVMe-OPT-21 forbids 23h, andpower_statesis refused withocpalready.
With ns_wp=on, Namespace Write Protection (84h, Base 2.3 8.1.17) takes
the No Write Protect (0) and Write Protect (1) states of a namespace, or
of all for NSID FFFFFFFFh. The other two states need an RPMB, which FEMU
does not have, so they fail with Feature Not Changeable.
Set 84h, WPS 1: refuse new writes --> wait for fetched requests to reach
the FTL (1 s at most, else Command Interrupted, state unchanged)
--> destage the namespace's write buffer --> release its Streams
I/O to a protected namespace (Figure 671):
Read, Compare, Verify, Reservations, DSM without AD as before
Flush success, no effect
Write, Write Zeroes, Write Uncorrectable, DSM with AD,
Copy into it, I/O Management, others 20h
Admin: Format NVM or Sanitize that covers it, Get LBA Status on it,
Streams Allocate Resources on it 20h
- Identify reports NWPC bit 0, and NSATTR bit 0 (CWP) for a protected namespace in CNS 00h and 08h. SMART does not report the media as read-only.
- A reset and a power cut keep the state. Default and Saved read the current state, since the spec gives 84h no default.
- A Copy from a protected namespace into another one works.
- bbssd and NoSSD namespaces only. Not with FDP (its namespaces share
one write buffer),
ns_mgmt, shared namespaces,cxl_ssdorpersist_file. - Logs 12h and 05h report NCC and NIC for 84h and Set Features: one Set can change CWP of several namespaces.
With sanitize_ops=on and sanitize_ns=on, Sanitize Namespace (8Ch,
Base 2.3 5.2.23) runs a Crypto Erase of one namespace. One runs at a
time (MNSOIP 1).
8Ch, NSID n, SANACT 100b, dataplane paused:
refuse n's new commands --> mark n's queued commands and retire its
unfinished ones (each completes with 2Ch) --> erase n: backend slice,
metadata, bitmaps, FTL mappings, buffered pages --> SOS 010b
sanitize_ms later, or at once: SOS 001b, NDE --> event (log 81h, NSID)
while it runs: I/O to n but Flush, a Copy from n, Format of n or of
all, Get LBA Status of n --> Sanitize Namespace In Progress (2Ch)
- The buffered pages are dropped, not written back, so the operation cannot fail. Work that finished before the start keeps its status.
- NDE clears before a write moves data, so a write that fails part way also clears it.
- Log 81h with NSID n reports that namespace: its SOS, NDE (bit 10), the subsystem's GDE, STNSID n. NSID 0 or FFFFFFFFh reports the subsystem, with MNSOIP 1.
- A second namespace sanitize fails with SCT 1, SC 12h (Figure 392; Figure 103 lists 3Ch for the same status). A Sanitize (84h) fails with 2Ch while one runs, and 8Ch fails with 1Dh during a Sanitize.
- NDE is set for a new namespace and by a sanitize, and cleared by a
write. The state lives in memory:
persist_fileis refused. - Admin commands while it runs (Figures 144 and 145) fail with 2Ch: Firmware Commit and Download and Lockdown always; Device Self-test and the Directives for n or for all; Get Log Page for n, except log 81h; Get or Set Features of a namespace-scoped feature for n, and Set for all.
- Log 7Fh lists NSIDs: NLT 0 the namespaces in progress, NLT 1 the idle ones with NDE set. GENCTR counts every change of either list.
- Not with a femu-subsys,
ns_mgmtorcxl_ssd; bbssd and NoSSD only.
With cancel=on, Cancel (I/O 18h, Base 2.3 7.1) aborts commands of its
own submission queue: one by CID (ACODE 00b) or all of a namespace
(ACODE 01b, NSID FFFFFFFFh for all). Dword 0 of its completion holds
CEDA (bits 31:16) and CMDA (bits 15:0).
poller: fetch 18h --> field checks --> fail: CQE without the FTL
--> pass: held for a bottom half
bottom half, dataplane paused, on the poller that owns the queue:
past the FTL step (return ring, pq, backlog), or a held Cancel
--> Command Abort Requested now; CMDA + 1
before the FTL step (to_ftl)
--> the step runs, then Command Abort Requested; CEDA + 1
Cancel CQE
- A command past its FTL step has moved its data and changed the mapping, so only its CQE is left: the abort leaves no effect after the Cancel completes (5.2.1.1). A command that already failed keeps its status.
- A single cancel finds the CID first. If that command names another namespace, the Cancel fails with Invalid Field and changes nothing. A CID that is not found gives CEDA 0 and CMDA 0. The Cancel's own CID gives status 184h.
- A held Cancel can be aborted; a Cancel that ran keeps its effects.
- A namespace retirement (sanitize, detach, delete) wins over a deferred abort, so CEDA counts the commands eligible for one.
- Needs
mapping=page. Not withbuffer_size,cxl_ssd,ocp,power_lossor a femu-subsys: each leaves work after the FTL step, or more than one controller.
With ocp=on the OCP 2.7 features answer through UUID index 0 or 1 (OCP
4.15). C5h configures the latency monitor, C7h selects a DSSD power state
and C9h configures the DSSD events. C6h, C8h and CAh hold one value each,
with a current, default and saved value. C2h exists with DP-CFG-6 A
only, and picks what a failed PLP does (plp=on, see
FTL). C6h sets how often the PLP
check runs. FEMU models no idle mode. DP-CFG-13 prohibits CBh, so it
fails.
FID name CDW11 bits default note
C2h PLP failure mode 31:30 01b ROM; 10b WTM, 11b normal
C3h clear PCIe error counters 31 - not saveable; reads 0
C6h PLP health check interval 31:16 0 minutes; 15 with plp
C8h telemetry profile 7:0 0 saved without SV
CAh idle wake up time 15:0 0 ms
The NVM subsystem features (C0h, C6h, C7h, C8h, CAh) take NSID 0 or FFFFFFFFh only.
C0h is OCP Event Injection (4.15.3). FEMU supports one event type, 0016h
NAND Die Offline, on a controller with a bbssd or CSD namespace; any other
type fails with Invalid Field. The entry takes NRTDE, the number of
Read commands that complete before the event, with any status. NUME 0 clears the event, and
Get Features returns NUME 1 while the event waits. The event is not
saveable, and simulate-power-loss clears it. See
Die parity for its effect.
With hmb_mb (bbssd, mapping=dftl), the controller is DRAM-less and
asks for a Host Memory Buffer: Identify reports HMPRE and HMMIN as
hmb_mb in 4 KiB pages, and HMMAXD as 4096 descriptors. hmb_mb must
hold the whole mapping table, 8 bytes per page. Set Features 0Dh with EHM
checks the descriptor list (MPS-aligned entries whose sizes add up to
HSIZE, HSIZE at least HMMIN; entries of size 0 are skipped) and refuses a
second enable with Command Sequence Error. While the
buffer is enabled the mapping table lives there: a lookup costs hmb_ns
and writes move no translation page. A disable, a Controller Level Reset
or a system reset returns the lookups to the DFTL cache. FEMU stores
nothing in the buffer.
When the buffer goes off, the controller owns the table again. Each translation page whose mappings changed while it was on goes dirty in the cache, or pending if the cache does not hold it. The next request writes the pending ones, as for any other change.
lookup, HMB off: CMT hit, or a translation page read from flash on a miss
lookup, HMB on: hmb_ns (PCIe round trip), no flash read
HMB on --> off: TPs changed under the HMB --> dirty (cached) or pending
With ocp=on the PCIe function also has Function Level Reset (OCP PCI-6),
Advanced Error Reporting (PCI-26) and the Device Serial Number (PCI-25).
The serial number is a 64-bit FNV-1a hash of serial. An FLR, or a
system reset, is a Controller Level Reset that also clears CC, CSTS, AQA,
ASQ, ACQ and the interrupt mask. Without ocp none of this is there, and
a system reset leaves the controller as it was.
config space 80h PCIe capability (DEVCAP.FLR with ocp)
100h AER (48h bytes) --> 148h Device Serial Number
Only the FDP features report a saved value: FDP Mode is fixed at realize,
FDP Events can be changed at run time. A controller reset
(nvme_reset_features()) restores Arbitration, Power Management, the
temperature thresholds, Volatile Write Cache, Number of Queues, Interrupt
Coalescing, Interrupt Vector Configuration, Write Atomicity, Asynchronous
Event Configuration, Host Behavior Support and each namespace's Error
Recovery; the Software Progress Marker, LBA Range Type data, Key Value
Configuration and FDP event filters keep their values.
Asynchronous events
The controller holds up to aerl + 1 Asynchronous Event Requests; one more
fails with Asynchronous Event Request Limit Exceeded. An event is queued (at
most 16; more are dropped) and posted by nvme_process_aers() from a
main-loop bottom half, because a poller may raise it and must not touch the
admin CQ. A request that arrives while events are queued is matched at once
on the vCPU thread. The
event types and their log pages are listed in
log pages and counters.
Parameters
The frontend reads these -device femu properties. Their defaults and
ranges are in the generated reference; follow the link of each group.
| Property | Effect on the frontend | Reference |
|---|---|---|
queues | number of I/O queue pairs; sizes BAR0, MSI-X vectors, sq[], cq[] and the shadow doorbell page | queues, pollers and interrupts |
entries | CAP.MQES; the largest queue is entries + 1 | same |
max_sqes, max_cqes | Identify SQES and CQES; only 64-byte SQEs and 16-byte CQEs are accepted | same |
stride | doorbell spacing, 4 << stride bytes; also the shadow doorbell entry size | same |
multipoller_enabled, poller_ratio | number of pollers and queue ownership | same |
hiops_inline | NoSSD only: performance option, on by default; set off only when debugging | same |
aerl, elpe | held AER limit, Error log length | same |
mdts | largest data transfer per command | same |
intc, intc_thresh, intc_time | reset values of the coalescing features | same |
vid, did | PCI IDs | controller identity and capabilities |
cqr | whether queues must be physically contiguous | same |
mpsmin, mpsmax | memory page sizes CC.MPS may choose | same |
oacs, oncs, vwc | which optional admin and NVM commands are answered | same |
sgl | accept SGL data pointers | same |
cmbsz, cmbloc | Controller Memory Buffer on BAR 2 | same |
femu_mode, namespaces, namespace_modes | which mode tables are installed, NSID range | mode, capacity and namespaces |
ns_mgmt, streams, power_loss | admin commands answered; FTL thread applies data with power_loss | namespace management, streams and power loss |
pcie_bandwidth_mbps, pcie_prop_delay_ns, fw_cpu_ns | time added in nvme_process_cq_cpl() | host link and controller firmware |
subsys | CNTLID, FDP, shared namespaces | femu-subsys |
Interactions:
- With the defaults (
queues=8,stride=0) BAR0 is 16 KiB. Largequeuesorstridevalues grow it;queuesabove 511 withstride=0and 4 KiB pages leaves Doorbell Buffer Config unusable (the host then rings the registers). multipoller_enabled=1withpoller_ratio=1starts one poller per queue, which needs that many spare host cores.- The host's CC.MPS must fall within
mpsmin..mpsmax, or enabling the controller sets CSTS.CFS.
Counters
| Counter | Where kept | How to read it |
|---|---|---|
| Host read and write commands and bytes | poller_ctr[i], counted on the poller after execution, successful commands only, for every mode (the FTL thread counts them with power_loss) | SMART log: Host Read/Write Commands, Data Units Read/Written |
| Completions, and completions posted 20 us or more past their due time | poller_ctr[i] | BBSSD vendor command 0xEF, CDW10=5, prints both and resets them (timing model) |
| Error log entries | num_errors | SMART log, Error Information log |
The per-poller counters are padded to separate cache lines so pollers never share one; readers sum them. Media counters (WAF, GC pages, buffer hits) come from the FTL, not the frontend; see log pages and counters.
Validation
The frontend is covered by the qtest suite in
hw/femu/tests/qtest/femu-test.c, which drives the controller through its
registers with no guest. Cases that target this chapter include:
| Case | Checks |
|---|---|
bar0-size | BAR0 size for the queue count and stride |
cc-states | enable, disable, shutdown and failure transitions of CC and CSTS |
doorbell-errors | invalid doorbell writes raise Error events |
io-by-doorbell, io-by-shadow-doorbell | I/O through registers and through Doorbell Buffer Config |
dbbuf-too-many-queues | Doorbell Buffer Config refused when the entries do not fit |
intx-shadow-doorbell | the pin drops with shadow doorbells |
queue-mapping | Create I/O queue refused when the ring cannot be mapped |
shared-cq, shared-cq-pollers | SQs that share one CQ, with one and several pollers |
poller-burst, poller-burst-per-queue, poller-burst-sharded | full-queue bursts complete exactly once, in their own queue, under each poller layout |
cq-full | completions wait for CQ space |
delete-sq-in-flight, ns-retire-pollers | queue and namespace removal with I/O in flight |
aer-limit | the AER limit |
abort | Abort leaves the SQ unwritten, aborts queued admin and I/O commands, and enforces ACL |
cc-css | an unoffered CC.CSS fails the enable |
admin-psdt | admin commands naming SGLs are refused |
media-dnr | Unrecovered Read and Compare Failure set DNR |
fid-effects, fid-effects-kv, fid-effects-fdp | log 12h agrees with Get Features; log 13h is zero |
frmw | FRMW and the firmware slot log agree |
caps-* (22 configurations: every mode, the optional commands on, off and in pairs, Namespace Management, Streams, PI, FDP, mixed namespace modes) | every admin opcode, every I/O opcode on every namespace and every log id per command set is answered exactly when logs 05h and 00h list it; log 12h agrees with Get Features; OACS, ONCS, OCFS, LPA, SANICAP, VWC, FRMW, SGLS, CNS 1Ch and the Copy limits in Identify Namespace agree with the logs and with what the controller does; ONCS and the Format bit match what the configuration asks for; VS, CAP.CRMS, CRTO and BPCAP match the reported version (2.1, or 1.4 for Open-Channel); CNS 06h reports the version of each command set it offers and AOCS bit 0, CAP.AMS is 0, and LPA bit 5 and every command scope are clear |
ns-mgmt-before-identify | a managed bbssd namespace is addressed correctly before the host reads Identify Controller |
v2-refusals | CC.AMS other than round robin fails the enable; Identify CNS 00h on a KV namespace fails with Invalid I/O Command Set while CNS 08h answers; logs 00h and 05h refuse an unknown CSI; CNS 07h for KV is refused under CC.CSS 000b while CNS 1Ah is not; CNS 1Fh answers an allocated NSID and refuses 0 and FFFFFFFFh |
endgrp-reported | with a subsystem and FDP off, CTRATT bit 4, ENDGIDMAX and the ENDGID of a block and a KV namespace (CNS 08h, and CNS 05h for KV) report the one endurance group; CNS 19h lists group 1 for a start of 0 or 1 and no group for 2 or FFFFh; the FDP pages, FDP Mode and I/O Management are refused |
endgrp-list-absent | without a subsystem, CNS 19h fails with Invalid Field |
features-reset, features-reset-vwc | features return to defaults on reset |
admin-fuzz, io-fuzz and its variants | structured fuzzing of admin and I/O commands |
Run them all, or one by name, from a build directory (testing):
QTEST_QEMU_BINARY=./qemu-system-x86_64 ./tests/qtest/qos-test -m quick \
-p /x86_64/pc/i440FX-pcihost/pci-bus-pc/pci-bus/femu/femu-tests/io-by-shadow-doorbell
The documentation example above (frontend-sharded-pollers) is started by
check-docs, which enables the controller and moves one block.
Limits
- At most 2047 I/O queue pairs (the MSI-X table size), at most 65535 entries per queue.
- SQEs are 64 bytes and CQEs 16 bytes; other sizes are refused at realize.
- One controller per PCI function. No SR-IOV, no NVM Subsystem Reset, no controller memory for PMR.
- No interrupt coalescing; the coalescing features only store values.
- Arbitration is round robin by default (CAP.AMS 0). Each poller sweeps its queues in id order and takes at most 2^AB commands from each, AB being the Arbitration Burst of feature 01h; AB 7, the reset value, is no limit. The queue priority of Create I/O SQ is ignored.
- With
wrr=on(one poller only), CAP.AMS offers weighted round robin with an urgent class, and the host selects it with CC.AMS 001b. Each poller pass makes one grant: an urgent queue first, else high, medium and low by credits of HPW + 1, MPW + 1 and LPW + 1. A grant takes at most 2^AB commands (no limit for AB 7) and at most what the queue held when the grant chose it. A queue that can take none is passed over without a charge. No I/O grant starts while admin commands run; a grant already under way finishes. - A queue can have at most
sizecommands outstanding, and a poller's rings hold 65535 requests each. - Abort only reaches commands not yet fetched.
- Latency is a lower bound that depends on poller scheduling.
- The controller is not migratable.
Extension points
- A new mode: write a
FemuExtCtrlOpstable and anvme_register_*()function, add the mode to the enum inhw/femu/nvme.h, to thefemu_modebound innvme_check_constraints(), tonvme_mode_from_token()fornamespace_modes, tonvme_register_extensions(), and tofemu_needs_ftl_thread()andfemu_ftl_process_req()if it charges time on the FTL thread. A mode that charges time on the poller adds it toreq->expire_timein itsio_cmd. - A new I/O command shared by all block modes: add it to
nvme_io_cmd()and tonvme_io_effects()inhw/femu/nvme-caps.c. - A new admin command, feature or log page:
nvme_admin_cmd()andnvme_admin_effects();nvme_set_feature(),nvme_get_feature()and the support tablesnvme_feature_support[]andnvme_feature_cap[](withnvme_fid_supported()for a feature that needs something else present);nvme_get_log()andnvme_log_support(). An Identify bit that sums up commands or pages is derived innvme_caps_id_ctrl(), not set by hand. - A new host-side cost: add it next to the host-link and firmware-CPU
models in
nvme_process_cq_cpl(). - Anything that changes state the I/O path reads: wrap it in
nvme_pause_pollers()andnvme_resume_pollers(). - A new property: define it in
femu_props[]inhw/femu/femu.c, describe it inhw/femu/femu-props.c, check it innvme_check_constraints(), and regenerate the property reference (docs maintenance).
Source map
| File | Functions |
|---|---|
hw/femu/femu.c | nvme_init_pci(), nvme_init_cmb(), nvme_init_ctrl() (Identify Controller, CAP), nvme_check_constraints(), nvme_mmio_write(), nvme_write_bar(), nvme_process_db_admin(), nvme_process_db_io(), nvme_start_ctrl(), nvme_clear_ctrl(), nvme_reset_features(), femu_ftl_thread(), femu_ftl_process_req(), femu_needs_ftl_thread(), nvme_register_extensions(), nvme_register_extensions_ns(), femu_realize(), femu_exit() |
hw/femu/nvme-admin.c | nvme_create_sq(), nvme_create_cq(), nvme_del_sq(), nvme_del_cq(), nvme_init_poller(), nvme_start_dataplane(), nvme_set_db_memory(), nvme_identify(), nvme_get_feature(), nvme_set_feature(), nvme_get_log(), nvme_abort_req(), nvme_admin_cmd(), nvme_process_aers(), nvme_process_sq_admin() |
hw/femu/nvme-caps.c | nvme_admin_effects(), nvme_io_effects(), nvme_ns_io_effects(), nvme_log_support(), nvme_log_csi(), nvme_caps_id_ctrl() |
hw/femu/nvme-io.c | nvme_poller(), nvme_process_sq_io(), nvme_update_sq_eventidx(), nvme_process_cq_cpl(), nvme_post_cqe(), nvme_rw(), nvme_io_cmd() |
hw/femu/nvme-util.c | nvme_pause_pollers(), nvme_resume_pollers(), nvme_update_sq_tail(), nvme_update_cq_head(), nvme_update_cq_eventidx(), nvme_init_sq(), nvme_init_cq() |
hw/femu/intr.c | nvme_isr_notify_io(), nvme_isr_notify_admin(), nvme_irq_update(), nvme_irq_mask_changed(), nvme_setup_virq(), vector notifiers |
hw/femu/dma.c | PRP and SGL mapping, femu_dma_rw |
hw/femu/lib/rte_ring.c, hw/femu/lib/pqueue.c | the rings and the completion heap |
hw/femu/nvme.h | NvmeSQueue, NvmeCQueue, NvmeRequest, FemuCtrl, FemuExtCtrlOps, FemuPollerCtr, register layout |