Skip to main content

FEMU documentation

Mirrored from the FEMU repository

This page is hw/femu/docs/README.md at FEMU d8b29b1e0 (2026-11-16), licensed GPL-2.0-or-later. Send corrections to the FEMU repository.

Start with the goal you have. What changed since the last release is in the changelog.

Where FEMU sits: the guest drives FEMU’s NVMe or CXL device, the timing model decides when each request completes, and the data goes straight to the memory backend in host memory.

Figure: Where FEMU sits: the guest drives FEMU's NVMe or CXL device, the timing model decides when each request completes, and the data goes straight to the memory backend in host memory.

I am new to FEMU​

Read these in order:

  1. Requirements: host OS, KVM, memory, and the guest kernel each mode needs.
  2. Build: dependencies, femu-compile.sh, optional features, common build errors.
  3. Guest image: make-guest-image.sh, other ways to get an image, SSH access.
  4. Quick start: build, boot a BlackBox SSD, run fio and read the write amplification factor.

I want to learn by doing​

The tutorials walk through nine tasks in a real guest, with the output each step should print: a first SSD, GC and WAF, ZNS, FDP, latency tuning, several namespaces, KV, CXL memory and configuration files.

I want a specific kind of SSD​

Choosing a mode has the full decision table: every femu_mode value, the settings for each goal, which features combine, and what the guest needs. In short:

GoalModeLauncherGuide
A fast NVMe drive with no FTL timingNoSSD (femu_mode=2, the default)run-nossd.shNoSSD
A conventional SSD with a device FTL, GC and WAFBlackBox SSD (femu_mode=1)run-blackbox.shBlackBox
A Zoned Namespace SSDZNS (femu_mode=3)run-zns.shZNS
A host-managed OpenChannel SSDOCSSD (femu_mode=0)run-whitebox.shOCSSD
A key-value SSDKV (femu_mode=5)run-kvssd.shKV
Computational storageCSD (femu_mode=4)run-csd.shCSD, CSD guest tools
NVMe Computational Programs and Subsystem Local MemoryCSD with csd_slm_size, csd_cp=onnoneCP and SLM
Flexible Data PlacementBBSSD with femu-subsys,fdp=onrun-blackbox-fdp.shFDP
One NAND timing file for bbssd and ZNSnand_profilenoneNAND profiles
A pSLC cache in front of MLC, TLC or QLC NANDBBSSD with nand_cell_type and tier_mode=staticnoneTiered NAND
Create and delete namespaces from the guestNoSSD or BBSSD with ns_mgmt=onnoneNamespace management
Per-block metadata and protection informationNoSSD or BBSSD with meta, mc, pi=onnoneMetadata and PI
A TCG Opal self-encrypting drive: locking ranges and an MBR shadowNoSSD or BBSSD with x-opal=onnoneTCG Opal
Several namespaces on one controllerany NVMe modenoneSeveral namespaces
An LSM-tree key index for KV namespacesKV with kv_index=lsmrun-kvssd.sh with KV_INDEX=lsmLSM-tree key index
Small KV values packed into shared NAND pagesKV (default; kv_pack=off for one value a page)run-kvssd.shKV value packing
A CXL memory-semantic SSDfemu-cxl-ssd devicerun-cxlssd.shCXL SSD, design note
Guest control of the CXL SSD cache: pin, drop, uncached rangesfemu-cxl-ssd,cca=onrun-cxlssd.shCXL caching API
The CXL SSD medium also as an NVMe namespacefemu,femu_mode=1,cxl_ssd=<id>run-cxlssd.sh plus -device femu,...CXL NVMe link

The guest kernel each mode needs is in requirements.md.

I want to look up a parameter or counter​

  • Device properties: every -device femu, femu-subsys and femu-cxl-ssd property, generated from the binary.
  • Parameter manual: the parameters grouped by component, with units, valid values, interactions and worked configurations.
  • Runtime properties: QOM properties and counters you read or set with qom-get and qom-set.
  • Log pages and counters: vendor log C0h (WAF and media counters), telemetry, supported log pages, asynchronous events, keeping the Persistent Event log in a file.
  • QMP query-femu: the FTL state of a namespace, read from the host with no guest tools. It gives the lines of a bbssd namespace, the reclaim units of an FDP namespace, or the zones of a znssd namespace. query-femu-stages gives the time of each stage of an I/O command, with io_stages=on.
  • Changelog: what changed since femu-v9.0.1, including properties that are now refused and settings whose effect changed.
  • Scripts and tools: every shipped script and tool, its arguments and environment variables, and which ones are legacy.

I want to understand how FEMU works​

  • Architecture: the layers from the guest interface to the memory backend, the threads, and a walk through one NVMe write and one CXL load.
  • Choosing a mode: which mode or feature fits a goal, and which ones combine.
  • Timing model: how latency is computed and enforced, the properties that control it, and how to measure it.
  • Security and limits: what a guest can do to the host, migration and snapshots, property compatibility, host sizing.
  • CXL SSD design: the design note for femu-cxl-ssd.

I want the full design​

The design manual has one chapter per component, each with diagrams, data structures, algorithms, parameters, counters, limits and a source map:

The same pages, with figures, are collected in one PDF: the FEMU Manual, hosted on the FEMU website. The Markdown pages are the reference when the two differ.

I want to measure or tune​

  • Measuring: WAF and counters from log page C0h, SMART, CXL counters, fio recipes per mode, repeatable numbers.
  • Performance tuning: pollers, CPU pinning, hugepages, NUMA, host settings, and what each knob trades.

Something does not work​

I want to change FEMU​

  • Testing: unit tests, the qtests, the documentation checks, guest-side tests, and how to add a test.
  • Keeping the documentation correct: the generated references, the mode table and the example checks.
  • Code structure: what lives where under hw/femu/, and where to start a change. Architecture explains how the parts work together.
  • Contributing: style, tests, sign-off and pull requests.
  • AI policy: using AI coding tools, disclosure with Assisted-by:, and what is not allowed.
  • hw/femu/scripts/gen-property-docs.py regenerates the property reference, and hw/femu/scripts/check-doc-links.py checks that every relative link in the docs resolves. CI runs both.

How to cite​

If you use FEMU in your research, cite the FAST '18 paper. The BibTeX entry is in the README, and CITATION.cff has the same entry for citation tools. If you use FDP, the CXL SSD or CSD, also cite the paper that mode comes from: WARP (FAST '26), Cylon (FAST '26) or CEMU (ASPLOS '26).