Skip to main content

Scripts and tools

Mirrored from the FEMU repository

This page is hw/femu/docs/reference/scripts.md at FEMU d8b29b1e0 (2026-11-16), licensed GPL-2.0-or-later. Send corrections to the FEMU repository.

Every script and tool shipped under hw/femu/scripts/ and hw/femu/tools/: what it does, its arguments and the environment variables it reads. The top-level femu-scripts link points to hw/femu/scripts/, so from build-femu/ you can also run them as ../femu-scripts/NAME.

Scripts marked legacy do not start FEMU. They start QEMU's stock nvme device or another disk on the old x86_64-softmmu/ binary path, set thread affinity by hand, or prepare a specific lab machine. They are kept for reference. Do not use them.

Build and setup​

ScriptRun fromWhat it does
pkgdep.shanywhere, as rootInstalls the build dependencies with apt-get on Debian and Ubuntu. Exits with pkgdep: unsupported system type elsewhere. CI does not run it.
femu-compile.shbuild-femu/Runs make clean, then ../configure --enable-kvm --target-list=x86_64-softmmu --enable-slirp --disable-libnfs --disable-libiscsi --disable-curl, then make with one job per CPU. --enable-csd-ubpf adds uBPF CSD programs, --enable-csd-ubpf=PATH uses the uBPF tree at PATH, and --werror makes a compiler warning fail the build. Without that flag the script disables warnings as errors, since a newer compiler can warn in QEMU code. Any other argument is an error. See build.md.
femu-copy-scripts.shbuild-femu/Copies pkgdep.sh, femu-compile.sh, make-guest-image.sh, run-guest-ssh.sh, the run-blackbox.sh, run-blackbox-fdp.sh, run-whitebox.sh, run-nossd.sh, run-zns.sh, run-csd.sh and run-kvssd.sh launchers, pin.sh and ftk/ into the current directory, overwriting earlier copies. It does not copy run-cxlssd.sh, ssd-config.sh, configs/ or the guest tools; run those from ../femu-scripts/.

Guest image and access​

ScriptWhat it doesArguments and environment
make-guest-image.shBuilds an Ubuntu 24.04 guest image from the official cloud image. It checks the download's SHA256 sum, and its signature when gpgv and the Ubuntu keyring are installed, with user femu, an SSH key, a serial console, nvme-cli and fio. Needs no root, but needs access to /dev/kvm unless you pass --no-kvm. See guest-image.md.-o DIR output directory, -n FILE image name (default u20s.qcow2, the name the launchers expect), -s SIZE (default 32G), -k FILE public key, -p PW console password, --cxl adds ndctl, daxctl and cxl-cli, --packages LIST extra packages, --qemu PATH, --qemu-img PATH, --no-kvm, --timeout SEC (default 1800), -f replace an existing image. Reads IMGDIR (default output directory $HOME/images), QEMU and QEMU_IMG.
run-guest-ssh.shLogs in to a running guest built by make-guest-image.sh, or runs one command in it: ./run-guest-ssh.sh sudo nvme list.IMGDIR (default $HOME/images), SSH_KEY (default $IMGDIR/femu-guest-key), SSH_PORT (default 8080), GUEST_USER (default femu).

Launchers​

Run each NVMe launcher from build-femu/ after femu-copy-scripts.sh. They start ./qemu-system-x86_64 with sudo, KVM, -cpu host, -nographic, -name NAME,debug-threads=on (so the host shows FEMU's thread names), a virtio-scsi boot disk and user networking that forwards host port SSH_PORT to the guest's port 22. run-cxlssd.sh is different: it is not copied, so run it as ../femu-scripts/run-cxlssd.sh, and it adds no disk, no network and no sudo (see below). All except run-cxlssd.sh read:

VariableDefaultMeaning
IMGDIR$HOME/imagesDirectory of the guest image
OSIMGF$IMGDIR/u20s.qcow2Guest image file; the script exits with status 1 if it does not exist
SSH_PORT8080Host port forwarded to the guest's SSH port; use a different one for each VM you run at the same time

The device geometry, size and timing are shell variables at the top of each script. Edit them there; they are not read from the environment.

LauncherDeviceGuestOutput files
run-blackbox.shBlackBox SSD, 12 GiB namespace on 16 GiB of NAND (BlackBox)4 vCPUs, 4 GiBlog, qmp-sock
run-blackbox-fdp.shBlackBox with a femu-subsys that has FDP on, 4 handles, 1 reclaim group (FDP)4 vCPUs, 4 GiB/tmp/femu-fdp.log, qmp-sock
run-nossd.shNoSSD, 4 GiB (NoSSD)4 vCPUs, 4 GiBqmp-sock
run-zns.shZNS, 4 GiB, QLC timing, 16 zones of 256 MiB (ZNS)4 vCPUs, 4 GiBlog, qmp-sock
run-whitebox.shOpen-Channel 2.0 (OCVER=2 in the script; 1 selects 1.2), 4 GiB (OCSSD)4 vCPUs, 4 GiBqmp-sock
run-csd.shComputational storage, 4 GiB, 4 compute units (CSD). It does not set csd_program_dir, so only the built-in program type loads4 vCPUs, 4 GiBlog, qmp-sock
run-kvssd.shKey-value SSD, 4 GiB of value space on the BlackBox geometry (KV); the guest needs Linux 6.0 or newer4 vCPUs, 4 GiBlog, qmp-sock
run-cxlssd.shOne femu-cxl-ssd below a CXL host bridge (CXL SSD)4 vCPUs, 4 GiB, no disk and no network unless you add themcxlssd-stats.log, cxlssd-io-N.log and cxlssd-spt.log in LOG_DIR when the guest asks for them through lsa-control

run-blackbox.sh also passes FEMU_EXP_LOG, FEMU_SECRET and FEMU_DUMP_LPN through sudo to QEMU (environment variables). The other launchers pass no variables to QEMU.

qmp-sock is created by root in the current directory; log is written by tee as you. Two launchers started from the same directory share them, so start a second VM from another directory.

run-cxlssd.sh runs QEMU without sudo and adds its own arguments after the ones it builds, so you append a boot disk, a network and a QMP socket on its command line. Its settings are environment variables:

VariableDefaultSets
QEMU./qemu-system-x86_64the QEMU binary
CXL_SIZE256Mmedia size, a number followed by M or G
CACHE_PAGESsize in MiB / 20 x 256cache-pages
CACHE_WAYS1cache-ways; full means CACHE_PAGES
BLOCKS_PER_PLANE867 for 48G, 1732 for 96G, else 0blocks-per-plane
CACHE_POLICYfifocache-policy
DERoffder
CYLON_KERNEL_ACKoffcylon-kernel-ack
PREFETCH_DEGREE, PREFETCH_STRIDE0, 1prefetch-degree, prefetch-stride
CHANNELS, LUNS_PER_CHANNEL, PAGES_PER_BLOCK8, 8, 256NAND geometry
READ_NS, PROGRAM_NS, ERASE_NS, CHANNEL_NS40000, 200000, 2000000, 0NAND timing
GC_THRESHOLD, GC_THRESHOLD_HIGH75, 95GC thresholds
FTLonftl
LSA_CONTROLonlsa-control; see security
CYLON_FIRST_TOUCH_PROGRAM, CYLON_FREE_WRITEBACKoffthe matching properties
LOG_DIR, LOG_LIMIT., 64Mlog-dir, log-limit
TRACEFS_DIRunsettracefs-dir, only when set
CXL_BACKENDmemory-backend-rammemory backend type and options
ACCEL, CPU, CPUS, RAMkvm, host, 4, 4Gaccelerator, CPU model, vCPUs, guest RAM
DRY_RUN01 prints the command instead of running it

These defaults follow Cylon's launch script and differ from the device's own defaults: one cache way instead of 16, a cache of size / 20 (3072 pages for 256 MiB) instead of 1024 pages, 8x8 channels and LUNs instead of 4x4, fixed blocks-per-plane for the 48G and 96G sizes (the least the device accepts; Cylon's sizes have no spare blocks), and lsa-control on instead of off.

Configuration files​

ScriptWhat it does
ssd-config.sh CONFIG [--device-only | --check]Expands an INI-style file into -device femu,... arguments (and a -device femu-subsys,... for a [subsys] section). Keys are device property names; mode = bbssd and the like stand for femu_mode. With --device-only it omits the -device words; with --check it only validates. It checks keys against -device femu,help of the binary in FEMU_BIN, or of build-femu/, build/ or build-official/ under the source tree.
ssd-config-test.sh [QEMU]Expands every file in configs/, starts QEMU with each and requires the device to come up, and checks that the parser rejects bad input. The binary is the argument, FEMU_BIN, or the first one found as above. CI runs it.

The files in configs/:

FileDevice
bbssd.confBlackBox, 4 GiB, 8 channels of 8 LUNs
bbssd-overprovisioned.confBlackBox sized with op_pcent=10
csd.confComputational storage, the device of run-csd.sh without its no-effect scheduler properties
fdp.confBlackBox with FDP on a [subsys] section
heterogeneous.confOne controller with a BlackBox, a ZNS and a NoSSD namespace
kvssd.confKey-value SSD, the device of run-kvssd.sh
nossd.confNoSSD with link and firmware time from the NVMeCHA controller (NoSSD)
ocssd.confOpen-Channel 2.0, the device of run-whitebox.sh; lver = 1 selects 1.2
qlc.confBlackBox with QLC per-page-type timing
write-buffer.confBlackBox with a 2048-page write buffer, vwc=1 and Write Zeroes
zns.confZNS, 16 zones of 256 MiB, at most 16 active and 8 open

Run it from build-femu/ with FEMU_BIN set. Through the ../femu-scripts link the script cannot find the binary on its own, and then it skips the key check:

FEMU_BIN=./qemu-system-x86_64 ../femu-scripts/ssd-config.sh ../femu-scripts/configs/zns.conf

The launchers take no arguments. To boot a config, copy a launcher and put the output in place of its -device femu option, or call QEMU yourself:

QEMU_ARGS=$(FEMU_BIN=./qemu-system-x86_64 ../femu-scripts/ssd-config.sh my-ssd.conf)
./qemu-system-x86_64 -enable-kvm -cpu host -smp 4 -m 4G $QEMU_ARGS ...

The file format: keys are femu device properties and mean what -device femu,help says. # and ; start comments, section headers are labels for the reader (except [subsys]), and a key with an empty value is ignored. List values such as namespace_modes = bbssd,znssd,nossd get their commas doubled for QEMU automatically.

[device]
mode = bbssd # friendly name for femu_mode
devsz_mb = 4096

[geometry]
secs_per_pg = 8 # 4 KiB pages
luns_per_ch = 8
nchs = 8

[timing]
pg_rd_lat = 40000 # ns
pg_wr_lat = 200000

A misspelled key stops the expansion when the binary is found:

ssd-config: unknown property 'gc_polcy' -- not one FEMU accepts
ssd-config: config rejected; see the warnings above

Guest-side test tools​

Copy these into the guest and run them there. The C programs build with gcc -O2 -o NAME NAME.c and need no headers beyond libc.

ToolArgumentsWhat it checks
femu-test.sh--yes [DEVICE], default /dev/nvme0n1Data integrity, counters, deallocate, and zone or key-value commands, chosen by what the namespace reports. It overwrites the whole namespace, so --yes is required, and it refuses a mounted device. See testing.
kv-probe.c[CONTROLLER], default /dev/nvme0A key-value Store, Exist, Retrieve, Delete and a Retrieve of the deleted key (KV)
kv-bench.c[OPTIONS] NODE, the generic node such as /dev/ng0n1; build with gcc -O2 -pthreadA key-value workload of Store, Retrieve, Delete, Exist and List with the kvbench knobs (--keys, --key-len, --values, --ratio, --mix, --threads). It checks each answer against a model of the keys and prints the rate and latency of each command. --control legacy-opcodes or corrupt-model makes it fail on purpose. Needs an empty namespace and deletes its keys at the end. Exits 1 on an error or a mismatch. See KV benchmark.
aer-probe.c[CONTROLLER], default /dev/nvme0That crossing the temperature threshold completes an Asynchronous Event Request, and that the event is re-armed after the log is read
zone-aen-probe.c[CONTROLLER] [NAMESPACE], defaults /dev/nvme0 /dev/nvme0n1That a zone taken read only raises the Zone Descriptor Changed notice; needs a ZNS device with err_write_fail_ppm set (ZNS)
fdp-test-nvme-admin.shnone; uses /dev/nvme0, /dev/nvme0n1 and /dev/ng0n1The FDP admin commands of nvme-cli against the configuration run-blackbox-fdp.sh creates (4 handles, 1 reclaim group); written for a Linux 6.12 guest
blktests-guest.shGROUP [DEVICE], default device /dev/nvme0n1; reads BLKTESTS_REPO, BLKTESTS_REF and WORKDIR (parent of a new scratch directory, default /tmp)Clones, builds and runs one blktests group against the namespace, and prints the counts of passed, failed and not run tests. Exits 1 when a test that ran on the namespace failed or the run stopped early, and 2 when the setup failed. Needs network access and writes to the namespace. See conformance suites.
nvme-cli-e2e-guest.sh[CONTROLLER [NAMESPACE]], defaults /dev/nvme0 /dev/nvme0n1; reads NVMECLI_REPO, NVMECLI_REF (default v3.1), E2E_TIMEOUT and WORKDIR (parent of a new scratch directory, default /tmp)Builds nvme-cli and runs its end-to-end suite. Exits 1 when a test failed or the suite ended abnormally, and 2 when the setup failed. A failure of test_get_lba_status counts as a known nvme-cli defect only when the same command passes with the numeric namespace ID. Needs network access, writes to the namespace and can format it.
zenfs-crash-guest.sh[NAMESPACE], default the first host-managed zoned namespace; reads ROCKSDB_REPO, ROCKSDB_REF (default v9.10.0), ZENFS_REPO, ZENFS_REF (default 919c2eb), ZENFS_PREFIX, ZENFS_JOBS, ZENFS_ROUNDS, ZENFS_INTERVAL, ZENFS_THREADS, ZENFS_KEYS, ZENFS_CONTROL and WORKDIRBuilds RocksDB with ZenFS and runs the RocksDB blackbox crash test on the namespace: rounds of db_stress killed with SIGKILL, then a final check of every key. Exits 1 when a round or the final check failed, and 2 when the setup failed. ZENFS_CONTROL=reset-zones makes it fail on purpose. Needs network access and formats the namespace with ZenFS. See ZenFS crash test.
kv-bench-guest.sh[NODE], default the one /dev/ngXnY without a block device; reads KVBENCH_KEYS, KVBENCH_OPS, KVBENCH_THREADS, KVBENCH_SEED, KVBENCH_CONTROL and WORKDIRBuilds kv-probe.c and kv-bench.c from its own directory, runs kv-probe, then a small-value and a large-value kv-bench run. Exits 1 when a check failed and 2 when the setup failed. See KV benchmark.

Run this one on the host:

ScriptWhat it doesArguments and environment
guest-conformance.shBoots a guest for each suite with a fresh overlay of the guest image, runs the suite's guest script in it over SSH, stops the guest and prints RESULT SUITE PASS, FAIL or SETUP-ERROR. An SMBIOS string checks that the SSH port reaches this guest. It refuses an overlay or PID file that an earlier run left in the result directory. CI does not run it. See conformance suites.Suites blktests-block, blktests-zbd, nvme-cli, zenfs-crash, kv-bench or all. Options --qemu PATH (default ./qemu-system-x86_64), --image FILE, --key FILE, --port N (default 8090), --outdir DIR, --timeout SEC (default 7200 for zenfs-crash, else 3600), --boot-timeout SEC (default 300), --device PROPS (the FEMU properties for a control run), --smp N and --mem SIZE (default 16 and 16G for zenfs-crash, else 4 and 4G), --sudo, --keep. Reads QEMU, IMGDIR, OSIMGF, SSH_KEY and SSH_PORT, and passes the BLKTESTS_*, NVMECLI_*, ROCKSDB_*, ZENFS_* and KVBENCH_* variables that --help lists to the guest. Exits 0 when every suite passed, 1 when a suite failed, and 2 when a suite could not run and no suite failed.
csd-pause-check/run.shBoots a small KVM guest from a kernel and a busybox initramfs twice, with csd_pause off and on. In the guest, csdlat.c times Execute of a program that sleeps on the host, and clockcheck.c counts clock steps back on each vCPU, sleeps that end early, and the gap between the TSC and the kvmclock. Prints both results; it checks nothing itself. See CSD.QEMU KERNEL MODDIR OUTDIR [VCPUS]: a FEMU binary, a guest bzImage, its /lib/modules/<version> for nvme.ko or none, an output directory, and the vCPU count (default 4). Needs gcc with static glibc, a static busybox, cpio and gzip.

Host monitoring tools​

Run these on the host, against a running or a dry-run QEMU. They need Python 3 and no other package.

ScriptWhat it doesArguments
femu-config-check.pyStarts QEMU under qtest with the given options and no guest, then reports either FEMU's refusal or the configuration it builds: capacity, geometry, over-provisioning, line or reclaim unit size, zones, mode and NAND timing per namespace, and QEMU's warnings. --manifest FILE also writes the host, the build and the source of each property value. See check a configuration.[--json] [--manifest FILE] [--qemu BINARY] [--timeout SEC] [--] QEMU-OPTIONS, or the value of one -device option; --self-test checks the tool. Exits 0 accepted, 1 refused, 2 when the check failed. It loads femu-stat.py from its own directory.
femu-steady-state.pyReports the first round where a series of per-round results is at steady state, by the SNIA PTS rule: over the last 5 rounds, the spread is at most 20% and the least-squares trend at most 10% of the average. See repeatable numbers.[--window W] [--excursion P] [--slope P] [--column N] [--metric iops|bw|lat_mean] [--json] [FILE...]: fio JSON outputs, one per round, or one value per line; --self-test checks the tool. Exits 0 steady, 1 not steady, 2 on an input error.
femu-trace2csv.pyTurns the femu_cqe lines of a QEMU trace log (-trace femu_cqe -D femu.log) into a CSV I/O trace: ISO 8601 time (-msg timestamp=on), completion queue, command ID, opcode, starting LBA, blocks, status and modelled time.[LOG] (stdin when absent); --self-test checks the parser.
femu-prom.pyServes the query-femu counters of every femu device of a running VM at /metrics in the Prometheus text format: page and erase counters, WAF and line counts, labelled by device and namespace. Each scrape reads the device again. It loads femu-stat.py from its own directory.SOCKET, --port PORT (9813), --bind ADDR (127.0.0.1), --once prints one exposition; --self-test renders a canned reply without QEMU.
femu-stat.pyShows the FTL counters of each femu device of a running VM, like top: pages, WAF since start and over the interval, rates, line, reclaim unit and zone counts, and I/O stage times. It polls QMP query-femu and query-femu-stages and never resets the counts. See live counters.SOCKET (a Unix socket path or HOST:PORT), --interval SEC, --count N, --once, --json, --ns LIST, --path QOMPATH; --self-test --qemu BINARY checks the tool under qtest. Exits 2 on a connection error.

Host tuning helpers​

ScriptWhat it does
ftk/qmp-vcpu-pin -s SOCKET CPU...Pins each vCPU thread to a host CPU with taskset, using QMP query-cpus-fast on SOCKET; vCPU i goes to the i-th CPU in the list, wrapping around. It imports ftk/qmp.py. Run it with sudo when QEMU runs as root. A Unix socket path longer than about 107 bytes fails with AF_UNIX path too long.
pin.sh [FIRST_CPU]Pins each vCPU thread, then each femu-poller, FEMU-FTL-Thread, femu-cxl-ftl and femu-cxl-cca, to its own host CPU, starting at FIRST_CPU (default 0), and moves the other QEMU threads, femu-csd-cu included, to the CPUs after those. It finds the threads by name, so QEMU must run with -name NAME,debug-threads=on (the launchers pass it), and it finds QEMU with pgrep -x qemu-system-x86; set QEMU_PID when several run. Run it after the guest has booted: the pollers start when the guest enables the controller. It stops if the host has too few CPUs. See performance tuning.
set_cpu_perf_mode.shSets every CPU's cpufreq scaling policy to performance through sysfs. Run it as root.

Documentation tooling​

These run in CI. docs-maintenance.md explains each.

ScriptArguments
gen-property-docs.py--qemu BINARY regenerates reference/properties.md and runtime-properties.md from the binary; --check compares instead of writing (--qemu is still required); --schema FILE also writes a JSON Schema (draft 2020-12) of each device's own properties: JSON type, integer range, default, description, QOM type and topic
gen-combinations.pyrewrites reference/combinations.md from the refusal messages in hw/femu; --check compares instead of writing; --self-test checks the parser
gen-mode-table.pyrewrites the mode tables from docs/modes.py; --check compares and checks modes.py against the code
check-doc-links.py[--root DIR] [PATH...]; checks every relative link and anchor
check-changelog-hashes.py[--head COMMIT] [FILE]; checks that every commit the CHANGELOG names is in the history of --head (needs a full clone)
check-c-style.py[DIR] (default hw/femu) or --self-test; finds C99 comments and trailing whitespace in .c and .h files, and fails on a tree with no source file
check-patches.shBASE HEAD or --self-test; runs QEMU's scripts/checkpatch.pl on each commit in BASE..HEAD, merges left out: an ERROR or a git or checker failure fails, a WARNING is printed in full. A base of all zeros (a new branch) checks HEAD alone; binary or mode-only commits are skipped. CI runs it on the commits of each push and pull request
check-property-tests.pyno argument or --self-test; checks that a file under tests/ or scripts/ names each property in reference/properties.md, or that tests/untested-properties.list lists it with a reason
check-unit-tests.pyno argument, --meson-names, --listed FILE or --self-test; checks that each tests/unit/test-*.c is in tests/meson.build or tests/Makefile, and that the femu-unit suite lists every meson unit test
check-doc-examples.py--lint, --list, --self-test, --qemu BINARY, --qos-test BINARY, --only NAME, --timeout SEC, [PATH...]; checks every code block's tag and runs the tagged examples

CXL caching API tools: hw/femu/tools/cca/​

Guest code for femu-cxl-ssd,cca=on. make -C hw/femu/tools/cca in the guest builds libcca.a, the ccactl command and the cca-test self-test. run-guest-tests.sh [-d MEMDEV] [-x DAX] [-o LOG] [CASE...] builds them and runs the self-test as root, logging to cca-guest-YYYYMMDD-HHMMSS.log by default. See the caching API guide and the tool README.

Legacy scripts​

None of these start FEMU. Several hard-code paths on one lab machine (/home/huaicheng/images), the old x86_64-softmmu/qemu-system-x86_64 binary, or a u14s.qcow2 image.

ScriptWhat it was for
femu-run.sh, m.sh, s1.sh, s2.shStart a VM with QEMU's stock -device nvme
f.sh, dp.sh, dp-run.sh, ide-run.sh, virtio-run.sh, null-run.shStart a VM with a raw, tmpfs-backed, IDE, virtio-blk or null data disk instead of FEMU, for comparison runs
gdb-run.shStart the stock nvme device under gdb; see debugging for a version that runs FEMU
valgrind-run.shStart the stock nvme device under valgrind, with properties that no longer exist
aff.sh PIDPin 20 consecutive thread IDs starting at PID to CPUs 5 to 24
getaff.shPrint the CPU affinity of every thread of every qemu process
pre.shTurn off address space randomization, format /dev/nvme0n1 with ext4 and mount it for one lab user. Destroys data
pre-all.shRun pre.sh and network setup over SSH on three named lab hosts
tuning.shStop a list of services on one lab machine

One more NoSSD launcher, and a benchmark harness in a subdirectory of hw/femu/scripts/ with its own README, are not covered here.