Practice incidents
Enter a broken Docker environment with real services, an SLA, hints, and a terminal transcript. Diagnose the failure and repair it.
Stateful infrastructure evaluation
Practice production failures yourself, or put an AI agent inside a fresh Linux host and measure what survives after it leaves.
Enter a broken Docker environment with real services, an SLA, hints, and a terminal transcript. Diagnose the failure and repair it.
Run Claux, OpenCode, Codex, or another harness on a disposable NixOS VM. An external verifier checks the repair after restart and reboot.
Replaybook began as a way to turn postmortems into playable incidents. The same idea now supports repeatable agent evaluation: reproduce a stateful failure, preserve the work already in flight, hide the answer from the agent, and verify the deployed system instead of its final explanation.
Ling 3.2 Flash Free Core via Claux / OpenRouter (20261115.0.0).
33 evaluated, 11 failed, 0 unavailable. Ling 3.2 Flash Free passed 22 of 33 Core trials (66.7%), with a 694-second median agent duration and $0 in provider-reported inference cost. Eleven attempts exceeded the deadline; seven left repairs that passed subsequent lifecycle verification but remain failures.
Explore Core scores · Inspect this cohort · All published releasescargo install replaybook
replaybook add ducks/replaybook-scenarios
replaybook list
replaybook run 001-nginx-502
Playable scenarios require Docker. Replaybook opens a workstation container on the incident network and records the complete terminal session. Prebuilt binaries for Linux and macOS are available on the releases page.
python integrations/host/run_host_matrix.py \
--scenario 014-missing-rails-migration \
--models deepseek/deepseek-v4-pro \
--attempts 1
Host-native evaluation runs from a Linux controller with Nix and QEMU. Each trial gets a new NixOS VM with real systemd services and no Docker socket. Claux is the default adapter; the normalized adapter contract can run another harness without changing the verifier.