$ replaybook

Usage

Play Docker incidents

# install and inspect the official pack
cargo install replaybook
replaybook add ducks/replaybook-scenarios
replaybook list

# run by ID, choose randomly, or force a fault variant
replaybook run 001-nginx-502
replaybook run --random --tag postgres
replaybook run 006-sidekiq-cant-connect --fault redis-auth

# change the default 15-minute SLA
replaybook run 001-nginx-502 --sla 5

Replaybook opens a tmux session inside a workstation container. The left pane is the incident shell. The right pane shows the page, SLA countdown, and hint status. Run get-hint inside the shell when needed.

Create and test a Docker scenario

replaybook new 010-checkout-down --pack ./company-incidents
replaybook validate ./company-incidents/010-checkout-down
replaybook test ./company-incidents/010-checkout-down
replaybook test --all ./company-incidents

add, run, and test validate scenario structure before starting Docker. See Docker scenarios for the format and trust model.

Evaluate agents on Linux hosts

The host-native runner builds a disposable NixOS VM, reproduces the selected incident, runs the agent directly as root, then verifies the repair immediately, after service restart, and after host reboot.

# list scenario IDs and versions
python integrations/host/run_host_matrix.py --list-scenarios

# test the hidden reference repair before spending model tokens
integrations/host/run-host-native.sh \
  --scenario 014-missing-rails-migration \
  --oracle

# run a model matrix through the default Claux adapter
python integrations/host/run_host_matrix.py \
  --scenario 013-sidekiq-wrong-redis \
  --scenario 014-missing-rails-migration \
  --models deepseek/deepseek-v4-flash-0731 openai/gpt-5.6-luna z-ai/glm-5.2 \
  --attempts 3 \
  --concurrency 2

Set OPENROUTER_API_KEY for the bundled Claux adapter. The controller needs Linux, Nix, nixos-generators, QEMU, SSH, and hardware virtualization when available. Each concurrent trial consumes one VM and two adjacent localhost ports.

Bring another harness

python integrations/host/run_host_matrix.py \
  --scenario 001-nginx-502-host \
  --models gpt-5.6-sol \
  --agent-adapter integrations/host/adapters/codex.sh \
  --agent-payload /path/to/codex \
  --agent-env-file /path/to/credentials.env \
  --agent-name codex \
  --attempts 1

Replaybook owns the VM, instruction, timeout, lifecycle, and verifier. The adapter invokes the harness and writes normalized result JSON. It does not decide whether the repair passed. The bundled harness skill scaffolds and validates a new adapter.

Hosted human sessions

A dedicated remote VM can host one isolated trainee session. Replaybook provisions the scenario, issues a restricted SSH credential, and attaches the local terminal to it.

replaybook remote 001-nginx-502 \
  --host replaybook@training-vm.example.com

REPLAYBOOK_CONTROL_TOKEN=... replaybook serve \
  --host replaybook@training-vm.example.com \
  --bind 127.0.0.1:8080

serve exposes an authenticated control plane for creating, inspecting, expiring, and destroying sessions. Use a TLS reverse proxy before binding it beyond loopback. See the hosting security model.

Results and transcripts

Human sessions are appended to ~/.local/share/replaybook/sessions/sessions.jsonl. Export them with replaybook export. Each record includes the scenario, outcome, elapsed time, hints, selected fault, and terminal transcript path.

Agent matrices write under jobs/host-matrix-*. Each trial keeps normalized results, complete harness transcripts when available, verifier outcomes, timing, token usage, cost, model, scenario version, Replaybook commit, and harness version. Provider and harness failures are recorded as unavailable rather than counted as failed repairs.