$ replaybook

Model results

Reflection Beam 1.0 501B A23B

reflection/beam-1.0-501b-a23b

Each row is an independent scenario cohort. Releases, providers and reasoning settings remain separate; these results are not pooled into a single model score.

Repairs, timing and cost

ScenarioReleaseProvider / reasoningRepairsMedian timeKnown costEvidence
033-tls-trust-rotation v2core 20261027.0.0OpenRouter / high 3/30 unavailable 61s $0.09593/3 trials report cost Inspect attempts →
028-nix-store-disk-pressure v1core 20261027.0.0OpenRouter / high 3/30 unavailable 63s $0.11733/3 trials report cost Inspect attempts →
024-discourse-interrupted-deploy v1core 20261027.0.0OpenRouter / high 3/30 unavailable 54s $0.09793/3 trials report cost Inspect attempts →
023-auth-secret-rollout v1core 20261027.0.0OpenRouter / high 3/30 unavailable 62s $0.08223/3 trials report cost Inspect attempts →
021-discourse-shared-uploads v1core 20261027.0.0OpenRouter / high 3/30 unavailable 356s $0.15123/3 trials report cost Inspect attempts →
019-rust-fd-leak v1core 20261027.0.0OpenRouter / high 3/30 unavailable 60s $0.05293/3 trials report cost Inspect attempts →
017-partial-rails-rollout v1core 20261027.0.0OpenRouter / high 3/30 unavailable 113s $0.12383/3 trials report cost Inspect attempts →
016-rails-pool-exhaustion v1core 20261027.0.0OpenRouter / high 3/30 unavailable 167s $0.09203/3 trials report cost Inspect attempts →
015-sidekiq-poison-pill v1core 20261027.0.0OpenRouter / high 3/30 unavailable 79s $0.13383/3 trials report cost Inspect attempts →
013-sidekiq-wrong-redis v3core 20261027.0.0OpenRouter / high 3/30 unavailable 61s $0.09783/3 trials report cost Inspect attempts →
001-nginx-502-host v1core 20261027.0.0OpenRouter / high 3/30 unavailable 165s $0.12193/3 trials report cost Inspect attempts →

Compare benchmark configurations → · All models