nightly loop · last read 2026-10-18 15:36:19 UTC
The nightly loop.
Every night a candidate is registered, run against a fixed suite, canaried against the incumbent, then promoted or rolled back. Every artifact of every night is committed to a public repository. This page reads those files.
10 last 10 days
0 candidates accepted
10 candidates rejected
0.70 min pass rate
$0 actual, public repo CPU minutes
last night
The most recent decision
2026-10-18 · rolled back
gate failed on pass_rate (pass_rate = 0.55)
gate 11 of 20 cases passed, pass rate 0.55, threshold 0.70 · p50 2,103 ms · p95 6,242 ms · verdict fail on pass_rate
canary 6 fixed cases, candidate quality 0.50 against incumbent 0.50 · verdict pass
serving after 18d11cde8a4e
artifacts runs/2026-10-18
the record
Every night in the window
| date | candidate variant | gate | canary | action | candidate | serving after |
|---|---|---|---|---|---|---|
| 2026-10-18 | candidate not scheduled to degrade |
✕ fail | ✓ pass | rollback | 0867e8775c40 | 18d11cde8a4e |
| 2026-10-17 | drift-truncate scheduled to degrade |
✕ fail | ✕ fail | rollback | b024ae0f4ce5 | 18d11cde8a4e |
| 2026-10-16 | candidate not scheduled to degrade |
✕ fail | ✓ pass | rollback | 0867e8775c40 | 18d11cde8a4e |
| 2026-10-15 | candidate not scheduled to degrade |
✕ fail | ✓ pass | rollback | 0867e8775c40 | 18d11cde8a4e |
| 2026-10-14 | drift-truncate scheduled to degrade |
✕ fail | ✕ fail | rollback | b024ae0f4ce5 | 18d11cde8a4e |
| 2026-10-13 | candidate not scheduled to degrade |
✕ fail | ✓ pass | rollback | 0867e8775c40 | 18d11cde8a4e |
| 2026-10-12 | not read | ✕ fail | ✕ fail | rollback | 18d11cde8a4e | 18d11cde8a4e |
| 2026-10-11 | drift-truncate scheduled to degrade |
✕ fail | ✕ fail | rollback | b024ae0f4ce5 | 18d11cde8a4e |
| 2026-10-10 | candidate not scheduled to degrade |
✕ fail | ✕ fail | rollback | 0867e8775c40 | 18d11cde8a4e |
| 2026-10-09 | candidate not scheduled to degrade |
✕ fail | ✕ fail | rollback | 0867e8775c40 | 18d11cde8a4e |
Counted from runs/<date>/summary.json, one file per night. The variant column is read from registry/versions/<hash>.json for the 6 most recent nights, so a night older than that shows "not read" rather than a guess.
Serving now: 18d11cde8a4e
variant drift-hot
model qwen2.5-0.5b-instruct-q4_k_m
No candidate has been promoted in the last 10 days. This version is what the registry serves because nothing has beaten it, not because it won a promotion in this window.
registry history as committed
| from | version | action |
|---|---|---|
| 2026-08-13 | 0867e8775c40 | rollback |
| 2026-08-13 | 0867e8775c40 | rollback |
| 2026-08-14 | b024ae0f4ce5 | rollback |
| 2026-08-15 | 0867e8775c40 | rollback |
| 2026-08-16 | 0867e8775c40 | rollback |
| 2026-08-17 | b024ae0f4ce5 | rollback |
| 2026-08-18 | 0867e8775c40 | rollback |
| 2026-08-19 | 18d11cde8a4e | rollback |
| 2026-08-20 | b024ae0f4ce5 | rollback |
| 2026-08-21 | 0867e8775c40 | rollback |
| 2026-08-22 | 0867e8775c40 | rollback |
| 2026-08-23 | b024ae0f4ce5 | rollback |
| 2026-08-24 | 0867e8775c40 | rollback |
| 2026-08-25 | 0867e8775c40 | rollback |
| 2026-08-26 | 18d11cde8a4e | promote |
| 2026-08-27 | 0867e8775c40 | rollback |
| 2026-08-28 | 0867e8775c40 | rollback |
| 2026-08-29 | b024ae0f4ce5 | rollback |
| 2026-08-30 | 0867e8775c40 | rollback |
| 2026-08-31 | 0867e8775c40 | rollback |
| 2026-09-01 | 0867e8775c40 | rollback |
| 2026-09-02 | b024ae0f4ce5 | rollback |
| 2026-09-03 | 0867e8775c40 | rollback |
| 2026-09-04 | 0867e8775c40 | rollback |
| 2026-09-05 | 18d11cde8a4e | rollback |
| 2026-09-06 | 0867e8775c40 | rollback |
| 2026-09-07 | 0867e8775c40 | rollback |
| 2026-09-08 | b024ae0f4ce5 | rollback |
| 2026-09-09 | 0867e8775c40 | rollback |
| 2026-09-10 | 0867e8775c40 | rollback |
| 2026-09-11 | b024ae0f4ce5 | rollback |
| 2026-09-12 | 18d11cde8a4e | rollback |
| 2026-09-13 | 0867e8775c40 | rollback |
| 2026-09-14 | b024ae0f4ce5 | rollback |
| 2026-09-15 | 0867e8775c40 | rollback |
| 2026-09-16 | 0867e8775c40 | rollback |
| 2026-09-17 | b024ae0f4ce5 | rollback |
| 2026-09-18 | 0867e8775c40 | rollback |
| 2026-09-19 | 18d11cde8a4e | rollback |
| 2026-09-20 | b024ae0f4ce5 | rollback |
| 2026-09-21 | 0867e8775c40 | rollback |
| 2026-09-22 | 0867e8775c40 | rollback |
| 2026-09-23 | b024ae0f4ce5 | rollback |
| 2026-09-24 | 0867e8775c40 | rollback |
| 2026-09-25 | 0867e8775c40 | rollback |
| 2026-09-26 | 18d11cde8a4e | rollback |
| 2026-09-27 | 0867e8775c40 | rollback |
| 2026-09-28 | 0867e8775c40 | rollback |
| 2026-09-29 | b024ae0f4ce5 | rollback |
| 2026-09-30 | 0867e8775c40 | rollback |
| 2026-10-01 | 0867e8775c40 | rollback |
| 2026-10-02 | b024ae0f4ce5 | rollback |
| 2026-10-03 | 0867e8775c40 | rollback |
| 2026-10-04 | 0867e8775c40 | rollback |
| 2026-10-05 | 18d11cde8a4e | rollback |
| 2026-10-06 | 0867e8775c40 | rollback |
| 2026-10-07 | 0867e8775c40 | rollback |
| 2026-10-08 | b024ae0f4ce5 | rollback |
| 2026-10-09 | 0867e8775c40 | rollback |
| 2026-10-10 | 0867e8775c40 | rollback |
| 2026-10-11 | b024ae0f4ce5 | rollback |
| 2026-10-12 | 18d11cde8a4e | rollback |
| 2026-10-13 | 0867e8775c40 | rollback |
| 2026-10-14 | b024ae0f4ce5 | rollback |
| 2026-10-15 | 0867e8775c40 | rollback |
| 2026-10-16 | 0867e8775c40 | rollback |
| 2026-10-17 | b024ae0f4ce5 | rollback |
| 2026-10-18 | 0867e8775c40 | rollback |
This table is printed exactly as registry/serving.json holds it, including a duplicated entry. The counts above are taken from the per night summaries instead, which have one file per night.
rollback events
10 rollbacks in the window
2026-10-18 · 0867e8775c40
Gate fail, canary pass. Variant candidate: this candidate was not scheduled to degrade. It failed the gate anyway and was rejected. Serving stayed on 18d11cde8a4e.
2026-10-17 · b024ae0f4ce5
Gate fail, canary fail. Variant drift-truncate: this candidate was scheduled to degrade, on a schedule published in the repository. The loop caught it. Serving stayed on 18d11cde8a4e.
2026-10-16 · 0867e8775c40
Gate fail, canary pass. Variant candidate: this candidate was not scheduled to degrade. It failed the gate anyway and was rejected. Serving stayed on 18d11cde8a4e.
2026-10-15 · 0867e8775c40
Gate fail, canary pass. Variant candidate: this candidate was not scheduled to degrade. It failed the gate anyway and was rejected. Serving stayed on 18d11cde8a4e.
2026-10-14 · b024ae0f4ce5
Gate fail, canary fail. Variant drift-truncate: this candidate was scheduled to degrade, on a schedule published in the repository. The loop caught it. Serving stayed on 18d11cde8a4e.
2026-10-13 · 0867e8775c40
Gate fail, canary pass. Variant candidate: this candidate was not scheduled to degrade. It failed the gate anyway and was rejected. Serving stayed on 18d11cde8a4e.
2026-10-12 · 18d11cde8a4e
Gate fail, canary fail. Variant not read for this night. Serving stayed on 18d11cde8a4e.
2026-10-11 · b024ae0f4ce5
Gate fail, canary fail. Variant drift-truncate: this candidate was scheduled to degrade, on a schedule published in the repository. The loop caught it. Serving stayed on 18d11cde8a4e.
2026-10-10 · 0867e8775c40
Gate fail, canary fail. Variant candidate: this candidate was not scheduled to degrade. It failed the gate anyway and was rejected. Serving stayed on 18d11cde8a4e.
2026-10-09 · 0867e8775c40
Gate fail, canary fail. Variant candidate: this candidate was not scheduled to degrade. It failed the gate anyway and was rejected. Serving stayed on 18d11cde8a4e.
disclosure
What this does not show
The claim under test is the loop, not the model
- The agent is qwen2.5-0.5b-instruct-q4_k_m on llama.cpp, on CPU, inside GitHub Actions. It is small on purpose. Nothing here says this model is good.
- Scoring is mechanical: exact match and schema validity. No model grades another model on this page.
- The drift schedule degrades the candidate on purpose, on a schedule published in the repository. Day of month mod 7 equal to 5 raises temperature to 1.4. Day of month mod 3 equal to 2 cuts the prompt template to 60 percent of its length. Every other day gets a benign paraphrase. The variant column above is read from the committed version artifact, so a night that was scheduled to degrade and a night that was not are labelled differently.
- A rollback is a successful run of the loop. The pipeline exits zero on a caught failure, because catching it is the job. A nonzero exit means infrastructure broke, not that a policy fired.
- The cost estimate in each gate file counts generated tokens only against a reference price. It is not a bill. Actual spend is zero because the repository is public and the minutes are free.
Where this runs
The loop runs in GitHub Actions on a schedule and commits every artifact of every night to a public repository. This page reads those files from the raw file host and caches them for 15 minutes. Every number here has a URL you can open yourself.
Machine readable: /live/status.json. Source of truth: https://github.com/coconut-labs/atlas-pipeline.