Pipeline health · detailed metrics
Trends
Windows end on the latest recorded run (2026-11-01 22:13 UTC).
Pipeline executions171↓ -40 % vs prior 7 d105 moved forward · 44 adverse · 0 failed · 22 no decisionReview approval rate88 %↑ +23 pts vs prior 7 d51 of 58 verdicts · 0 withheld · 7 changes requestedAverage run duration8.5 s↓ -15 % vs prior 7 dp95 16.2 s · 171 run(s)Operational LLM cost (estimate)$3.6126↓ -59 % vs prior 7 d104 call(s) · measured · list price
Pipeline results Executions →
failed adverse decision moved forward skip / no decision
Values
| Day | failed | adverse decision | moved forward | skip / no decision | Total |
|---|---|---|---|---|---|
| 2026-10-26 | 0 | 5 | 20 | 6 | 31 |
| 2026-10-27 | 0 | 3 | 27 | 7 | 37 |
| 2026-10-28 | 0 | 9 | 27 | 7 | 43 |
| 2026-10-29 | 0 | 0 | 3 | 0 | 3 |
| 2026-10-30 | 0 | 7 | 17 | 1 | 25 |
| 2026-10-31 | 0 | 20 | 8 | 1 | 29 |
| 2026-11-01 | 0 | 0 | 3 | 0 | 3 |
Duration trend Metrics →
Averagep95
Values
| Day | Average | p95 |
|---|---|---|
| 2026-10-26 | 7.5 s | 15.9 s |
| 2026-10-27 | 9.5 s | 16.6 s |
| 2026-10-28 | 9.2 s | 15.7 s |
| 2026-10-29 | 3.6 s | 4.9 s |
| 2026-10-30 | 8.7 s | 16.5 s |
| 2026-10-31 | 8.5 s | 12.1 s |
| 2026-11-01 | 0.4 s | 0.5 s |
LLM providers (cost) Models →
- 1anthropic$3.6126100 %97 call(s) · 93 % of calls
- 2groqfree tier0 %7 call(s) · 7 % of calls
List price, never a bill; Groq free tier at $0.
Pipeline executions457↑ +1977 % vs prior 14 d255 moved forward · 136 adverse · 0 failed · 66 no decisionReview approval rate70 %↑ +59 pts vs prior 14 d177 of 253 verdicts · 54 withheld · 22 changes requestedAverage run duration9.5 s↓ -70 % vs prior 14 dp95 16.3 s · 457 run(s)Operational LLM cost (estimate)$12.3658↑ +36463 % vs prior 14 d333 call(s) · measured + estimated · list price
Pipeline results Executions →
failed adverse decision moved forward skip / no decision
Values
| Day | failed | adverse decision | moved forward | skip / no decision | Total |
|---|---|---|---|---|---|
| 2026-10-19 | 0 | 28 | 12 | 3 | 43 |
| 2026-10-20 | 0 | 38 | 17 | 5 | 60 |
| 2026-10-21 | 0 | 13 | 15 | 5 | 33 |
| 2026-10-22 | 0 | 5 | 20 | 6 | 31 |
| 2026-10-23 | 0 | 3 | 19 | 6 | 28 |
| 2026-10-24 | 0 | 0 | 13 | 4 | 17 |
| 2026-10-25 | 0 | 5 | 54 | 15 | 74 |
| 2026-10-26 | 0 | 5 | 20 | 6 | 31 |
| 2026-10-27 | 0 | 3 | 27 | 7 | 37 |
| 2026-10-28 | 0 | 9 | 27 | 7 | 43 |
| 2026-10-29 | 0 | 0 | 3 | 0 | 3 |
| 2026-10-30 | 0 | 7 | 17 | 1 | 25 |
| 2026-10-31 | 0 | 20 | 8 | 1 | 29 |
| 2026-11-01 | 0 | 0 | 3 | 0 | 3 |
Duration trend Metrics →
Averagep95
Values
| Day | Average | p95 |
|---|---|---|
| 2026-10-19 | 10.7 s | 25.9 s |
| 2026-10-20 | 8.9 s | 13.6 s |
| 2026-10-21 | 9.8 s | 17.1 s |
| 2026-10-22 | 10.2 s | 16.4 s |
| 2026-10-23 | 10.6 s | 16.1 s |
| 2026-10-24 | 10.8 s | 20.5 s |
| 2026-10-25 | 10.3 s | 17.5 s |
| 2026-10-26 | 7.5 s | 15.9 s |
| 2026-10-27 | 9.5 s | 16.6 s |
| 2026-10-28 | 9.2 s | 15.7 s |
| 2026-10-29 | 3.6 s | 4.9 s |
| 2026-10-30 | 8.7 s | 16.5 s |
| 2026-10-31 | 8.5 s | 12.1 s |
| 2026-11-01 | 0.4 s | 0.5 s |
LLM providers (cost) Models →
- 1anthropic$12.3658100 %247 call(s) · 74 % of calls
- 2groqfree tier0 %86 call(s) · 26 % of calls
List price, never a bill; Groq free tier at $0.
Pipeline executions479 no prior 30 d data257 moved forward · 155 adverse · 0 failed · 67 no decisionReview approval rate66 % no prior 30 d data179 of 271 verdicts · 67 withheld · 25 changes requestedAverage run duration10.5 s no prior 30 d datap95 16.4 s · 479 run(s)Operational LLM cost (estimate)$12.3996 no prior 30 d data354 call(s) · measured + estimated · list price
Pipeline results Executions →
failed adverse decision moved forward skip / no decision
Values
| Day | failed | adverse decision | moved forward | skip / no decision | Total |
|---|---|---|---|---|---|
| 2026-10-16 | 0 | 3 | 2 | 1 | 6 |
| 2026-10-17 | 0 | 15 | 0 | 0 | 15 |
| 2026-10-18 | 0 | 1 | 0 | 0 | 1 |
| 2026-10-19 | 0 | 28 | 12 | 3 | 43 |
| 2026-10-20 | 0 | 38 | 17 | 5 | 60 |
| 2026-10-21 | 0 | 13 | 15 | 5 | 33 |
| 2026-10-22 | 0 | 5 | 20 | 6 | 31 |
| 2026-10-23 | 0 | 3 | 19 | 6 | 28 |
| 2026-10-24 | 0 | 0 | 13 | 4 | 17 |
| 2026-10-25 | 0 | 5 | 54 | 15 | 74 |
| 2026-10-26 | 0 | 5 | 20 | 6 | 31 |
| 2026-10-27 | 0 | 3 | 27 | 7 | 37 |
| 2026-10-28 | 0 | 9 | 27 | 7 | 43 |
| 2026-10-29 | 0 | 0 | 3 | 0 | 3 |
| 2026-10-30 | 0 | 7 | 17 | 1 | 25 |
| 2026-10-31 | 0 | 20 | 8 | 1 | 29 |
| 2026-11-01 | 0 | 0 | 3 | 0 | 3 |
Duration trend Metrics →
Averagep95
Values
| Day | Average | p95 |
|---|---|---|
| 2026-10-16 | 2 min | 9 min |
| 2026-10-17 | 8.5 s | 36.7 s |
| 2026-10-18 | 5.7 s | 5.7 s |
| 2026-10-19 | 10.7 s | 25.9 s |
| 2026-10-20 | 8.9 s | 13.6 s |
| 2026-10-21 | 9.8 s | 17.1 s |
| 2026-10-22 | 10.2 s | 16.4 s |
| 2026-10-23 | 10.6 s | 16.1 s |
| 2026-10-24 | 10.8 s | 20.5 s |
| 2026-10-25 | 10.3 s | 17.5 s |
| 2026-10-26 | 7.5 s | 15.9 s |
| 2026-10-27 | 9.5 s | 16.6 s |
| 2026-10-28 | 9.2 s | 15.7 s |
| 2026-10-29 | 3.6 s | 4.9 s |
| 2026-10-30 | 8.7 s | 16.5 s |
| 2026-10-31 | 8.5 s | 12.1 s |
| 2026-11-01 | 0.4 s | 0.5 s |
LLM providers (cost) Models →
- 1anthropic$12.3996100 %248 call(s) · 70 % of calls
- 2groqfree tier0 %106 call(s) · 30 % of calls
List price, never a bill; Groq free tier at $0.
Waiting on a person29 / 153Steps recovered automatically20 / 72Lead time per work item4 min · p90 9.2 hWaiting on rate limits12 min
Metric definitions · source, method, period, population, limits
| Indicator | Source | Method | Period | Population | Known limits |
|---|---|---|---|---|---|
| Failed executions | Redacted run records on the `metrics` branch (ADR-0040) | Runs whose recorded outcome is `failed`, plus runs with a stage span closed as failed and no decision event explaining it | Retained window: the newest 6 months of records | Every loaded run record | A run that crashed before its tracer wrote a record is absent (ADR-0040). `partial` is a business decision, not a failure. |
| Runs with an adverse decision | Redacted run records on the `metrics` branch (ADR-0040) | Runs whose step decided not to move the item forward: WITHHELD or REQUEST_CHANGES review, rejected validation, blocked generation, auto-fix without a push | Retained window: the newest 6 months of records | Every loaded run record | An adverse decision can be correct (a real defect); it measures friction, not errors. |
| Review approval rate | Redacted run records on the `metrics` branch (ADR-0040), `review.verdict` events | APPROVE verdicts ÷ all recorded verdicts (skipped reviews excluded) | Retained window: the newest 6 months of records | Review runs that recorded a verdict | Each review run counts, so a PR reviewed several times weighs more. Not a merge rate. |
| Work items needing a person | Trace indexes on the `metrics` branch (ADR-0041) joined to the run records | Work items whose last recorded state is failed, escalated (needs-human), approval withheld or rejected by validation | Retained window: the newest 6 months of records | Work items with a trace index and at least one loaded run record | The current label state on GitHub is not recorded: a person may already have acted. Closings and merges are journaled (item.closed) only since item-closed.yml existed; before that a merged PR is only known live (Pipelines page). |
| Steps recovered automatically | Trace indexes on the `metrics` branch (ADR-0041) joined to the run records | Steps whose final attempt moved the item forward after at least one failed or adverse attempt | Retained window: the newest 6 months of records | Steps with at least one failed or adverse attempt | Recovery within the retained window only. |
| Lead time per work item | Trace indexes on the `metrics` branch (ADR-0041) joined to the run records | First run start → last run end of the item; median and 90th percentile (nearest rank) | Retained window: the newest 6 months of records | Work items with a lead time and at least one loaded run record | Observed duration, not the item lifetime: time before the first run and after the last one (human merge) is not recorded. An item whose first runs predate the window keeps its full indexed lead time. Runner clocks differ (ADR-0041 §6). |
| Share of lead time spent running | Trace indexes on the `metrics` branch (ADR-0041) joined to the run records | Σ time inside runs ÷ Σ lead time; the rest is waiting between runs | Retained window: the newest 6 months of records | Work items with a lead time | Approximate: runs are placed by their own runner clocks. |
| Review runs per PR | Redacted run records on the `metrics` branch (ADR-0040) | Review runs that recorded a verdict ÷ PRs with at least one verdict | Retained window: the newest 6 months of records | PRs with a recorded verdict | Counts duplicate triggers (push + pull_request on one head) as separate runs. |
| Time waiting on rate limits | Redacted run records on the `metrics` branch (ADR-0040), `retry.wait` events with status 429 | Σ wait_ms of 429 retry waits | Retained window: the newest 6 months of records | Every loaded run record | Waits logged by the retry helper only; provider queueing is not visible. |
| Operational LLM cost (estimate) | Redacted run records on the `metrics` branch (ADR-0040), `llm_call` spans priced by `llm_cost.mjs` | List price × tokens (measured where the record has usage, chars/4 estimate otherwise); Groq free tier at $0; the change against the previous window only when neither side has an unpriced call | Retained window: the newest 6 months of records | Every recorded pipeline LLM call (eval runs excluded: Health, Eval LLM cost) | List price, never a bill. Unpriced calls are counted apart, not as $0. |
| Pipeline executions | Redacted run records on the `metrics` branch (ADR-0040) | Runs started in the window, by class: moved forward, adverse decision, failed, skip / no decision; the change is against the same number of days just before | The selected window (7, 14 or 30 days) ending on the latest recorded run | Runs with a start time | A run in progress is not recorded yet, so there is no running count. |
| Average run duration | Redacted run records on the `metrics` branch (ADR-0040) | Mean of run durations (start → end of the workflow job record) in the window; p95 by nearest rank | The selected window | Runs with both timestamps | Job time only: queueing before the job and waiting between runs are not included (see lead time). |
| LLM providers | Redacted run records on the `metrics` branch (ADR-0040), `llm_call` spans | Calls and list-price cost per provider in the window; shares of the window total | The selected window | Recorded LLM calls | Groq free-tier calls cost $0: their share of cost is 0 while their share of calls can be large. |
| Pipeline stages | Redacted run records on the `metrics` branch (ADR-0040), the decision event of each step (`pipeline_journey.mjs` `stepDecision`) | Per step, the runs of the window that recorded its decision, per UTC day: Issue = validations (passed / rejected), Code = PRs opened or updated by PR preparation (patches generated beside it), Review = the approval rate of the verdicts, Auto-fix = PRs with a fix pushed, Merge = work items whose latest recorded state is merged (the Pipelines filter it opens) | The last 7 days ending on the latest recorded run | Runs with a start time and a decision of that step; for Merge, work items active in the period | A run counts once per step (its last decision). Merge bars are 24 h slices ending on the latest recorded run, closings included (the reference of the count); the other bars are UTC days. Merges are journaled only since item-closed.yml exists; an earlier merge is not counted. |
| Runs per day | Redacted run records on the `metrics` branch (ADR-0040) | Runs per UTC day by class (failed, adverse decision, moved forward, skipped / no decision), last 14 days ending on the latest recorded run; the 7-day comparison sets the latest 7 days against the 7 before | 14 days ending on the latest recorded run (not the build time) | Runs with a start time | Small daily counts: a rate on a few runs is noisy; n is shown next to every rate. |
556 run record(s), 153 work-item trace(s), 2026-10 → 2026-11. Token counts are provider-measured where the record has them (ADR-0041 amendment), chars/4 estimates for older records.
Operational LLM cost · per stage, run & work item
Operational LLM cost
LLM cost
$12.3996
LLM calls
354
Input tokens
6.6M
of which cache reads
520.6k
Output tokens
227.9k
Mean per run
$0.0350
Mean per work item
$0.1039
provider-measured tokens where the record has them, chars/4 estimates for older records; list price (PRICES_USD_PER_MTOK), never a bill; 106 Groq free-tier call(s) at $0. The mean per work item counts work items with a trace index only. Pipeline runs only: the cost of the eval runs is the next panel, never added here.
Per stage
| Stage | Runs | Calls | Input | Cache reads | Output | Cost | Per run |
|---|---|---|---|---|---|---|---|
issue_validation | 45 | 45 | 256.6k | 107.5k | 83.6k | $0.0560 USD (6 Groq free-tier call(s) at no cost) | $0.0012 |
code_gen | 18 | 18 | 95.4k | 0 | 11.8k | $0.2466 USD (6 Groq free-tier call(s) at no cost) | $0.0137 |
review | 271 | 271 | 6.2M | 413.2k | 126.5k | $12.0970 USD (74 Groq free-tier call(s) at no cost) | $0.0446 |
autofix | 20 | 20 | 48.9k | 0 | 6.0k | $0.0000 USD (20 Groq free-tier call(s) at no cost) | free tier |
Per workflow
| Workflow | Runs | Calls | Input | Cache reads | Output | Cost | Per run |
|---|---|---|---|---|---|---|---|
| PR Review | 271 | 271 | 6.2M | 413.2k | 126.5k | $12.0970 USD (74 Groq free-tier call(s) at no cost) | $0.0446 |
| Code Generation | 18 | 18 | 95.4k | 0 | 11.8k | $0.2466 USD (6 Groq free-tier call(s) at no cost) | $0.0137 |
| Issue Validation Agent | 45 | 45 | 256.6k | 107.5k | 83.6k | $0.0560 USD (6 Groq free-tier call(s) at no cost) | $0.0012 |
| Auto-Fix PR | 20 | 20 | 48.9k | 0 | 6.0k | $0.0000 USD (20 Groq free-tier call(s) at no cost) | free tier |
Most expensive runs
Ranked on fully priced runs only; a run with an unpriced call has an incomplete cost and is listed apart.
| Run | Workflow | Work item | Calls | Input | Cache reads | Output | Cost |
|---|---|---|---|---|---|---|---|
| 40158408506-1 2026-10-25 03:31:09 | PR Review | PR #294 | 1 | 101.7k | 0 | 529 | $0.2107 USD |
| 40157927330-1 2026-10-25 03:23:11 | PR Review | PR #294 | 1 | 101.0k | 0 | 505 | $0.2089 USD |
| 40158687452-1 2026-10-25 03:35:27 | PR Review | PR #294 | 1 | 100.6k | 3.9k | 693 | $0.2007 USD |
| 39890028296-1 2026-10-23 02:32:40 | PR Review | PR #276 | 1 | 98.2k | 4.4k | 713 | $0.1953 USD |
| 39889793048-1 2026-10-23 02:30:14 | PR Review | PR #276 | 1 | 98.2k | 4.4k | 616 | $0.1943 USD |
| 40159194422-1 2026-10-25 03:43:51 | PR Review | PR #294 | 1 | 95.0k | 3.9k | 742 | $0.1900 USD |
| 40198235010-1 2026-10-25 13:35:48 | PR Review | PR #296 | 1 | 70.5k | 3.9k | 847 | $0.1421 USD |
| 40197101399-1 2026-10-25 13:21:18 | PR Review | PR #296 | 1 | 64.9k | 0 | 673 | $0.1386 USD |
| 40197878790-1 2026-10-25 13:31:15 | PR Review | PR #296 | 1 | 69.2k | 3.9k | 743 | $0.1385 USD |
| 40627563514-1 2026-10-28 03:37:11 | PR Review | PR #314 | 1 | 65.5k | 0 | 428 | $0.1372 USD |
Most expensive work items
| Work item | Runs | Calls | Input | Cache reads | Output | Cost |
|---|---|---|---|---|---|---|
| PR #296 | 23 | 14 | 804.3k | 43.2k | 11.5k | $1.6475 USD |
| PR #294 | 5 | 4 | 398.3k | 7.8k | 2.5k | $0.8103 USD |
| PR #283 | 10 | 9 | 278.4k | 22.7k | 5.8k | $0.5778 USD |
| PR #314 | 10 | 4 | 256.9k | 12.3k | 2.4k | $0.5165 USD |
| PR #246 | 7 | 7 | 199.5k | 12.5k | 3.4k | $0.3923 USD (2 Groq free-tier call(s) at no cost) |
| PR #276 | 2 | 2 | 196.4k | 8.7k | 1.3k | $0.3895 USD |
| PR #319 | 11 | 6 | 183.8k | 16.4k | 4.1k | $0.3817 USD |
| PR #254 | 5 | 5 | 175.9k | 8.4k | 2.1k | $0.3438 USD (2 Groq free-tier call(s) at no cost) |
| PR #313 | 7 | 5 | 146.6k | 12.3k | 3.4k | $0.3081 USD |
| PR #290 | 4 | 3 | 124.2k | 0 | 2.4k | $0.2785 USD |
Per day (UTC)
| Day | Runs | Calls | Input | Cache reads | Output | Cost |
|---|---|---|---|---|---|---|
| 2026-10-31 | 24 | 24 | 153.0k | 76.7k | 46.5k | $0.0958 USD (1 Groq free-tier call(s) at no cost) |
| 2026-10-30 | 14 | 14 | 178.4k | 12.2k | 17.2k | $0.3355 USD (1 Groq free-tier call(s) at no cost) |
| 2026-10-28 | 29 | 29 | 559.3k | 40.3k | 30.2k | $1.1468 USD (3 Groq free-tier call(s) at no cost) |
| 2026-10-27 | 22 | 22 | 586.2k | 61.4k | 13.4k | $1.1947 USD (1 Groq free-tier call(s) at no cost) |
| 2026-10-26 | 15 | 15 | 399.1k | 27.7k | 8.7k | $0.8399 USD (1 Groq free-tier call(s) at no cost) |
| 2026-10-25 | 46 | 46 | 1.7M | 125.3k | 31.2k | $3.5097 USD (2 Groq free-tier call(s) at no cost) |
| 2026-10-24 | 13 | 13 | 463.5k | 41.6k | 7.1k | $0.9230 USD |
| 2026-10-23 | 22 | 22 | 553.9k | 35.8k | 12.5k | $1.1631 USD |
| 2026-10-22 | 25 | 25 | 364.1k | 38.4k | 11.7k | $0.7759 USD (3 Groq free-tier call(s) at no cost) |
| 2026-10-21 | 28 | 28 | 645.6k | 36.3k | 19.8k | $1.2889 USD (6 Groq free-tier call(s) at no cost) |
| 2026-10-20 | 55 | 55 | 653.2k | 25.0k | 18.6k | $1.0298 USD (32 Groq free-tier call(s) at no cost) |
| 2026-10-19 | 40 | 40 | 219.2k | 0 | 7.4k | $0.0629 USD (36 Groq free-tier call(s) at no cost) |
| 2026-10-18 | 1 | 1 | 6.3k | 0 | 130 | $0.0000 USD (1 Groq free-tier call(s) at no cost) |
| 2026-10-17 | 15 | 15 | 82.9k | 0 | 2.5k | $0.0000 USD (15 Groq free-tier call(s) at no cost) |
| 2026-10-16 | 5 | 5 | 31.5k | 0 | 966 | $0.0338 USD (4 Groq free-tier call(s) at no cost) |
Eval LLM cost · per suite & run
Eval LLM cost
Eval cost
$2.8310
Eval runs
6
Priced
2
Free tier
4
Unpriced
0
Input tokens (est.)
1.4M
Output tokens (est.)
114.2k
Live eval runs of the scorecard, all history it keeps (not the pipeline window), apart from the operational cost above. Chars/4 harness estimates at list price (PRICES_USD_PER_MTOK): input ignores the prompt cache, output excludes thinking, never a bill. Groq runs are free tier; a run whose model has no price is counted as unpriced, not as $0.
Per suite
| Suite | Runs | Input (est.) | Output (est.) | Cost |
|---|---|---|---|---|
review | 2 | 492.1k | 43.1k | $2.8310 |
validation | 4 | 918.8k | 71.1k | free tier |
Most expensive eval runs
| Suite | Run | Model | Cost |
|---|---|---|---|
review | 38994398984-review | anthropic:claude-opus-5-5 | $2.1180 |
review | 39000563980-review | anthropic:claude-opus-5-5 | $0.7130 |
Delivery funnel · progress through the loop
Funnel
Issue work items that reached each stage (each stage counts items that also reached the previous one). PRs without an issue: 132, 82 approved.
| Stage | Work items |
|---|---|
| Issues validated | 17 |
| PR opened | 10 |
| Approved | 7 |
| Merged | not recorded |
Review decisions · approvals & changes needed
Reviews
WITHHELD rate
24.7 %
67 of 271 verdicts
APPROVE
179
REQUEST_CHANGES
25
REQUEST_CHANGES → WITHHELD (#220)
0
unseen-code backstop
Reviews of a cut diff
84
66 withheld (ADR-0038)
WITHHELD causes
| Cause | Verdicts |
|---|---|
| Approval withheld: code cut from the reviewed diff (ADR-0038) | 55 |
| REQUEST_CHANGES withheld: every finding cites only unseen code (#220) | 0 |
| Code cut from the diff, pre-backstop verdict not recorded (record before withheld_from) | 11 |
| Approval withheld: adversarial analysis incomplete (ADR-0049) | 1 |
| CI evidence missing (ADR-0026) | 0 |
| CI evidence stale (ADR-0026) | 0 |
| Evidence unverified / other | 0 |
Review cycles per PR
| PR | Review cycles |
|---|---|
| #331 | 1 |
| #329 | 1 |
| #328 | 2 |
| #327 | 1 |
| #326 | 1 |
| #322 | 5 |
| #320 | 3 |
| #319 | 6 |
| #318 | 1 |
| #317 | 4 |
| #316 | 3 |
| #315 | 1 |
| #314 | 4 |
| #313 | 5 |
| #312 | 5 |
| #311 | 2 |
| #310 | 3 |
| #309 | 1 |
| #308 | 4 |
| #307 | 2 |
| #306 | 2 |
| #305 | 1 |
| #304 | 2 |
| #303 | 1 |
| #302 | 2 |
| #301 | 2 |
| #300 | 1 |
| #299 | 2 |
| #298 | 2 |
| #297 | 3 |
Auto-fix outcomes · fixes, skips & guardrails
Auto-fix outcomes
| Guardrail rule | Rejections |
|---|---|
protected_path | 4 |
unshown_file | 1 |
2 outcome(s) from legacy runs.jsonl lines whose run has no record.
Providers · failures, fallbacks & rate limits
LLM providers
LLM calls
354
Fallbacks
18
llm_fallback
No provider answered
0
llm_call_failed
Fallback disabled
0
Rate-limit waits
28
retry.wait on 429
Circuit-breaker trips
not recorded
eval runs only, see note
The circuit breaker is an eval-harness mechanism: a tripped eval run fails error_rate, is never added to the scorecard, and eval traces stay workflow artifacts (no record on the metrics branch). Its trips are in the Evals workflow logs (eval.error, reason: provider_quota).
Performance · latency & critical path
Latency per stage
| Stage | Runs | p50 | p95 |
|---|---|---|---|
autofix | 21 | 3.9 s | 5.1 s |
code_gen | 18 | 7.7 s | 13.3 s |
conflicts_detect | 57 | 2.5 s | 14.1 s |
issue_validation | 45 | 11.2 s | 25.9 s |
pr_prepare | 1 | 6 ms | 6 ms |
review | 338 | 11.8 s | 16.5 s |
Critical path per work item
Lead time (first start → last end) split into time spent in runs and time between them; runner clocks differ, so this is approximate (ADR-0041 §6). Longest 25.
runningwaiting between runs
| Work item | Runs | Lead time | Running | Waiting |
|---|---|---|---|---|
| PR #314 | 10 | 2 d 23 h | 1 min 19 s | 2 d 23 h |
| PR #319 | 11 | 2 d 15 h | 1 min 39 s | 2 d 15 h |
| issue #321 | 15 | 1 d 23 h | 2 min 18 s | 1 d 23 h |
| issue #324 | 7 | 1 d 23 h | 46.6 s | 1 d 23 h |
| issue #325 | 7 | 1 d 23 h | 53.3 s | 1 d 23 h |
| issue #323 | 6 | 1 d 23 h | 29.4 s | 1 d 23 h |
| PR #328 | 7 | 1 d 20 h | 26.0 s | 1 d 20 h |
| PR #310 | 8 | 1 d 16 h | 50.0 s | 1 d 16 h |
| PR #311 | 8 | 1 d 3 h | 42.2 s | 1 d 3 h |
| PR #283 | 10 | 1 d 0 h | 2 min 09 s | 1 d 0 h |
| PR #296 | 23 | 20 h 36 min | 4 min 11 s | 20 h 32 min |
| issue #150 | 3 | 12 h 28 min | 20.1 s | 12 h 28 min |
| issue #211 | 3 | 12 h 24 min | 1 min 55 s | 12 h 22 min |
| issue #209 | 4 | 12 h 09 min | 43.2 s | 12 h 09 min |
| PR #222 | 3 | 11 h 18 min | 49.6 s | 11 h 17 min |
| PR #295 | 5 | 9 h 12 min | 54.1 s | 9 h 11 min |
| PR #297 | 10 | 9 h 11 min | 1 min 26 s | 9 h 09 min |
| PR #241 | 11 | 8 h 21 min | 1 min 29 s | 8 h 19 min |
| issue #330 | 8 | 7 h 35 min | 1 min 36 s | 7 h 34 min |
| PR #226 | 5 | 7 h 23 min | 27.6 s | 7 h 23 min |
| PR #289 | 3 | 6 h 45 min | 13.3 s | 6 h 45 min |
| PR #316 | 7 | 5 h 32 min | 44.2 s | 5 h 31 min |
| PR #317 | 7 | 5 h 13 min | 1 min 04 s | 5 h 12 min |
| PR #318 | 4 | 4 h 09 min | 16.9 s | 4 h 09 min |
| issue #284 | 4 | 4 h 06 min | 19.6 s | 4 h 06 min |
Token usage · daily estimates & quotas
Groq tokens per day
Tokens (in + out) of Groq calls per UTC day over 30 days, pipeline runs by workflow and eval runs from the scorecard, against the 200.0k tokens-per-day free-tier limit. Pipeline calls recorded since the ADR-0041 amendment carry Groq's measured counts; older records and eval runs are chars/4 estimates (the tooltip gives the estimated share per day).
PR ReviewAuto-Fix PREvalsOther▲ V / R: weekly eval cron (validation / review)
| Day | Auto-Fix PR | Code Generation | Evals (validation) | Issue Validation Agent | PR Review | Total | % of TPD | Eval cron |
|---|---|---|---|---|---|---|---|---|
| 2026-10-04 | 0 | 0 | 94.9k | 0 | 0 | 94.9k | 47.5 % | — |
| 2026-10-05 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 % | validation |
| 2026-10-07 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 % | review |
| 2026-10-12 | 0 | 0 | 304.6k | 0 | 0 | 304.6k | 152.3 % | validation |
| 2026-10-14 | 0 | 0 | 0 | 0 | 0 | 0 | 0.0 % | review |
| 2026-10-16 | 0 | 0 | 295.3k | 0 | 25.7k | 321.0k | 160.5 % | — |
| 2026-10-17 | 7.8k | 0 | 0 | 0 | 77.5k | 85.4k | 42.7 % | — |
| 2026-10-18 | 0 | 0 | 0 | 0 | 6.4k | 6.4k | 3.2 % | — |
| 2026-10-19 | 2.9k | 2.8k | 295.0k | 21.7k | 174.3k | 496.8k | 248.4 % | validation |
| 2026-10-20 | 6.2k | 3.3k | 0 | 0 | 175.8k | 185.4k | 92.7 % | — |
| 2026-10-21 | 12.7k | 4.0k | 0 | 0 | 0 | 16.7k | 8.3 % | review |
| 2026-10-22 | 7.4k | 0 | 0 | 0 | 0 | 7.4k | 3.7 % | — |
| 2026-10-25 | 6.0k | 0 | 0 | 0 | 0 | 6.0k | 3.0 % | — |
| 2026-10-26 | 2.9k | 0 | 0 | 0 | 0 | 2.9k | 1.5 % | validation |
| 2026-10-27 | 2.7k | 0 | 0 | 0 | 0 | 2.7k | 1.3 % | — |
| 2026-10-28 | 3.1k | 6.2k | 0 | 0 | 0 | 9.3k | 4.7 % | review |
| 2026-10-30 | 3.1k | 0 | 0 | 0 | 0 | 3.1k | 1.5 % | — |
| 2026-10-31 | 0 | 5.3k | 0 | 0 | 0 | 5.3k | 2.6 % | — |