The public record now includes 14 test methods and their criteria, a 49-entry model catalogue, and 14 additional dated study records. The catalogue consolidates 36 historical artifact variants after merging a duplicate original/rematch entry, then adds later runtime and hardware studies. One entry is the current Studio agent with no local score in the reviewed archive.
These are curated numeric summaries from retained local harnesses, result files and run notes reviewed October 10. Raw model answers, exercise solutions, private logs and configurations remain outside the website. Public leaderboard entries and upstream reproduction scripts are not counted as lab results.
What the percentages mean
The framework gate has eight prompts and 56 pattern criteria. Each prompt earns its own percentage, then the eight percentages are averaged. It does not compile code. The developer tool suite checks expected functions and argument fragments across 12 scenarios without executing those calls. The 34-scenario broad tool suite passes a scenario only when every applicable check passes; its component check groups overlap.
Writing constraints, blind editor scores, screenshot facts and context needles have separate denominators. The methods page publishes the synthetic framework, developer-tool and creative prompts alongside their checks. Historical records retain their dates and protocols.
The July Coder-Next calibration used six framework runs with median 86.4%, mean 86.8% and standard deviation 2.9 points. The ±4-point repeat heuristic was a practical gate rule, not a statistical confidence interval. The promotion criteria explain why area regressions, real latency, memory headroom and rollback matter alongside an overall score.
Aider: complete and incomplete evidence
The Qwen3.8-27B NVFP4 + DFlash2 run started September 26 and recorded all 225 polyglot exercises. It passed 21 on the first try (9.3%) and 112 within two tries (49.8%). Each exercise used whole-file editing and test feedback, with thinking off.
The September MiaAI Flash-Next run has 94 of 225 result files. Among recorded tasks, 28 passed first try and 58 within two tries. The other 131 are unrecorded. We publish observed counts and harness error counters, without a final suite pass rate or a model winner. This arm used thinking on and a different serving configuration.
The recipe specifies 65536 context while model metadata lists 262144 input and 32768 output. Earlier history-budget notes differ, so the effective history cap is not independently frozen. The incomplete September MiaAI result does not establish quality for the current October NVIDIA Flash artifact.
EQ-Bench long-form generation for the 27B arm was also recorded as completed with judging skipped. It supplies no writing-quality score.
Keep unresolved identities visible
An August 3 DeepSeek gate survives under a label suggesting IQ2M, but its exact quant identity was not independently established. Its measured gate stays under an unresolved report entry. The separately verified August 2 UD-IQ2_XXS artifact has throughput and retrieval evidence only; it receives no transferred quality score.
Rejected and superseded variants remain in the catalogue, including OOM-limited partial runs. Missing measurements remain missing. Short timing probes and configured context windows do not become full qualification.
The downloadable methods, model catalogue and dated records use the same curated data as the pages.