Local evidencePrevious coder; rollback preserved
GLM-5.3-Flash · production control
EXL3 · abliterated serving checkpoint
2026-10-22
TensorFold native · TP2 · primary Hermes handoff panel
DGX Spark + ASUS GX10 · two 128 GB GB10 hosts
Matched Hermes panel: routine 1/1, hard 0/3 for both models. Correct admission and healthy-review decisions, but incorrect approval of the defective-change review. Do not pool with the earlier GLM speed artifact.
- routine passed
- 1
- routine tasks
- 1
- hard passed
- 0
- hard tasks
- 3
All results, context & sources
- routine passed
- 1
- routine tasks
- 1
- hard passed
- 0
- hard tasks
- 3
- defective review correct
- 0
Context: Frozen local fixture context only; not long-context qualification.
Limits: One routine coding task and three hard coding tasks, one visit each, one harness. Not official SWE-bench Verified. Normal DeepSeek versus abliterated GLM. Different dates, tools, cache effects and production tuning prevent a general quality or speed ranking.
Dated exact-identity matched primary Hermes receipts · October 22–23, 2026; current DeepSeek settings independently inspected October 23.
Local evidenceActive deployment; hard-coding gain unproven
DeepSeek V4.1 Flash 1017
EXL3 EAQ 2.9 bpw · normal · ABLIT=0
2026-10-23
TensorFold native · TP2 · primary Hermes handoff panel
DGX Spark + ASUS GX10 · two 128 GB GB10 hosts
Matched Hermes panel: routine 1/1, hard 0/3 for both models. DeepSeek correctly rejected the defective-change review that GLM approved. Production high-reasoning tuning is separate from this thinking-off panel.
- routine passed
- 1
- routine tasks
- 1
- hard passed
- 0
- hard tasks
- 3
All results, context & sources
- routine passed
- 1
- routine tasks
- 1
- hard passed
- 0
- hard tasks
- 3
- defective review correct
- 1
Context: 524,288 configured; local 21,875-position validation and separate 63,164-position history replay do not qualify full-window retrieval.
Limits: One routine coding task and three hard coding tasks, one visit each, one harness. Not official SWE-bench Verified. Normal DeepSeek versus abliterated GLM. Different dates, tools, cache effects and production tuning prevent a general quality or speed ranking.
Dated exact-identity matched primary Hermes receipts · October 22–23, 2026; current DeepSeek settings independently inspected October 23.
Local evidenceRetired
Cydonia 24B v4.3
Q6_K
Historical two-round study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Strong prose in the original qualitative review, with dense decode around 10 tok/s.
- reported decode approx
- 10tok/s
All results, context & sources
- reported decode approx
- 10 tok/s
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceIdentity unresolved
DeepSeek-V4-Flash · unconfirmed quant
Retained report label ds4-iq2m
Aug 3, 2026
Runtime/artifact revision not independently frozen
Historical lab pool
Keep the measured report without attributing its quality to the verified August 2 IQ2_XXS artifact.
- framework
- 84.8%
- developer tools
- 41.7%
- single stream
- 12.4tok/s
- 4-stream aggregate
- 16.5tok/s
All results, context & sources
- framework
- 84.8 %
- developer tools
- 41.7 %
- single stream
- 12.4 tok/s
- 4-stream aggregate
- 16.5 tok/s
Context: Not qualified by this gate.
Limits: The retained label is not proof of quant identity. Scores belong to this unresolved report only.
Dated public study and retained numeric archive; reviewed October 10, 2026.
Local evidenceRejected
DeepSeek-V4-Flash-0731
UD-IQ2_XXS GGUF
Aug 2, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
One-Spark load and 262K retrieval were demonstrated, but decode and prefill were too slow for the primary coder. Quality was not measured for this verified quant.
- decode low
- 16.6tok/s
- decode high
- 17.8tok/s
- prefill
- 530tok/s
- prefill input positions
- 7,000
All results, context & sources
- decode low
- 16.6 tok/s
- decode high
- 17.8 tok/s
- prefill
- 530 tok/s
- prefill input positions
- 7,000
- weight
- 85 GiB
- available memory floor
- 31.1 GiB
- cold load
- 376 s
Context: 262144 retrieval verified on one GB10 host; this quant has throughput evidence only.
Limits: The August 3 gate has an unresolved quant identity and is listed separately. Do not assign its quality score to this artifact.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Empero Qwythos-27B-v1
Q4_K_M
Jul 29, 2026
llama.cpp · revision not frozen in registry
Historical GB10 / lab pool; use dated record for exact host
Matched base A/B showed 84.6% versus 84.3% framework and 11.0 versus 10.8 tok/s; no useful role improvement.
- framework
- 84.6%
- developer tools
- 75%
- broad tools
- 100%
- single stream
- 11tok/s
All results, context & sources
- framework
- 84.6 %
- developer tools
- 75 %
- broad tools
- 100 %
- single stream
- 11 tok/s
Context: 262K configured for the gate; advertised 1M not deep-tested
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSuperseded
Gemma-4 26B-A4B Heretic / ultra-uncensored
Q6_K
Historical two-round study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Stable, careful former writing candidate, later superseded by QAT/Q4 qualification. No numeric quality score is supplied here.
No numeric score retained.
All results, context & sources
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceWriting role
Gemma-4 26B-A4B QAT
Q4_K_M
Sep 22–23, 2026
llama.cpp Metal · build 10964
Mac Studio · M5 Max · 48 GB
Highest mechanical prose constraints in the Studio comparison. HumanEval was not run for this writing role.
- code decode
- 78.1tok/s
- prose decode
- 76.2tok/s
- framework
- 84.3%
- broad tools
- 94.1%
All results, context & sources
- code decode
- 78.1 tok/s
- prose decode
- 76.2 tok/s
- framework
- 84.3 %
- broad tools
- 94.1 %
- developer tools
- 66.7 %
- prose constraints
- 97 %
Context: Synthetic retrieval reached about 34K input positions; not 64K/262K qualified.
Limits: Old HumanEval can be contaminated by training data. Custom suite percentages are not autonomous coding success rates; regex criteria can penalize valid alternatives.
Mac Studio evaluation scorecard and retained numeric summary, 2026-09-22/23; reviewed directly 2026-10-02.
Local evidenceFormer default
Gemma-4 26B-A4B QAT uncensored
Q4_K_M
Jul 19, 2026
llama.cpp · GB10-era agent
Historical GB10 / lab pool; use dated record for exact host
Former multimodal agent; blind-writing mean 8.35/10 and a production context ceiling of 160K. The recorded edge test reached 119,593 input positions.
- reported decode
- 60.4tok/s
- broad tools
- 91.2%
- vision
- 87.5%
- blind editor mean score
- 8.35/10
All results, context & sources
- reported decode
- 60.4 tok/s
- broad tools
- 91.2 %
- vision
- 87.5 %
- blind editor mean score
- 8.35 /10
- deep edge input positions
- 119,593
- deep edge
- 78.3 s
Context: Historical production ceiling 163840; retained edge probe at 119,593 input positions. No 262K qualification.
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Gemma-4 26B-A4B-it clean
Q6_K
Historical two-round study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
The retained long-form study records a benign essay refusal with a claimed platform limit. No numeric literary-quality score was retained.
No numeric score retained.
All results, context & sources
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
GLM-4.5-Air 106B-A12B
Q4
Historical two-round study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Stable serving but slower decode prevented a general-agent default role.
- reported decode low
- 19tok/s
- reported decode high
- 20tok/s
All results, context & sources
- reported decode low
- 19 tok/s
- reported decode high
- 20 tok/s
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSpeed probe
GLM-5.3-Flash
EXL3
2026-10-05
TensorFold two-node TP2; exact probe revision not retained
DGX Spark + ASUS GX10 · two GB10 nodes
Historical October 5 speed probe: 97.59 tok/s code and 43.14 tok/s prose. The later abliterated production checkpoint has a separate matched Hermes entry; these speed results do not transfer to it.
- code single stream
- 97.59tok/s
- prose single stream
- 43.14tok/s
- prefill ttft
- 8.062s
- code four request mean decode
- 78.9tok/s
All results, context & sources
- code single stream
- 97.59 tok/s
- prose single stream
- 43.14 tok/s
- prefill ttft
- 8.062 s
- code four request mean decode
- 78.9 tok/s
- code four request rate sum
- 315.6 tok/s
Context: ~6K input timing probe; configured 262144 is not full-window qualification.
Limits: One wave per workload/concurrency, no repeat spread. Dates, model tokenizers and output lengths differ; exact runtime revision, effective thinking policy, cache state and external contention were not independently frozen. The prefill probe does not qualify 262K retrieval. No executed correctness, tool suite, or writing-quality result.
Retained lab speed report · October 5, 18:04:27–18:05:20 CDT
Local evidenceRejected
Goetia / Gemma-4 26B-A4B Heretic
Q4_K_M
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Near the former agent’s speed and vision accuracy but a lower blind-writing mean.
- reported decode
- 59.8tok/s
- vision
- 87.5%
- blind editor mean score
- 6.65/10
All results, context & sources
- reported decode
- 59.8 tok/s
- vision
- 87.5 %
- blind editor mean score
- 6.65 /10
Context: deep retrieval passed
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSuperseded
Huihui Qwen3-30B-A3B-2507 abliterated
Q6_K
Historical two-round study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Former general agent, later superseded by Gemma QAT qualification.
- reported decode approx
- 60tok/s
All results, context & sources
- reported decode approx
- 60 tok/s
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Huihui Qwen3.6-35B-A3B abliterated
NVFP4
Historical gate · archived Sep 12, 2026
Historical GB10 runtime
Historical GB10 / lab pool; use dated record for exact host
One of three long runs collapsed. This evidence applies to the tested quant, rather than every variant of this family.
- reported decode low
- 57tok/s
- reported decode high
- 60tok/s
- long runs
- 3
- collapsed runs
- 1
All results, context & sources
- reported decode low
- 57 tok/s
- reported decode high
- 60 tok/s
- long runs
- 3
- collapsed runs
- 1
Context: long-form suite
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSelected
Mage-Flow Turbo 4B
1024×1024 local pipeline
Jul 22, 2026
Dated image-generation pipeline
Historical local image lane
Historical preferred fit for warm latency, memory and visual quality.
- warm generation
- 3.96s
- peak
- 17.03GiB
All results, context & sources
- warm generation
- 3.96 s
- peak
- 17.03 GiB
Context: Not a language-model context test.
Limits: Warm timings only; pipeline/model styles and quality goals differ.
Dated public study and retained numeric archive; reviewed October 10, 2026.
Local evidenceRejected
Meta Muse Glimmer 30B
Dynamic kquant + projector + DFlash
Aug 10, 2026
llama.cpp b10358
Historical GB10 / lab pool; use dated record for exact host
Fast loading and complete screenshot-fact coverage, but 25% developer tools, dense-model latency and a configured 131072 ceiling failed the historical coder role.
- framework
- 81.5%
- developer tools
- 25%
- developer perception facts
- 30
- developer perception total
- 30
All results, context & sources
- framework
- 81.5 %
- developer tools
- 25 %
- developer perception facts
- 30
- developer perception total
- 30
- vision
- 75 %
- single stream
- 10.1 tok/s
- cold ready
- 17 s
Context: 131072 configured (Meta default); Unsloth claims up to 262K — not needle-verified here
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Partial runIncomplete
MiaAI Qwen3.8-Flash-Next
September NVFP4 artifact · thinking on
Run started Sep 26, 2026
vLLM · Aider whole-file editing
Two GB10 nodes
58 passes within two tries among 94 recorded exercises, out of a planned 225. No final pass rate or model winner.
- within two tries passed
- 58
- first try passed
- 28
- tasks recorded
- 94
- suite tasks
- 225
All results, context & sources
- within two tries passed
- 58
- first try passed
- 28
- tasks recorded
- 94
- suite tasks
- 225
- tasks unrecorded
- 131
- context exhaustion counter
- 72
- request error output counter
- 73
Context: Recipe specifies 65536; metadata lists 262144. Effective history cap not frozen.
Limits: Incomplete archive and different thinking policy. September MiaAI quality does not transfer to the current October NVIDIA artifact.
Dated public study and retained numeric archive; reviewed October 10, 2026.
Local evidenceRetained candidate
Mistral Small 4 119B-A6B
Official NVFP4
Sep 12, 2026
vLLM 0.29.0 · TP1
Historical GB10 / lab pool; use dated record for exact host
Quality candidate left unrouted: stronger tools and prose constraints, but slower decode and long readiness.
- framework
- 81.4%
- developer tools
- 83.3%
- broad tools
- 100%
- prose constraints
- 86.9%
All results, context & sources
- framework
- 81.4 %
- developer tools
- 83.3 %
- broad tools
- 100 %
- prose constraints
- 86.9 %
- single stream
- 30.7 tok/s
- 2-stream aggregate
- 52.2 tok/s
- 4-stream aggregate
- 75.9 tok/s
- cold ready
- 674 s
- cached ready
- 585 s
- model
- 66.12 GiB
- kv
- 27.33 GiB
Context: 262144 configured; vLLM allocated 1,273,776 KV tokens / 4.86× capacity; retrieval not needle-verified
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceFormer default
Nemotron 3 Nano Omni 30B-A3B
UD-Q4_K_XL + projector
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Historical developer-vision sidecar: all 30 screenshot facts in 10 seconds. Generic fixtures were 5/8.
- developer perception facts
- 30
- developer perception total
- 30
- developer perception wall
- 10s
- vision passed
- 5
All results, context & sources
- developer perception facts
- 30
- developer perception total
- 30
- developer perception wall
- 10 s
- vision passed
- 5
- vision tasks
- 8
Context: 240K verified; 262K configured
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Partial runCapacity rejected
Nemotron 3 Super 120B-A12B
MXFP4
Historical partial gate · archived Sep 12, 2026
Historical GB10 runtime
Historical GB10 / lab pool; use dated record for exact host
Answered three framework tasks, then was OOM-killed. Individual task scores are retained; no full-suite percentage is inferred.
- tasks recorded
- 3
- task one
- 100%
- task two
- 100%
- task three
- 77.8%
All results, context & sources
- tasks recorded
- 3
- task one
- 100 %
- task two
- 100 %
- task three
- 77.8 %
- reported decode
- 16.6 tok/s
Context: 262K configured; not qualified
Limits: Three recorded tasks are not a completed eight-task gate. Configured context was not qualified.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSpecialist
Nemotron Cascade 2 30B-A3B
Q4
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Context specialist with a recorded 240K edge pass in 141 seconds; did not replace the coder.
- framework
- 80.6%
- developer tools
- 66.7%
- deep edge
- 141s
All results, context & sources
- framework
- 80.6 %
- developer tools
- 66.7 %
- deep edge
- 141 s
Context: 240K verified; native up to 1M
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceFormer default
Nemotron-3.5-Lightning 30B-A3B
MLX 4-bit
Sep 22–23, 2026
MLX / MLX-VLM
Mac Studio · M5 Max · 48 GB
Fast September Studio agent artifact. The October agent alias serves a different model.
- code decode
- 150.1tok/s
- prose decode
- 150.4tok/s
- framework
- 71.5%
- broad tools
- 88.2%
All results, context & sources
- code decode
- 150.1 tok/s
- prose decode
- 150.4 tok/s
- framework
- 71.5 %
- broad tools
- 88.2 %
- developer tools
- 66.7 %
- prose constraints
- 73.5 %
- humaneval passed
- 140
- humaneval tasks
- 164
- humaneval
- 85.4 %
Context: Synthetic retrieval reached about 34K input positions; not 64K/262K qualified.
Limits: Old HumanEval can be contaminated by training data. Custom suite percentages are not autonomous coding success rates; regex criteria can penalize valid alternatives.
Mac Studio evaluation scorecard and retained numeric summary, 2026-09-22/23; reviewed directly 2026-10-02.
Local evidenceFormer default
Nemotron-3.5-Lightning 30B-A3B
Official NVFP4 + DSpark
Aug 11 / Sep 12, 2026
vLLM 0.29.0
Historical GB10 / lab pool; use dated record for exact host
September agent default. The clean rebaseline improved the framework gate to 80.3%; the long essay stopped at 3,935 of 10,000 requested words.
- framework
- 80.3%
- developer tools
- 75%
- broad tools
- 82.4%
- prose constraints
- 63.8%
All results, context & sources
- framework
- 80.3 %
- developer tools
- 75 %
- broad tools
- 82.4 %
- prose constraints
- 63.8 %
- single stream
- 77.6 tok/s
- 2-stream aggregate
- 120.4 tok/s
- 4-stream aggregate
- 169 tok/s
- longform
- 85.9 tok/s
- longform ttft
- 0.13 s
- longform words
- 3,935
- longform requested words
- 10,000
Context: production 128K; 1M native, not re-needle-tested in this pass
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
North Mini Code 30B-A3B
Q4
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Thinking retest improved framework checks, but tools and deep-context/scaffold qualification still failed.
- framework thinking off
- 58.3%
- framework thinking retest
- 74.4%
- developer tools
- 41.7%
All results, context & sources
- framework thinking off
- 58.3 %
- framework thinking retest
- 74.4 %
- developer tools
- 41.7 %
Context: incomplete/failing at depth
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRetired
Ornith 1.0 9B uncensored
Q6_K
Historical two-round study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Long outputs at about 28 tok/s, without a current routed advantage.
- reported decode approx
- 28tok/s
All results, context & sources
- reported decode approx
- 28 tok/s
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSpecialist
Ornith-1.0-35B
UD-Q4_K_XL
Jul 30, 2026
llama.cpp · tuned gate
Historical GB10 / lab pool; use dated record for exact host
Five framework runs: median 84.2%, standard deviation 5.55 points; Swift median 60.7%. Four tool runs: median 75%. Faster, smaller and quick to load, but not promoted over the primary coder.
- framework median
- 84.2%
- framework stddev points
- 5.55
- swift median
- 60.7%
- developer tools median
- 75%
All results, context & sources
- framework median
- 84.2 %
- framework stddev points
- 5.55
- swift median
- 60.7 %
- developer tools median
- 75 %
- framework runs
- 5
- developer tool runs
- 4
- single stream
- 61.6 tok/s
- cold ready
- 27 s
- allocated
- 28.2 GiB
Context: 2 x 262144 configured; native 262144
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSpecialist
Ortenzya 31B
Q4_K_M
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Optional creative specialist; blind-writing mean 8.17/10, with dense decode around 9.8 tok/s.
- blind editor mean score
- 8.17/10
- reported decode
- 9.8tok/s
All results, context & sources
- blind editor mean score
- 8.17 /10
- reported decode
- 9.8 tok/s
Context: deep retrieval passed
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceReversed promotion
Poolside Laguna-S-2.1 117.6B MoE
NVFP4 · revision 0761412
Jul 23–26, 2026
vLLM · SM121
Historical GB10 / lab pool; use dated record for exact host
Promoted July 23 and reverted July 26. Same-day control won framework quality and wall time; Laguna retained a tool advantage but had heavy memory use and slow cold startup.
- framework
- 87.2%
- developer tools
- 75%
- framework wall
- 436s
- single stream
- 17.7tok/s
All results, context & sources
- framework
- 87.2 %
- developer tools
- 75 %
- framework wall
- 436 s
- single stream
- 17.7 tok/s
- allocated
- 101 GiB
- cold ready low
- 480 s
- cold ready high
- 900 s
Context: 257.1K verified (middle + edge pair); 262K configured
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Poolside Laguna-XS-2.1 33B
NVFP4
Jul 30, 2026
vLLM
Historical GB10 / lab pool; use dated record for exact host
Smaller weights did not solve the long-context KV footprint. Four-stream throughput used different slot settings than the llama.cpp incumbent; it is not an isolated model advantage.
- framework
- 83.9%
- developer tools
- 50%
- single stream
- 35.3tok/s
- 4-stream aggregate
- 127.7tok/s
All results, context & sources
- framework
- 83.9 %
- developer tools
- 50 %
- single stream
- 35.3 tok/s
- 4-stream aggregate
- 127.7 tok/s
- cold ready
- 200 s
Context: 262144 configured
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3-235B-A22B-2507
Q2_K_XL; separate Q3 load failed
Historical study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Q3 load failed with OOM. The tested Q2 ran around 14 tok/s with qualitative quantization damage; legacy refusal counts are separate from quality.
- reported decode approx
- 14tok/s
- legacy refusal cases
- 6
- legacy refusals
- 1
All results, context & sources
- reported decode approx
- 14 tok/s
- legacy refusal cases
- 6
- legacy refusals
- 1
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3-30B-A3B-Instruct-2507 clean
Q6_K
Historical two-round study · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Stable serving, but outline and long-form decay prevented a default role.
- reported decode approx
- 62tok/s
All results, context & sources
- reported decode approx
- 62 tok/s
Context: suite context only
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3-Coder-Next 80B-A3B
Q5_K_M
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Framework score near the Q4 baseline, but Swift and Tailwind regressed and code decode was slower.
- framework
- 86.2%
- developer tools
- 66.7%
- code decode
- 45.3tok/s
All results, context & sources
- framework
- 86.2 %
- developer tools
- 66.7 %
- code decode
- 45.3 tok/s
Context: same native 262K; deep test reserved for best Q5
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceFormer default
Qwen3-Coder-Next 80B-A3B
UD-Q4_K_XL
Jul 21–30, 2026
llama.cpp · dated runtime/tuning variants
Historical GB10 / lab pool; use dated record for exact host
Restored after the Laguna reversal. Six-run calibration median 86.4%; a separate July 26 run reached 90.9%. These are different summaries, not two contradictory scores.
- framework median
- 86.4%
- framework stddev points
- 2.9
- jul26 framework
- 90.9%
- jul26 developer tools
- 66.7%
All results, context & sources
- framework median
- 86.4 %
- framework stddev points
- 2.9
- jul26 framework
- 90.9 %
- jul26 developer tools
- 66.7 %
- jul26 single stream
- 52.7 tok/s
- jul26 4-stream aggregate
- 76 tok/s
- allocated
- 53.9 GiB
- cold ready approx
- 90 s
Context: 2 x 262K configured on GX10; 2 x ~241K concurrently verified
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3-Coder-Next 80B-A3B
UD-Q5_K_M
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
No assigned-role win over the Q4 incumbent.
- framework
- 87.4%
- developer tools
- 58.3%
- code decode
- 41.6tok/s
All results, context & sources
- framework
- 87.4 %
- developer tools
- 58.3 %
- code decode
- 41.6 tok/s
Context: same native 262K; deep test reserved for best Q5
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceWatch
Qwen3-Coder-Next 80B-A3B
UD-Q5_K_XL
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
88.0% overall and 74.1% Swift, with slower decode and Tailwind/Bash regressions. Retained as a Swift watch candidate.
- framework
- 88%
- swift
- 74.1%
All results, context & sources
- framework
- 88 %
- swift
- 74.1 %
Context: 231.7K + edge pair verified
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3.5-122B-A10B
Q4 + projector
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Good developer tools, but weaker framework quality and long deep-context wall time.
- framework
- 79.5%
- developer tools
- 83.3%
- deep edge
- 480.6s
All results, context & sources
- framework
- 79.5 %
- developer tools
- 83.3 %
- deep edge
- 480.6 s
Context: 231.7K verified; native 262K
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceControl
Qwen3.5-27B
Q4_K_M
Jul 29, 2026
llama.cpp
Historical lab pool
Matched Qwythos control. Framework 84.3%, tools 75%, broad tools 100% and single-wave throughput 10.8 tok/s.
- framework
- 84.3%
- developer tools
- 75%
- broad tools
- 100%
- single stream
- 10.8tok/s
All results, context & sources
- framework
- 84.3 %
- developer tools
- 75 %
- broad tools
- 100 %
- single stream
- 10.8 tok/s
Context: 262K configured; no deep qualification in this matched gate.
Limits: Applies to the tested artifact, runtime, hardware and dated protocol. It does not transfer to other variants.
Dated public study and retained numeric archive; reviewed October 10, 2026.
Local evidenceFallback
Qwen3.5-9B
Q6_K + BF16 projector
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Complete screenshot-fact coverage, but the six images took 80.7 seconds versus Omni’s 10 seconds.
- developer perception facts
- 30
- developer perception total
- 30
- developer perception wall
- 80.7s
- vision passed
- 6
All results, context & sources
- developer perception facts
- 30
- developer perception total
- 30
- developer perception wall
- 80.7 s
- vision passed
- 6
- vision tasks
- 8
Context: 231.7K verified; native 262K
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceFallback
Qwen3.6-27B
Q4
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
A fallback with slower deep retrieval, rather than a primary upgrade.
- framework
- 81.8%
- developer tools
- 75%
- vision
- 75%
- deep edge
- 535s
All results, context & sources
- framework
- 81.8 %
- developer tools
- 75 %
- vision
- 75 %
- deep edge
- 535 s
Context: 231.7K verified; native 262K
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3.6-27B Fable Fusion
Q4_K_M
Historical gate · archived Sep 12, 2026
llama.cpp
Historical GB10 / lab pool; use dated record for exact host
Long-form repetition collapse. Vision reached 75% only with tuned settings.
- blind editor mean score
- 4.77/10
- reported decode
- 10.8tok/s
- tuned vision
- 75%
All results, context & sources
- blind editor mean score
- 4.77 /10
- reported decode
- 10.8 tok/s
- tuned vision
- 75 %
Context: deep retrieval passed
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3.6-35B-A3B
UD-Q4_K_XL
Jul 21 / Jul 30, 2026
llama.cpp · original and tuned rematch
Historical GB10 / lab pool; use dated record for exact host
Original 75.2% framework rose to 81.7% in the tuned rematch, still below the historical primary baseline. Same artifact appears twice in the archive and is consolidated here.
- original framework
- 75.2%
- rematch framework
- 81.7%
- developer tools
- 75%
- vision
- 75%
All results, context & sources
- original framework
- 75.2 %
- rematch framework
- 81.7 %
- developer tools
- 75 %
- vision
- 75 %
- single stream
- 62.4 tok/s
- 4-stream aggregate
- 95.7 tok/s
Context: 231.7K retrieval recorded in the original gate; two 262144 lanes configured for the later rematch, without a new two-lane qualification.
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceSuperseded
Qwen3.6-35B-A3B Heretic
Q6_K + native MTP
Jul 11, 2026
llama.cpp b9967 in July speed A/B
Historical GB10 / lab pool; use dated record for exact host
Six draft positions sped up code and slowed creative writing in the same quiet single-slot comparison.
- code mtp off
- 64.1tok/s
- code mtp on
- 100.9tok/s
- creative mtp off
- 64.6tok/s
- creative mtp on
- 53.6tok/s
All results, context & sources
- code mtp off
- 64.1 tok/s
- code mtp on
- 100.9 tok/s
- creative mtp off
- 64.6 tok/s
- creative mtp on
- 53.6 tok/s
Context: ~255K verified in candidate config
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceCoding candidate
Qwen3.8-27B
MLX 4-bit
Sep 22–23, 2026
MLX / MLX-VLM
Mac Studio · M5 Max · 48 GB
Higher HumanEval and framework checks than Lightning in the same Studio study.
- code decode
- 33tok/s
- prose decode
- 33tok/s
- framework
- 89.5%
- broad tools
- 100%
All results, context & sources
- code decode
- 33 tok/s
- prose decode
- 33 tok/s
- framework
- 89.5 %
- broad tools
- 100 %
- developer tools
- 50 %
- prose constraints
- 87.2 %
- humaneval passed
- 156
- humaneval tasks
- 164
- humaneval
- 95.1 %
Context: Synthetic retrieval reached about 34K input positions; not 64K/262K qualified.
Limits: Old HumanEval can be contaminated by training data. Custom suite percentages are not autonomous coding success rates; regex criteria can penalize valid alternatives.
Mac Studio evaluation scorecard and retained numeric summary, 2026-09-22/23; reviewed directly 2026-10-02.
Local evidenceFormer default
Qwen3.8-27B
NVFP4 target + DFlash2 draft
Aug 19 / Sep 12 / Sep 26, 2026
SGLang · dated draft/runtime variants
Historical GB10 / lab pool; use dated record for exact host
September coder default; isolated August gate 92.0%, September gate 85.9%. Completed Aider run: 112/225 passed within two tries. Draft/runtime and workloads vary across these dates.
- framework
- 85.9%
- developer tools
- 75%
- broad tools
- 94.1%
- single stream
- 19.6tok/s
All results, context & sources
- framework
- 85.9 %
- developer tools
- 75 %
- broad tools
- 94.1 %
- single stream
- 19.6 tok/s
- 2-stream aggregate
- 31.5 tok/s
- 4-stream aggregate
- 42.7 tok/s
- aider first try passed
- 21
- aider within two tries passed
- 112
- aider tasks recorded
- 225
- aider suite tasks
- 225
- aider within two tries
- 49.8 %
Context: 262144 configured/native; not needle-verified in this engine pass
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceRejected
Qwen3.8-27B
Official fine-grained FP8
Aug 14, 2026
vLLM 0.27.1
Historical GB10 / lab pool; use dated record for exact host
Strong broad tool and screenshot checks did not offset 8 tok/s decode, a 1,072-second framework run and 433-second cold readiness.
- framework
- 87.5%
- developer tools
- 50%
- broad tools thinking off
- 100%
- broad tools thinking on
- 100%
All results, context & sources
- framework
- 87.5 %
- developer tools
- 50 %
- broad tools thinking off
- 100 %
- broad tools thinking on
- 100 %
- developer perception facts
- 30
- developer perception total
- 30
- vision
- 62.5 %
- prose constraints
- 77.8 %
- single stream
- 8 tok/s
- 4-stream aggregate
- 31.2 tok/s
- framework wall
- 1,072 s
- cold ready
- 433 s
- allocated
- 91.8 GiB
Context: 262144 configured; advertised native — not needle-verified (speed already failed)
Limits: Historical protocols and revisions vary. Registry-only values retain their original rounding; repeat spread, timing denominator or blind-judge task count may be unspecified. See each linked study for stronger provenance.
Archived local variant registry, last updated September 12, 2026; retained summaries reviewed October 10. Exact dates are shown only where retained.
Local evidenceHistorical runtime evidence
Qwen3.8-Flash-Next
NVFP4 · October artifact
2026-10-09
TensorFold two-node TP2; exact probe revision not retained
DGX Spark + ASUS GX10 · two GB10 nodes
October TensorFold speed evidence; runtime and workload conditions remain attached to the dated report. Superseded as the active coder; current DeepSeek has separate evidence.
- code single stream
- 80.09tok/s
- prose single stream
- 50.26tok/s
- prefill ttft
- 2.108s
- code four request mean decode
- 53.7tok/s
All results, context & sources
- code single stream
- 80.09 tok/s
- prose single stream
- 50.26 tok/s
- prefill ttft
- 2.108 s
- code four request mean decode
- 53.7 tok/s
- code four request rate sum
- 214.78 tok/s
Context: ~6K input timing probe; configured 262144 is not full-window qualification.
Limits: One wave per workload/concurrency, no repeat spread. Dates, model tokenizers and output lengths differ; exact runtime revision, effective thinking policy, cache state and external contention were not independently frozen. The prefill probe does not qualify 262K retrieval. No executed correctness, tool suite, or writing-quality result.
Retained lab speed report · October 9, 22:56:32–22:57:14 CDT
Local evidenceHistorical runtime evidence
Qwen3.8-Flash-Next
NVFP4 · October NVIDIA serving artifact
2026-10-05
vLLM two-node TP2; exact probe revision not retained
DGX Spark + ASUS GX10 · two GB10 nodes
Historical October vLLM speed evidence and October 10 configuration inspection. The short probe does not qualify quality or full context. Superseded as the active coder; current DeepSeek has separate evidence.
- code single stream
- 67.25tok/s
- prose single stream
- 44.79tok/s
- prefill ttft
- 1.851s
- code four request mean decode
- 53.48tok/s
All results, context & sources
- code single stream
- 67.25 tok/s
- prose single stream
- 44.79 tok/s
- prefill ttft
- 1.851 s
- code four request mean decode
- 53.48 tok/s
- code four request rate sum
- 213.91 tok/s
Context: ~6K input timing probe; configured 262144 is not full-window qualification.
Limits: One wave per workload/concurrency, no repeat spread. Dates, model tokenizers and output lengths differ; exact runtime revision, effective thinking policy, cache state and external contention were not independently frozen. The prefill probe does not qualify 262K retrieval. No executed correctness, tool suite, or writing-quality result.
Retained lab speed report · October 5, 16:41:01–16:41:49 CDT
Local evidenceAlternative
Sana Sprint 1.6B
1024×1024 local pipeline
Jul 22, 2026
Dated image-generation pipeline
Historical local image lane
Fastest warm timing of these three; the study did not establish a shared quantitative image-quality score.
- warm generation
- 0.79s
- peak
- 10.19GiB
All results, context & sources
- warm generation
- 0.79 s
- peak
- 10.19 GiB
Context: Not a language-model context test.
Limits: Warm timings only; pipeline/model styles and quality goals differ.
Dated public study and retained numeric archive; reviewed October 10, 2026.
Awaiting benchmarkConfigured
SuperQwen · current Studio agent
MLX 4-bit · October alias
Configured artifact reviewed Oct 10, 2026
MLX
Mac Studio · M5 Max · 48 GB
The current agent artifact has no local score in the reviewed archive. September Nemotron results remain attached to Nemotron.
No numeric score retained.
All results, context & sources
Context: No independent full-context qualification in the reviewed evidence.
Limits: Packaged upstream reproduction scripts are not measurements from this lab.
Direct October 10 configuration review.
Local evidenceAlternative
Z-Image-Turbo
1024×1024 local pipeline
Jul 22, 2026
Dated image-generation pipeline
Historical local image lane
Historical warm budget exceeded the under-20-GiB target.
- warm generation
- 5.7s
- peak
- 21.69GiB
All results, context & sources
- warm generation
- 5.7 s
- peak
- 21.69 GiB
Context: Not a language-model context test.
Limits: Warm timings only; pipeline/model styles and quality goals differ.
Dated public study and retained numeric archive; reviewed October 10, 2026.