Mac Studio · writing-specialist evaluation
Gemma-4 26B-A4B QAT · Q4_K_M
Mac Studio
- code decode
- 78.1tok/s
- prose decode
- 76.2tok/s
- framework
- 84.3%
All measurements & context
- code decode
- 78.1 tok/s
- prose decode
- 76.2 tok/s
- framework
- 84.3 %
- broad tools
- 94.1 %
- developer tools
- 66.7 %
- prose constraints
- 97 %
Measured on M5 Max, 40 GPU cores, 48 GB unified memory. Single-stream decode is the median of three short 512-output probes on a loaded backend, with warm/fresh prefixes retained. Framework scores are regex criteria, not compilation. Broad/developer tools and prose constraints are custom suites. HumanEval was not run for this writing role; no coding benchmark score is inferred. Synthetic retrieval checks reached about 34K input positions, not 64K/262K qualification.
Protocol: Sequential large-model tests; thinking off; repaired proxy path. Quality suites and timing probes use different prompts.
Limits: Old HumanEval can be contaminated by training data. Custom suite percentages are not autonomous coding success rates; regex criteria can penalize valid alternatives.
Source: Mac Studio evaluation scorecard and retained numeric summary, 2026-09-22/23; reviewed directly 2026-10-02.
Runtime: llama.cpp Metal · build 10964
Source record updated 2026-09-23