Skip to content
← All journal notes

Journal / Configuration

DeepSeek on two GB10 hosts, with the coding gate still intact

Normal DeepSeek now serves four shared slots through TensorFold. The matched GLM panel tied on coding; capacity, cache replay, memory and maintenance have separate evidence.

/6 MIN READ

The primary coder has changed since the October 10 review. Qwen/vLLM was followed by GLM/TensorFold; normal DeepSeek V4.1 Flash 1017 now runs across our DGX Spark and ASUS GX10. Both GB10 containers, installed settings and gateway selectors were checked again on October 23. Each host still reports NVIDIA driver 580.173.02.

The installation and evaluation completed overnight. This website review inspected the running setup and retained receipts; it did not switch models or restart the lab.

The current deployment

The checkpoint is EXL3 EAQ 2.9 bpw, served by TensorFold's native Zig engine with tensor parallelism 2, one rank on each 128 GB GB10 host. The installed recipe is pinned at 370daf0, the engine at 629c524, and the image patch set at cc4781a03264. Both ranks use the same image.

The inspected policy is 524,288 configured context, four shared inference slots, a 32,768 output allowance, and thinking enabled with high reasoning effort by default. Coder, scout and native vision alias this same backend. Three role names do not create twelve slots. The Studio's SuperQwen agent and embeddings remain independent routes.

DeepSeek runs in normal mode, ABLIT=0. The optional abliteration overlay was downloaded and verified for the archive, but was not applied. The earlier GLM production checkpoint used in the comparison was abliterated; that distinction belongs beside the result.

Staging and rollback are part of the setup

The verified checkpoint and separate Engram inputs total 413,743,842,936 bytes in the model archive. Manifest-listed inputs were staged and checked before the incumbent stopped. Engram inputs are local to both ranks. Archive-verified superseded files made room while the exact GLM checkpoint, image, spills and gateway configuration were preserved for rollback.

The worker's second RoCE rail needed a GID correction. Both direct links passed connectivity checks. These checks establish connectivity, not measured link bandwidth. Preparation also recovered from a missing CLI path without changing the model or build pins.

The recipe retains its 112 GiB available-memory admission minimum, runtime reserve, 2.5 GiB guard floor, 1 GiB hard floor and four-second hold policy. The scoped memory policy restores each host's prior policy when the supervised runtime stops. Removing a guard would change the operating configuration, not establish extra capacity.

The matched coding panel tied

The fresh primary Hermes comparison froze tasks, sources, sampling, budgets, grading and cleanup gates. Both arms used temperature 0.2, thinking disabled and 4,096 output positions per request. The routine visit allowed 360 seconds and 32 requests; hard visits allowed 1,440 seconds and 96 requests, including repairs and reporting.

  • Routine coding: GLM 1/1; DeepSeek 1/1. The configuration repair took 63.5 seconds with GLM and 74.7 seconds with DeepSeek.
  • Hard coding: GLM 0/3; DeepSeek 0/3. Menu behavior, strict typing and native socket repair all remained failures under the full acceptance gate.
  • Defective-change review: DeepSeek correctly rejected the defective menu change; GLM approved it incorrectly.
  • Admission and healthy-review controls: both models made the correct decisions.

Acceptance required executable behavior, permitted scope, a valid structured report, exact native identity and verified cleanup. Passing public commands did not erase a scope or behavior failure. All scored identity, report and cleanup gates passed; the coding failures remain failures.

This is a small local repository handoff panel, with one visit per case and one primary harness. It is not an official SWE-bench Verified score. The result does not establish a difficult-coding advantage or a general speed ranking. Both dated records preserve the denominators and wall times.

Short serving probes answer a different question

One native wave each at one, two and four concurrent requests measured 41.16, 75.78 and 91.57 aggregate output positions per second. These rates divide total completion positions by complete wave wall time, including prefill. The waves returned 160, 307 and 598 output positions in about 3.887, 4.051 and 6.531 seconds respectively.

Each request had 2,787 input positions, temperature zero, thinking off and a 160-position output cap. The first wave was uncached; later requests reported 2,048 cached positions. Seven requests, changing cache state and no repeat spread do not establish a cold-cache scaling curve or coding quality. The older October harness summed individual decode rates, so its numbers use a different denominator.

Installation validation also passed text, streaming completion and usage, structured tools and tool-result roundtrip, image input, thinking controls and a 21,875-position prompt. Eight simultaneous short clients were checked for deterministic replies; only four are admitted to inference at once.

A long setting still needs a long test

A copy of a held session supplied 63,164 input positions, 220 messages and 110 complete tool-call/result pairs. Two high-reasoning requests returned valid structured responses in 43.9 seconds cold and 8.8 seconds warm. The second request reported 61,440 cached positions. No tools were executed in this compatibility replay, and the original session was preserved.

Those are complete request wall times, not TTFT. The replay does not measure task success or retrieval. Neither it nor the shorter validation qualifies the configured 524,288-position window, four full windows together, or unrelated conversation histories. Upstream recipe benchmarks are linked as references and are not republished as this lab's measurements.

Initial memory and resumed work

From roughly 03:02 to 04:02 CDT on October 23, 1,800 samples per host recorded minimum available memory of 4.93 GiB on the head and 6.55 GiB on the worker. No OOM or supervisor restart was recorded during this initial observation. The minima occurred at different instants. A bounded sample window does not establish days of stability, diagnose a leak, or qualify heavy concurrent long prompts.

Production tuning then selected high reasoning with the native 32,768 allowance, a four-worker loop target and four shared inference leases. Short non-thinking inspection and planning use separate budgets. These settings differ from the frozen quality panel and cannot be credited as a same-budget model-only improvement.

After 1,122 Python and 46 JavaScript regression checks, the loop restarted with its three held workers preserved and resumed at 03:57 CDT. Held sessions continued on DeepSeek; new work used the updated prompt revision. Useful maintenance and test changes began reaching draft PRs, while structured-report validation and other blockers remained visible. Held runs can contain GLM work, and book/editorial output differs from coding work: raw PR totals are not a coding-productivity score. Draft delivery, independent review and merging are separate outcomes.

Installed updates and pending maintenance

sparkDash was updated from 2.0.0 to 2.70.0, preserving the fleet, histories, routing customizations and existing proxy mount. This is separate from NVIDIA DGX Dashboard. The updated build passed type checking, production build and frontend/backend checks.

The October 22 maintenance inventory identified updates that remain pending:

  • Customized llama-swap v257 → upstream v266. Capacity, affinity, peer-readiness and UI patches need review before replacing the gateway.
  • NVIDIA driver 580.173.02 → 580.178.04, plus OS/kernel and DGX Dashboard updates. The installed driver already serves the pinned DeepSeek image.
  • Caddy 2.11.4 → 2.11.7, patched Grafana 13.1.2 → 13.2.4, and Prometheus 3.13.2 → 3.15.1.

NVIDIA's October release notes also describe Linux 7.0 support and dynamic display-reserved memory. They apply to Founders Edition systems; ASUS firmware availability needs its own check. Available software is not installed software, and this review applies none of those platform changes.

The DeepSeek setup guide, operations guide, software stack and benchmark methods carry the practical details. Primary references are the installed recipe revision, NVIDIA release notes and llama-swap v266 release.