Skip to content
← All setup guides

Guide 08 / Operations

Operating a coder across two GB10 hosts

Operate the current DeepSeek TensorFold pair: shared admission, model identity, Engram staging, memory guards, readiness, rollback and pending maintenance.

5 MIN READ / REVIEWED Oct 23, 2026

01Check both ranks and the actual model

The October 23 inspection confirmed DeepSeek V4.1 Flash 1017 EXL3 containers on both GB10 hosts, using the same pinned TensorFold image. The DGX Spark is the head and the ASUS GX10 supplies rank 1. Both still report driver 580.173.02. Check the running artifacts and native identity rather than inferring them from a dashboard label.

Read-only runtime checks

# Run on each GB10 host
nvidia-smi --query-gpu=name,driver_version --format=csv
free -h
docker ps --format '{{.Names}}  {{.Image}}  {{.Status}}'
# Substitute your coder container name
docker inspect YOUR_CODER_CONTAINER --format '{{json .Config.Cmd}}'

03Budget host memory and shared admission

The recipe retains a 112 GiB available-memory minimum before startup and runtime guards at 2.5 GiB, with a 1 GiB hard floor and four-second hold policy. Scoped memory-policy changes restore each host’s original policy when the supervised runtime stops. Preserve these guards when tuning.

The initial one-hour observation recorded minimum available memory of 4.93 GiB on the head and 6.55 GiB on the worker, across 1,800 samples each. No OOM or supervisor restart was recorded during that bounded window. Four slots and 524,288 context do not establish four concurrent full windows.

04Record the effective settings

Production uses normal DeepSeek with high reasoning and 32,768 output positions. The optional abliteration overlay was archived but not applied. Matched quality controls used thinking off and a 4,096 output budget; later tuning cannot be credited as a same-budget model improvement.

The active coder's settings

# Inspected deployment policy · 23 Oct 2026
# Inventory excerpt; not a standalone launch command
model: DeepSeek-V4.1-Flash-1017
quantization: EXL3 EAQ 2.9 bpw
runtime: TensorFold native Zig
nodes: 2
tensor_parallel_size: 2
mode: normal
ABLIT: 0
CONTEXT: 524288
PARALLEL: 4
MAX_TOKENS: 32768
THINKING: 1
REASONING_EFFORT: high
MIN_AVAIL_GIB: 112
MEMGUARD_GIB: 2.5
MEMGUARD_HARD_GIB: 1
MEMGUARD_HOLD_S: 4
shared_roles: [coder, scout, vision]

05Keep readiness and persistence explicit

Keep the persistent runtime supervised and distinguish container state, native health, exact model identity and a completed routed request. Installation checks passed text, streaming/usage, structured tool roundtrip, image input and a 21,875-position prompt. Eight simultaneous clients were also checked for deterministic short replies; the engine still admits only four at a time.

The Studio launcher and exclusive primary group remain a separate lifecycle. The lightning/agent alias serves SuperQwen, so older Nemotron Lightning results stay historical. The October 10 Qwen startup cost is not a DeepSeek startup measurement.

06Make the switch an owned operation

Archive and verify the checkpoint plus Engram inputs before freeing local disk, preserve the incumbent’s exact rollback resources, and verify both ranks before moving aliases. The DeepSeek installation retained GLM rollback, repaired the worker rail and checked all three gateway entry points. This website review performs no runtime switch.

  • Record the incumbent's checkpoint, image, arguments and rollback target before a change.
  • Stop the competing two-node backend before starting a replacement; both own the same GB10 pair.
  • Wait for the replacement's health and readiness, then move coder and vision to the same target.
  • Keep the Studio agent route independent, and test tools, native vision, cancellation and stream usage after switching.
  • Run a quality gate and a representative context/concurrency soak before promoting a speed-probe candidate.

07Keep operating samples distinct from experiments

The October 23 1/2/4-request probe measured 41.16 / 75.78 / 91.57 aggregate output positions per second, divided by complete wave wall time. The first wave was uncached; later requests reused 2,048 of 2,787 prompt positions. One wave each and seven requests do not establish a general scaling curve.

October 10 mixed-traffic counters and October 5/9 sums of individual decode rates remain historical with their original denominators. The matched Hermes panel and live draft delivery answer separate quality and workflow questions.

08Schedule updates as a separate change

sparkDash was updated to 2.70.0. Customized llama-swap v257 remains installed; upstream v266 is available, but local admission, affinity, readiness and UI changes must be ported before replacement. NVIDIA DGX Dashboard is a different component and remains pending.

The October 22 inventory also identified OS/kernel/driver, Caddy, Grafana and Prometheus updates. Driver 580.173.02 still serves the pinned DeepSeek image. Coordinate maintenance and rollback; verify ASUS partner firmware separately from NVIDIA Founders Edition release notes.

Primary references

Upstream docs describe supported behavior. Lab-specific configurations and numeric results retain their inspection or experiment dates.

Keep going

Related guides