DeepInfer FAQ — Env Vars, Eval Pass@1, Production Attribution

Attribution — production vs OSS. Figures such as ~155 trillion tokens/day per cluster and ~US$38 per GPU-hour (valley) describe DeepSeek’s production inference system as published in doc/inference_deployment_qa.md and related explainers. They are not what you automatically get after installing open-source DeepInfer. OSS is an offline engine release; KV store, full PD-disagg serving, and DP LB are not included.

DeepInfer vs recipe vs Harness?

DeepInfer — offline inference engine for DeepSeek-V4.1-Flash (Apache-2.0).
deepseek-recipe — protocols / prompts / tokenization; does not infer.
DeepSeek Harness (DSH) — agent runtime · dshbase.com.

Environment variables (DEEPINFER_* + Ascend debug)

Variable	Role	Source
`DEEPINFER_ACCELERATOR`	`nvidia` or `ascend` — keep exported for build and run	README
`DEEPINFER_HF_MODEL_DIR`	Local checkpoint path instead of HF cache download	README
`DEEPINFER_ENABLE_MTP_VARLEN`	Enable DSpark / MTP varlen verification (set `1`)	README
`DEEPINFER_USE_NEXTN_DECODE`	Max draft length (default 5); set `0` to disable speculative decode	README / examples
`DEEPINFER_YOCO_PREFILL_TRUNCATION_MODE`	V4.1 default `compact` for SWA bounded replay	README
`DEEPINFER_YOCO_PREFILL_KEEP_TOKENS`	Default `140` retained suffix	README
`DEEPINFER_BATCH_INVARIANT`	`1` batch-invariant (NVIDIA); pair with MTP off + fixed `seq_seed` for determinism	README
`ASCEND_LAUNCH_BLOCKING`	Sync ops for readable Ascend stack traces	examples/README
`ASCEND_SLOG_PRINT_TO_STDOUT`	Print CANN logs to stdout	examples/README

Eval Pass@1 (reference)

From eval/README.md — V4.1 reference under the documented default config. Stochastic; use as a sanity check.

Benchmark	Pass@1
AIME 2026	99.9%
GPQA Diamond	93.4%
HLE-Full	42.6%

Production system numbers (not OSS install outcomes)

Values below are reported for DeepSeek’s production V4.1 Flash inference deployment / official Q&A explainers. Cite the Q&A; do not market them as OSS defaults.

Fact	Value	Label
Cluster throughput	~155 trillion tokens/day	Production system
Revenue (valley)	~US$38 / GPU-hour	Production system
Avg input / cache hit	~77K tokens · ~96% hit	Production system
TTFT	~400 ms average	Production system
Prefill (no-cache → with-cache)	~60k → ~1.5M tok/s/GPU (~25×)	Production system
Decode / user speed	~14k tok/s/GPU · ~230 tok/s user (code ~300)	Production system
Efficient unit	~64P + 64D GPUs · P:D ≈ 1:1	Production / Q&A
Topology hint	Prefill often 8-GPU servers; Decode 64–128 GPU supernodes	Production / Q&A

DSpark table caveat. Confidence-scheduled vs fixed-5 gains in the Q&A (e.g. roughly +4–7% around 60 conc/rank, up to ~+11% at higher concurrency in the published table) are from production / EP64·DP64 offline benchmark accounting — not a promise of your laptop build.

What is DSpark?

Draft tokens + confidence-scheduled verification. Enable with DEEPINFER_ENABLE_MTP_VARLEN=1; draft length via DEEPINFER_USE_NEXTN_DECODE (default 5).

SWA Bounded Replay?

Replay a bounded decoder suffix. V4.1 README defaults: compact mode and KEEP_TOKENS=140. Production discussants also describe a 128-token SWA window and bounded replay of about the last 128 tokens — still one engine, two phases.

Why missing KV / PD disagg / DP LB?

OSS focuses on offline inference. A complete service still needs a KV cache system, VE/Prefill/Decode disaggregation service glue, and DP load balancing — listed upstream as not included.

Docker image?

No official Docker image in the README. Any Dockerfile you see on community blogs is a community example only — not an official DeepSeek artifact.

Medical deepinfer.org?

Unrelated. This guide is deepinfer.run.

License?

DeepInfer Apache-2.0 · deepseek-recipe MIT.

Sources: README · inference_deployment_qa.md · eval/README · WeChat explainers (rewritten, not copied)