On October 20, 2026, DeepSeek open-sourced the stack that runs DeepSeek-V4.1-Flash in its own production: DeepInfer under Apache-2.0, with deepseek-recipe under MIT.
No new model is in this announcement. What got opened is the software that decides what every token costs to produce — a very different event from downloading weights.
- DeepInfer is an inference engine (the software that runs a model to produce responses), not a model, and not yet an API server. The open release runs offline batches only and names support for NVIDIA SM100 and Huawei Ascend 950.
- deepseek-recipe is the protocol layer. As DeepSeek describes it, it makes any community deployment speak the language of the official API. Install with
pip install deepseek-recipe; usable today. - The real economics live in the layer not yet opened: the persistent cache, prefill/decode disaggregation, routing. The figures of 155 trillion tokens per day and $38 per GPU-hour (at valley pricing) are DeepSeek's own disclosures for its internal system, with no independent verification.
In our reading, Engram is DeepSeek's most important breakthrough of 2026, and that says a great deal given that the company has just open-sourced its entire inference-serving stack to the public for free. Engram is a "conditional memory" module: a sparsity axis alongside MoE that looks up learned knowledge at constant complexity. The repo is open under Apache-2.0; the paper, from the DeepSeek-AI team with Wenfeng Liang listed among the authors, scales the module to 27 billion parameters and beats a same-size MoE baseline. V4.1-Flash itself carries Engram tables — that linkage is our reading, not a measurement.
DeepInfer: a real engine, but only halfway out the door
The announcement leads with its goals: helping the community "lower compute costs, reduce energy use, and make AI more affordable." It promises that inference code for future models will keep landing in DeepInfer, and closes with three phrases: "Open models. Open kernels. Open inference."
The most trustworthy detail is in the README: the repo was "originally built primarily on vLLM, with additional inspiration from SGLang and TensorRT-LLM." At the code level, 23 files carry a "Derived from vLLM" header. This is not a new rival to vLLM; it is DeepSeek retelling its own system in source code.
What is real in the release:
| Technique | Mechanism | DeepSeek's self-measured number |
|---|---|---|
| DSpark, speculative decoding | Draft model emits 5 tokens per step; a confidence head picks how many to verify as length varies | +4 to +7% throughput at the production operating point |
| SWA Bounded Replay | Reuses cache by recomputing only the last 128 tokens | decoder-side replay branch nearly halves prefill computation |
| Async scheduler | GPU runs the forward pass of step N while the CPU prepares step N+1 (an idea anticipated by SGLang and NanoFlow) | on by default |
| MoE parallelism | Attention runs DP with #DP = #EP; Wide-EP decode at 64–128; the repo states plainly "TP and CP are not used for now" |
EPLB already in the engine |
The familiar kernel layer of the DeepSeek system ships alongside: FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, in-house TileLang kernels, plus Ascend ports. You build from source (PyPI currently carries deepinfer only as an 0.1.0 sdist); run conditions: Linux, GCC >= 13, Python 3.12, CUDA 13 or CANN 9.2.
The scope is narrower than the headline: one model, DeepSeek-V4.1-Flash; offline batch only, through LLM.generate(). There is no OpenAI-compatible HTTP server and no continuous-batching loop for online serving. Two features are rare goods for eval and RL teams: batch-invariant inference and deterministic sampling, committed on NVIDIA hardware only; the engine also takes structured output as regex, JSON schema or EBNF. The open kernels run 5 to 10% slower than the internal versions at decode.
deepseek-recipe: the piece you can use today
If DeepInfer is the engine, deepseek-recipe is the interpreter. MIT-licensed, Rust core with Python bindings, version 0.2.0 on both PyPI and crates.io. It accepts requests as Messages, Chat Completions or Responses; converts them into the exact prompt and token format of V4/V4.1; parses the output stream, separating reasoning, body and tool calls into a well-formed response.
Three details mark it as production code: special-token injection defense (users cannot forge control tokens), segmented parallel tokenization for long contexts, and a streaming state machine that keeps UTF-8 intact across chunks. Generation-parameter semantics are pre-cooked: temperature and top_p resolve per model version, and thinking mode pins temperature to 1.0.
Two cautions before you ship anything. The recipe does not run inference, does not execute tools, and does not handle HTTP transport — per DeepSeek's README, "The caller must provide model inference, tool execution, and HTTP transport." And the default image fetcher connects to any address, so you attach a DestinationFilter to block SSRF whenever you take URLs from users. For a company planning to sell "DeepSeek inside the customer's network, behaving like the official API," this is the cheapest piece to start with.
The complete pipeline, and the part you still have to draw
Request (Messages / Chat Completions / Responses) -> deepseek-recipe: encode prompt + tokens in the V4/V4.1 format -> [ HTTP server + auth + quota: YOU BUILD THIS ] -> DeepInfer: prefill + decode engine (offline batch) -> DeepSeek-V4.1-Flash (weights ~510 GB) <- raw token stream <- deepseek-recipe: parse streaming <- [ persistent KV cache + P/D disaggregation + load balancing: YOU ASSEMBLE THIS ] Response (normalized to official-API behavior)
The two "you build this" brackets are not decoration. The README lists outright what a complete service needs that this release does not include:
| Missing piece | What DeepSeek says | What you must do |
|---|---|---|
| KV cache system | The engine has paged KV blocks; the persistent cache tier across requests is absent. DeepSeek keeps KV for over 72 hours on 3FS (about 100 PiB of SSD in its own system) | Build your own cache manager, or use vLLM/SGLang prefix caching; multi-month scale |
| Vision Encoder / Prefill / Decode disaggregation | Most of the necessary engine functionality is implemented, per DeepSeek's account: what is missing is the wiring, not the engine | A router between pools, a KV transfer path, admission control |
| Load balancing across DP replicas | Described only in the Q&A doc; DeepSeek's best balancer reaches roughly 80% of a perfectly balanced offline benchmark | Your own router sits further below that bar |
| Online HTTP API | No OpenAI-compatible server; the two sample servers in the recipe repo are mock inference | Server, streaming, auth, quota, observability |
| Operating platform | No Dockerfile, no CI, no release tags; 119 open issues, repo 5 days old as of the morning of October 21, 2026 | Containers, CI, manifests, and someone who owns the repo |
The hard limit: named support covers NVIDIA SM100 and Huawei Ascend 950 only. The repo does not claim to exclude H100, H200 or consumer GPUs, but it offers no deployment path for them either; running those cards today is the road vLLM and SGLang already paved.
Self-hosting economics: which numbers move, which hold
DeepSeek's cost advantage sits in the service layer it kept home. The 155 trillion tokens per day and $38 of revenue per GPU-hour at valley pricing are numbers for the entire system — the cache tier and the four unreleased load balancers included, none of which appear in the open release. Bloomberg notes that no independent benchmark of DeepInfer exists yet. The biggest lever is the cache engine the field test measured at 96% input cache hits: a KV tier held for more than 72 hours (a 3-day TTL for agent traffic), roughly 14.4 TB per GPU at logical size, and the prefill/decode disaggregation tier. Copying DeepInfer alone does not reproduce this economics.
The absurd-cheap option remains the official API. The price card in effect since September 10, 2026, per million tokens: input costs $0.30 on a cache miss and just $0.006 on a cache hit; output is $1.20 at peak, and every tier drops by half outside peak hours. Caching, not kernels, is the pricing story: DeepSeek's documentation computes the KV memory footprint at 890 bytes per token, about a quarter of what V4-Flash required.
Field tests outweigh any claim. A consultancy rented a 4x-H200 node to serve V4.1-Flash to its coding agents: $440.88 a day at on-demand rates, while the identical workload through the official API cost $184 to $223 a day, because 96% of the input was cache hits. Their published conclusion: rent or self-host when your data cannot leave your network, or when you keep the hardware busy around the clock; use the API when cost is the goal.
The floor of self-hosting did not move with the open release. The 510 GB checkpoint needs at minimum one 8x-H200 node or 4x-B200. DeepSeek states that the smallest efficient unit of DeepInfer is 64 prefill GPUs plus 64 decode GPUs, costing roughly $633.60 to $1,423.36 per hour at Google Cloud B200 rental rates. For most small businesses, the local tier in reach is still a quantized GGUF build on a 128 to 256 GB RAM machine: 10 to 19 tokens per second, with reduced quality.
Self-reported, but the price side checks out: start from the off-peak price card (every tier exactly half), apply 96% input cache hits and an agent workload with an input-to-output ratio of roughly 100:1, and blended revenue lands at about $14 to 15 per billion tokens. Connecting that thread to $38 per GPU-hour would still require per-GPU throughput — DeepSeek has published only charts, not numbers — so the reverse check stops at internal consistency, with no independent audit. The gap versus outside GPU rental is a comparison, not an audited margin: $38 is the valley rate, peak hours pull the average up, and the real economics are set by DeepSeek's own hardware and electricity prices in China.
For Vietnamese businesses running local AI: three things change, two do not
What changes. "Official-API behavior, inside our network" becomes a sellable commitment thanks to deepseek-recipe, hitting exactly the problem Vietnamese SMBs must answer: Decree 13/2023/ND-CP on personal data and cross-border transfer, and the 2025 Data Law. Ascend 950 support from day one opens a second hardware lane into procurement review, though no public Ascend performance figures exist and procurability in Vietnam is unverified. DeepInfer's code doubles as a free course on how V4.1-Flash was made cheap; vLLM has already thanked DeepSeek publicly and committed to folding those ideas into its support track.
What does not change. Vietnam's domestic high-end GPU base is still rented H100/H200 (FPT AI Factory, NTC, DBIZ), not SM100 floorspace. And the release's "cheaper AI for the community" goal does not turn self-hosting into the cheaper option for large token volumes: it makes DeepSeek's side cheaper. Your edge stays what it always was — control over your data.
Read the move as standardization: MIT weights, an open engine and an open recipe turn "DeepSeek-compatible" into a spec third parties must implement. Chinese tech press concluded on October 20 (translated from SMZDM): DeepInfer does not replace vLLM for individuals and small teams — what changes is the cost equation for vendors owning Blackwell or Ascend clusters. Add one more newly opened piece: DeepSeek Harness (dsh), an open-source agent runtime under MIT, now in public preview worldwide — so the chain of V4.1-Flash weights plus DeepInfer/recipe plus Harness is nearly complete for agentic and coding workloads. The wider frame: open source AI infrastructure for enterprises.
Takeaway: choose by the job, not by the press release
- You want low prices and no data constraints: the official API. Design for the cache to collect cache-hit pricing, and push batch jobs to off-peak hours, when every rate is half.
- "Never leaves our network" is a sales feature: vLLM or SGLang on an H200-class machine or a quantized build, plus deepseek-recipe to match official behavior. Integrating the recipe takes days to weeks.
- You own 128 or more SM100 GPUs (the Blackwell line, in industry shorthand) or an Ascend 950 cluster, with a dedicated infrastructure team: prototype DeepInfer for offline batch today; treat the KV store and P/D disaggregation as multi-quarter self-built projects.
- You run many model families: the generic engine wins, because DeepInfer currently supports exactly one model, DeepSeek-V4.1-Flash.
- You need bit-for-bit reproduction for eval or RL: batch-invariance and deterministic sampling are scarce goods, provided you are on the NVIDIA side.
Three things to watch: when DeepSeek opens the remaining KV cache and load balancing (no date promised), what vLLM and SGLang port from DeepInfer's techniques, and DeepSeek's own pricing — the rates have swung in both directions during 2026, so keep every cost anchor at a 90-day shelf life. At bottom, inference at this level of efficiency is the business: the advantage is not a cheap model but a system optimized until those numbers become feasible, which is why everyone wants a seat in the token flow. Want a measurement of your own workload before you commit money to GPUs? The 5ac team can run it with you.
FAQ: What Self-Hosters Ask About the DeepSeek Release
Q1: Is this a new DeepSeek model?
No new model is in this announcement. What got opened is the software that decides what every token costs to produce — a very different event from downloading weights.
Q2: Can DeepInfer serve online traffic today?
The scope is narrower than the headline: one model, DeepSeek-V4.1-Flash; offline batch only, through LLM.generate(). There is no OpenAI-compatible HTTP server and no continuous-batching loop for online serving.
Q3: What does deepseek-recipe not do?
The recipe does not run inference, does not execute tools, and does not handle HTTP transport — per DeepSeek's README, "The caller must provide model inference, tool execution, and HTTP transport." And the default image fetcher connects to any address, so you attach a DestinationFilter to block SSRF whenever you take URLs from users.
Q4: Does the open release make self-hosting the cheaper option?
And the release's "cheaper AI for the community" goal does not turn self-hosting into the cheaper option for large token volumes: it makes DeepSeek's side cheaper.
References
- DeepInfer GitHub: README on the offline scope, the not-included list, SM100/Ascend 950, toolchain.
- DeepInfer Q&A doc: production architecture, 155 trillion tokens per day and $38 per GPU-hour (self-reported), the 64+64 GPU threshold, balancer design, KV retention beyond 72 hours and about 14.4 TB per GPU at logical size.
- deepseek-recipe GitHub: protocol scope, generation settings, non-features, mock sample servers.
- DeepSeek's X announcement, October 20, 2026: goals of lower compute costs, energy savings, more affordable AI.
- DeepSeek architecture blog on Zhihu (Chinese; details used here are translated): SWA Bounded Replay, DSpark, the 3FS tier.
- Official DeepSeek API pricing: input $0.30/$0.006, output $1.20, off-peak halves every tier.
- Bloomberg via Berkeley Law: no independent benchmark of DeepInfer yet.
- Tensorrigs hardware guide, summarizing Tom's Hardware's test: the 4x-H200 field test versus the API, 96% cache hits.
- llm-integration.eu: 510 GB checkpoint, a floor of 8x-H200 or 4x-B200, and the 8x-H100 path only with Engram tables offloaded to host RAM.
- TensorUST: B200 rental rates on Google Cloud, Ascend repo and kernels predating the launch.
- Unsloth docs: the local GGUF tier at 10 to 19 tokens per second, with Engram tables streamed from SSD.
- vLLM README and SGLang docs: online serving, PD disaggregation, prefix caching.
- SMZDM, October 20, 2026 (Chinese; quotes translated): DeepInfer compared with the major frameworks.
- KWM APAC Regulatory Guide Vietnam: Decree 13, the 2025 Data Law.
- DeepSeek-V4.1-Flash paper (arXiv): KV at 890 bytes per token.
- DeepSeek Engram GitHub: the Engram module, Apache-2.0 license.
- Engram paper (arXiv): conditional memory, 27 billion parameters, DeepSeek-AI author list.
- Daily.dev digest: vLLM's public response.