DeepSeek V4.1 Flash Self-Hosted: VRAM, GPUs and vLLM 2026
Run DeepSeek V4.1 Flash self-hosted in the EU: 510 GB checkpoint, VRAM math, 8x H200 or 4x B200 setups, vLLM and SGLang commands, MIT license, GDPR.
TL;DR
DeepSeek V4.1 Flash self-hosted needs about 510 GB of GPU memory for the official FP8/FP4 checkpoint, so plan one 8x H200 node or four B200 GPUs. Serve it with a vLLM nightly or the SGLang preview image, since no stable release supports it yet. The MIT license allows commercial use, and no prompt data reaches DeepSeek.
What is DeepSeek V4.1 Flash?
DeepSeek V4.1 Flash is an open-weight multimodal Mixture-of-Experts model released on Hugging Face on 10 September 2026. It has 552B backbone parameters, activates 8B per token during prefill and 16B during decode, reads images and text, writes text, and supports contexts up to one million tokens. Weights are MIT licensed.
The model card describes a Causal Encoder-Decoder: 40 Transformer layers split into a 20-layer encoder and a 20-layer decoder. The decoder’s global KV cache is projected from the encoder output, which is why prefill only touches 8B parameters. On top of the backbone sits Engram, a 196B-parameter conditional memory looked up by token n-grams. You have to load it, even though each token touches only a few rows.
| Property | Value | Source |
|---|---|---|
| Backbone parameters | 552B | Model card |
| Active parameters | 8B prefill, 16B decode | Model card |
| Engram memory | 196B parameters | Model card |
| Experts | 1 shared + 384 routed, 6 routed active | Model card, config.json |
| Context | 1,048,576 tokens (YaRN factor 16 over 65,536) | config.json |
| Modalities | Image and text in, text out | Model card |
| Weight precision | FP8 dense (32x32 blocks, UE8M0 scales), FP4 experts | config.json |
| Global KV cache | 890 bytes per token | Model card |
| License | MIT (repository and weights) | Model card |
The card’s own benchmarks, run at maximum reasoning effort, put V4.1 Flash at 74.2 on DeepSWE v1.1 and 90.6 on Terminal-Bench 2.1. These are vendor numbers from the card, not independent measurements. Two practical details matter before deployment: the release ships no Jinja chat template, only a Python reference encoder, and the card recommends temperature=1.0 with top_p 0.95 and a max_tokens budget of at least 256K for reasoning.
How much VRAM does DeepSeek V4.1 Flash need?
The official checkpoint is 48 safetensors shards totalling 510.3 GB (475.3 GiB). That is the floor for weights on GPU. Add KV cache, activations, CUDA graphs and the sparse-attention indexer buffer. The vLLM recipe sets a planning minimum of 614 GB, which is the checkpoint size times a 1.2 headroom factor.
We measured the 510.3 GB by summing the shard sizes in the Hugging Face repository at revision dba1be0a. The vLLM recipe breaks it down: MXFP4 routed and draft experts 259.5 GiB, Engram tables in FP8 183.1 GiB, UE8M0 block scales 21.9 GiB, attention and dense projections 6.9 GiB, embedding and LM head 3.9 GiB. Engram is the surprise. It is more than a third of the checkpoint.
| Precision variant | Published by | Weights | Status |
|---|---|---|---|
| Native FP8 dense + MXFP4 experts | DeepSeek | 510.3 GB (measured) | Official, what the recipes serve |
| NVFP4 experts | NVIDIA | about 492 GiB per NVIDIA card | Official NVIDIA quant, Blackwell only |
| BF16 full upcast | nobody | about 1,526 GB (estimate) | Not published, not useful |
| 2-bit to 4-bit GGUF | community only | varies | No upstream llama.cpp support |
The BF16 line is our estimate: 557.2B expert entries plus 196.6B Engram plus 7.4B dense plus 2.0B embedding parameters, about 763B values times 2 bytes, gives roughly 1.53 TB. Nobody ships it and there is no reason to upcast a model that ships with FP4 experts. The NVIDIA NVFP4 checkpoint does not save memory either: NVIDIA states it grows from about 476 GiB to 492 GiB because of the finer scale layout. Its benefit is W4A4 compute on Blackwell.
KV cache estimate. The card gives 890 bytes of global KV cache per token. Our arithmetic, excluding the fixed 128-token sliding window per layer and runtime overhead:
- One full 1,048,576-token context: 1,048,576 x 890 B = 0.93 GB (estimate)
- 32 concurrent requests at 128K tokens: 32 x 131,072 x 890 B = 3.7 GB (estimate)
- 256 concurrent requests at 32K tokens: 256 x 32,768 x 890 B = 7.5 GB (estimate)
KV cache is therefore not the limiting factor. Runtime buffers are. The vLLM recipe notes that the indexer allocates a logits buffer of max batched tokens times max model length: 8,192 x 1M x 2 bytes is exactly 16 GiB. Budget at least 100 GB above the weights for a production server, which matches the recipe’s 614 GB minimum. That 100 GB figure is our estimate.
Which GPU setup runs it?
The safe choice is one 8x H200 node: 1,128 GB of HBM, verified by both vLLM and SGLang. Four B200 GPUs (720 GB) are verified as well. An 8x H100 node (640 GB) works only in vLLM, and only with Engram offloaded to host RAM. Four H200 cards fall below the recipe minimum.
GPU memory figures come from NVIDIA datasheets: H100 80 GB, H200 141 GB, DGX B200 1,440 GB for 8 GPUs (180 GB each), RTX PRO 6000 Blackwell 96 GB.
| Setup | Total GPU memory | Left after 510.3 GB weights | Engine status (19 Sep 2026) |
|---|---|---|---|
| 8x H200 141 GB | 1,128 GB | 617.7 GB | Verified in vLLM and SGLang (TP8) |
| 4x B200 180 GB | 720 GB | 209.7 GB | Verified in vLLM and SGLang (TP4) |
| 8x RTX PRO 6000 96 GB | 768 GB | 257.7 GB | Not in the official recipes; community reports only |
| 8x H100 80 GB | 640 GB | 129.7 GB | vLLM only, with Engram CPU offload |
| 4x H200 141 GB | 564 GB | 53.7 GB | Below the 614 GB minimum; we advise against it |
| 4x RTX PRO 6000 96 GB | 384 GB | does not fit | No |
The H100 case needs explanation. The vLLM recipe calls 8x 80 GB “the smallest NVIDIA node that serves this checkpoint”, and only with --engram-config '{"cpu_offload":true}', which moves the 183 GiB of Engram tables into pinned host DRAM. Your server then needs that much free system RAM on top of everything else. SGLang does not list H100 at all.
For RTX PRO 6000, the arithmetic fits on eight cards, but neither official recipe marks it as verified. The vLLM issue tracker contains a community field report for 8x RTX PRO 6000 and a separate bug about very low decode throughput on that setup. Treat it as a lab configuration, not a production target.
Our recommendation: rent or buy 8x H200 if you run Hopper, and 4x B200 if you can get Blackwell. Both leave room for long contexts and batching without offload tricks.
How do you serve it with vLLM or SGLang?
No stable release of either engine supports DeepSeek V4.1 Flash yet. vLLM’s recipe requires version 0.30.0, which has not shipped, and points to the vllm/vllm-openai:nightly image. SGLang’s cookbook says support “has not shipped in an SGLang release yet” and uses the preview image lmsysorg/sglang:dev-dsv41.
The latest vLLM release, v0.29.0, came out on 9 September 2026, one day before the model. The architecture landed on main in PR 56228 on 10 September. Pin a specific nightly digest in production, because nightly images change daily.
vLLM on 8x H200. The arguments below come from the vLLM recipe (base arguments, parsers, text-only flag). Run them inside vllm/vllm-openai:nightly:
export VLLM_ENGINE_READY_TIMEOUT_S=3600
vllm serve deepseek-ai/DeepSeek-V4.1-Flash \
--tensor-parallel-size 8 \
--tokenizer-mode deepseek_v41 \
--reasoning-parser deepseek_v41 \
--tool-call-parser deepseek_v41 --enable-auto-tool-choice \
--language-model-onlyDrop --language-model-only if you need image input. On 8x H100, the recipe adds --engram-config '{"cpu_offload":true}' --max-num-batched-tokens 4096 --gpu-memory-utilization 0.92 plus the environment variables VLLM_USE_V2_MODEL_RUNNER=1 and PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True. Note one trap from the recipe: in vLLM, thinking is on at effort 50 when you send no reasoning settings, so a small max_tokens returns empty content.
SGLang on 8x H200. This is the verified high-throughput cell from the SGLang launch configs:
docker run --gpus all --shm-size 32g -p 30000:30000 --ipc=host \
-v ~/.cache/huggingface:/root/.cache/huggingface \
lmsysorg/sglang:dev-dsv41 \
sglang serve --trust-remote-code \
--model-path deepseek-ai/DeepSeek-V4.1-Flash \
--tp 8 --ep-size 8 --mem-fraction-static 0.8 \
--attention-backend dsv4 --moe-runner-backend flashinfer_mxfp4 \
--max-running-requests 256 --cuda-graph-max-bs-decode 64 \
--reasoning-parser auto --tool-call-parser auto \
--host 0.0.0.0 --port 30000On B200, B300 and GB300 the cookbook uses --tp 4 --ep-size 4 and tells you not to override the backends. In SGLang, thinking is off by default until a request sends reasoning_effort. Because --trust-remote-code executes repository code, pin the model revision you reviewed.
Smoke test (from the vLLM recipe, port 8000; use 30000 for SGLang). A correct answer is 323:
curl http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{"model":"deepseek-ai/DeepSeek-V4.1-Flash","messages":[{"role":"user","content":"What is 17*19? Return only the integer."}]}'Ollama and llama.cpp. Not an option for local use today. The Ollama library lists only deepseek-v4.1-flash:cloud, which runs on Ollama’s servers, so your prompts leave your infrastructure. llama.cpp’s conversion support is an open, unmerged pull request. Community GGUF files exist on Hugging Face, but none comes from DeepSeek or the llama.cpp maintainers.
Is DeepSeek V4.1 Flash available on Bedrock or Google Cloud in the EU?
No. As of 19 September 2026, Amazon Bedrock documents DeepSeek-R1, V3.1 and V3.2, but no V4 or V4.1 model. Google Cloud’s managed DeepSeek offering lists V3.2, V3.1, R1-0528 and OCR. If you want V4.1 Flash with EU data residency, you host it yourself.
The Bedrock model list has model cards for three DeepSeek models. DeepSeek-V3.2 (deepseek.v3.2) is the newest, with a region table that includes eu-north-1 (Stockholm) and eu-west-2 (London), and no geo or global cross-region profile. On Google Cloud, the DeepSeek MaaS page shows the same generation gap. Which models Bedrock does keep in EU regions is mapped in our Amazon Bedrock guide. Kimi K3, another open-weight model, is on Bedrock but only through a global profile, as our Kimi K3 Bedrock analysis explains.
| Option | V4.1 Flash available | EU data residency |
|---|---|---|
| Self-hosted on your GPUs or EU colocation | Yes | Yes, you control it |
| GPU VMs from an EU cloud region, self-served | Yes, if you get H200 or B200 capacity | Yes, region of your choice |
| Amazon Bedrock | Not available | n/a |
| Google Cloud MaaS | Not available | n/a |
| Ollama cloud tag | Yes | Not documented on the model page |
If you need a managed Claude alternative with EU residency instead, our Claude on AWS Bedrock EU guide covers the EU inference profiles.
What do GDPR, the MIT license and the AI Act mean here?
Self-hosting open weights means no prompt or output ever reaches DeepSeek. The weights are a file; inference runs on hardware you control, so GDPR processing stays inside your own setup and your existing processors. The MIT license permits commercial use, modification and redistribution if you keep the copyright notice.
GDPR. With self-hosting, the classic questions about a model vendor as processor, third-country transfer and training on customer data disappear for the model layer. What remains is your own infrastructure: the hosting provider or colocation contract, access control, logging and retention. Two leaks to avoid: the Ollama :cloud tag and DeepSeek’s hosted chat or API are different data flows that send prompts to a third party. Our GDPR provider comparison shows the same logic for managed Claude setups.
License. The card states that “this repository and the model weights are licensed under the MIT License”. There are no usage restrictions, no user thresholds and no acceptable-use policy attached in the repository. The only condition is to include the copyright and permission notice when you redistribute the weights or code. NVIDIA’s NVFP4 variant is also MIT and marked “ready for commercial or non-commercial use”.
AI Act. GPAI obligations have applied to model providers since 2 August 2025, and the Act’s general application started on 2 August 2026, according to the European Commission. DeepSeek is the provider of the model. If you build an internal assistant on it, you are provider and deployer of that AI system, with duties such as AI literacy and transparency. If you fine-tune or modify the model itself, check whether provider duties for the model shift to you. Our AI Act guide for LLM deployers explains the roles.
FAQ
How much does it cost to run DeepSeek V4.1 Flash yourself?
The dominant cost is one 8x H200 node or four B200 GPUs, bought or rented. We have not verified EU rental prices, so we do not quote them. For comparison, Ollama’s hosted cloud tag lists 0.15 USD input and 0.60 USD output per 1M tokens at base rate, but that sends data to Ollama.
DeepSeek V4.1 Flash vs DeepSeek V4 Flash: what changes for hosting?
V4.1 Flash is much larger to load: 552B backbone plus 196B Engram, versus 284B backbone for V4 Flash on the card. In exchange, its global KV cache is about a quarter of V4 Flash’s, 890 bytes per token. You need more GPU memory for weights but far less for long contexts.
Can I run DeepSeek V4.1 Flash on a single GPU or a workstation?
Not the official checkpoint. At 510.3 GB of weights, even the 180 GB B200 needs three cards for weights alone, and the recipes use four. Community 2-bit quants exist, but they need non-upstream runtimes and lose accuracy. For production, treat 4x B200 or 8x H200 as the minimum.
Does self-hosting DeepSeek V4.1 Flash send data to China?
No. Inference with downloaded weights runs entirely on your servers and makes no calls to DeepSeek. Data only leaves if you use DeepSeek’s hosted API or chat, or the Ollama cloud tag. Check that your serving image and monitoring stack do not phone home.
Which engine should I pick, vLLM or SGLang?
Both have verified H200 and B200 configurations and both are pre-release for this model. Pick vLLM if you need 8x H100, which only the vLLM recipe covers. Pick SGLang if you want its verified low-latency cells with DSpark speculative decoding on Blackwell. Pin image digests either way.
Is DeepSeek V4.1 Flash on Amazon Bedrock in Frankfurt?
No. Bedrock documents DeepSeek-R1, V3.1 and V3.2 only, and V3.2’s region table does not include Frankfurt. There is no V4.1 model card on Bedrock as of 19 September 2026.
Sources
- Hugging Face: deepseek-ai/DeepSeek-V4.1-Flash model card (19 September 2026)
- Hugging Face: DeepSeek-V4.1-Flash config.json (19 September 2026)
- vLLM recipes: DeepSeek-V4.1-Flash (19 September 2026)
- SGLang cookbook: DeepSeek-V4.1 (19 September 2026)
- SGLang DeepSeek-V4.1 launch configs (19 September 2026)
- Hugging Face: nvidia/DeepSeek-V4.1-Flash-NVFP4 (19 September 2026)
- llama.cpp PR 28696: convert DeepSeek V4.1 (19 September 2026)
- Ollama library: deepseek-v4.1-flash (19 September 2026)
- NVIDIA H100 datasheet (19 September 2026)
- NVIDIA H200 datasheet (19 September 2026)
- NVIDIA DGX B200 specifications (19 September 2026)
- NVIDIA RTX PRO 6000 Blackwell Server Edition (19 September 2026)
- Amazon Bedrock: supported foundation models (19 September 2026)
- Google Cloud: DeepSeek models (MaaS) (19 September 2026)
- European Commission: AI Act regulatory framework (18 September 2026)