Models / Qwen3.8-2.4T-A95B / Qwen3.8-2.4T-A95B ~2.45e+03B

Qwen/Qwen3.8-2.4T-A95B

Text generation · served with vLLM · ↓ 25.7K downloads a month · View on Hugging Face

Weights: 4892 GB
GPU memory needed: about 5441.6 GB
Conversation length: 8,192 tokens
Model allows: 262,144 tokens
Memory · disk: 2454.2 GB · 5422 GB
Architecture: Qwen3_5MoeForCausalLM
Licence: other

Versions of Qwen3.8-2.4T-A95B ~2.45e+03B

The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.

Version	Runtime	Weights	GPU memory	Cheapest machine now
OriginalQwen/Qwen3.8-2.4T-A95B	vLLM	4892 GB	5441.6 GB	None big enough now
FP8Qwen/Qwen3.8-2.4T-A95B-FP8	vLLM	2496 GB	2779.1 GB	None big enough now
Quark MXFP4 (amd)amd/Qwen3.8-2.4T-A95B-Quark-MXFP4	vLLM	1372 GB	1530.5 GB	None big enough now
NVFP4 (nvidia)nvidia/Qwen3.8-2.4T-A95B-NVFP4	vLLM	1444 GB	1610.6 GB	None big enough now

Hardware and deploy

No machine available right now can serve the model: the largest its 8 GPUs have 140 GB each and the model needs about 682.1 GB on each.

Endpoint name

billed by the hour; stop it from its page. GridPup rents the machine with vLLM, which downloads the model from Hugging Face (usually ready in 10 to 15 minutes), and gives you an https address, keys and a Playground.

Prices are per hour for the whole machine and its disk, right now. Speed is Vast's DLPerf score of the machine, relative to the cheapest. Best value: the most speed per dollar. Only machines with a Vast reliability above 95% that download at 400 Mbit/s or more are offered: the machine fetches the model itself.

Loading…