Loading…
Loading…
Text generation · served with ONNX Runtime GenAI · ↓ 0 downloads a month · View on Hugging Face
Smaller versions (8-bit, 4-bit) need less GPU memory and answer a little less precisely.
The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.
| Version | Runtime | Weights | GPU memory | Cheapest machine now |
|---|---|---|---|---|
| OriginalQwen/Qwen3.5-4B-Base | vLLM | 9.3 GB | 13.8 GB | $0.16/h RTX 4060 Ti |
| InstructQwen/Qwen3.5-4B | vLLM | 9.3 GB | 13.8 GB | $0.16/h RTX 4060 Ti |
| AWQ (sanskar003)sanskar003/Qwen3.5-4B-AWQLicence to accept | vLLM | 5.0 GB | ~8.3 GB | $0.10/h RTX 3060 |
| AWQ 4bit (cyankiwi)cyankiwi/Qwen3.5-4B-AWQ-4bit | vLLM | 4.0 GB | 7.9 GB | $0.10/h RTX 3060 |
| AWQ (QuantTrio)QuantTrio/Qwen3.5-4B-AWQ | vLLM | 6.1 GB | 10.2 GB | $0.10/h RTX 3060 |
| super coder (jica98)jica98/qwen3.5-4B-super-coder | vLLM | 80 MB | 3.5 GB | $0.10/h RTX 3060 |
| onnx int4 cuda (md-bogush)md-bogush/Qwen3.5-4B-onnx-int4-cuda | ONNX Runtime GenAI | 3.7 GB |
Deploying ONNX models from the catalog is coming soon. Today, download the files from Hugging Face and upload them in Inference.
| 6.9 GB |
| $0.09/h Titan Xp |
| onnx (chayen1)chayen1/robotlegs-qwen3.5-4b-onnx | ONNX Runtime GenAI | 3.8 GB | 7 GB | $0.09/h Titan Xp |
Prices are per hour for the whole machine and its disk, right now. Speed is Vast's DLPerf score of the machine, relative to the cheapest. Best value: the most speed per dollar. Only machines with a Vast reliability above 95% that download at 400 Mbit/s or more are offered: the machine fetches the model itself.