Loading…
Loading…
Text generation · served with ONNX Runtime GenAI · ↓ 0 downloads a month · View on Hugging Face
Smaller versions (8-bit, 4-bit) need less GPU memory and answer a little less precisely.
The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.
| Version | Runtime | Weights | GPU memory | Cheapest machine now |
|---|---|---|---|---|
| OriginalQwen/Qwen3-1.7B-Base | vLLM | 3.4 GB | 7.1 GB | $0.10/h RTX 3060 |
| InstructQwen/Qwen3-1.7B | vLLM | 4.1 GB | 7.8 GB | $0.10/h RTX 3060 |
| FP8Qwen/Qwen3-1.7B-FP8 | vLLM | 2.6 GB | 6.2 GB | $0.10/h RTX 3060 |
| unsloth bnb 4bit (unsloth)unsloth/Qwen3-1.7B-unsloth-bnb-4bit | vLLM | 1.4 GB | 4.8 GB | $0.10/h RTX 3060 |
| oga (SamTheDev)SamTheDev/qwen3-1.7B-oga | ONNX Runtime GenAI | 1.6 GB | 4.3 GB | $0.08/h GTX 1660 SUPER |
| int4 onnx genai (Hug0007)Hug0007/qwen3-1.7b-int4-onnx-genai | ONNX Runtime GenAI | 1.1 GB | 3.7 GB | $0.08/h GTX 1660 SUPER |
Deploying ONNX models from the catalog is coming soon. Today, download the files from Hugging Face and upload them in Inference.
| ONNX Runtime GenAI |
| 1.1 GB |
| 3.7 GB |
| $0.08/h GTX 1660 SUPER |
| int4 onnx (divakar7c5)divakar7c5/wealthwand-qwen3-1.7b-int4-onnx | ONNX Runtime GenAI | 1.1 GB | 3.7 GB | $0.08/h GTX 1660 SUPER |
Prices are per hour for the whole machine and its disk, right now. Speed is Vast's DLPerf score of the machine, relative to the cheapest. Best value: the most speed per dollar. Only machines with a Vast reliability above 95% that download at 400 Mbit/s or more are offered: the machine fetches the model itself.