Text generation · served with ONNX Runtime · ↓ 5K downloads a month · View on Hugging Face
These ONNX files are an export for running in a browser: GridPup serves them as tensors in and out, not as a chat. For chat, pick the same model in its original form (served with vLLM).
Weights
6.2 GB
GPU memory needed
about 10.3 GB
Memory · disk
16.4 GB · 38 GB
Licence
not stated
Versions
Smaller versions (8-bit, 4-bit) need less GPU memory and answer a little less precisely.
Prices are per hour for the whole machine and its disk, right now. Speed is Vast's DLPerf score of the machine, relative to the cheapest. Best value: the most speed per dollar. Only machines with a Vast reliability above 95% that download at 400 Mbit/s or more are offered: the machine fetches the model itself.