Models / gpt2 / gpt2 ~137M

Xenova/gpt2

Text generation · served with ONNX Runtime · ↓ 7.5K downloads a month · View on Hugging Face

These ONNX files are an export for running in a browser: GridPup serves them as tensors in and out, not as a chat. For chat, pick the same model in its original form (served with vLLM).

Weights: 500 MB
GPU memory needed: about 1.7 GB
Memory · disk: 8 GB · 30 GB
Licence: not stated

Versions

Smaller versions (8-bit, 4-bit) need less GPU memory and answer a little less precisely.

full precisiononnx/decoder_model.onnx500 MB · 1.7 GB GPU full precisiononnx/decoder_model_merged.onnx500 MB · 1.8 GB GPU 8-bitonnx/decoder_model_merged_quantized.onnx130 MB · 1.2 GB GPU 8-bitonnx/decoder_model_quantized.onnx130 MB · 1.2 GB GPU full precisiononnx/decoder_with_past_model.onnx500 MB · 1.7 GB GPU 8-bitonnx/decoder_with_past_model_quantized.onnx130 MB · 1.2 GB GPU full precisiononnx/model.onnx500 MB · 1.8 GB GPU 4-bit (bnb)onnx/model_bnb4.onnx500 MB · 1.7 GB GPU float16onnx/model_fp16.onnx250 MB · 1.4 GB GPU 8-bitonnx/model_int8.onnx280 MB · 1.4 GB GPU 4-bitonnx/model_q4.onnx500 MB · 1.7 GB GPU 4-bit, float16onnx/model_q4f16.onnx250 MB · 1.4 GB GPU 8-bitonnx/model_uint8.onnx280 MB · 1.4 GB GPU

Versions of gpt2 ~137M

The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.

Version	Runtime	Weights	GPU memory	Cheapest machine now
Originalopenai-community/gpt2	vLLM	270 MB	2.6 GB	$0.10/h RTX 3060
Main (Xenova)Xenova/gpt2	ONNX Runtime	130 MB	1.2 GB	$0.08/h GTX 1660 SUPER

Hardware and deploy

Deploying ONNX models from the catalog is coming soon. Today, download the files from Hugging Face and upload them in Inference.

Loading…