IFM/K2-Horizon-32B-FP8

Text generation · served with vLLM · ↓ 7.7K downloads a month · View on Hugging Face

Weights: 37 GB
GPU memory needed: about 46.1 GB
Conversation length: 8,192 tokens
Model allows: 524,288 tokens
Memory · disk: 26.7 GB · 82 GB
Architecture: K2HorizonForCausalLM
Licence: apache-2.0

Versions of K2-Horizon 32B

The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.

Version	Runtime	Weights	GPU memory	Cheapest machine now
OriginalIFM/K2-Horizon-32B	vLLM	70 GB	81.9 GB	$0.90/h 2x RTX A6000
FP8IFM/K2-Horizon-32B-FP8	vLLM	37 GB	46.1 GB	$0.60/h Nvidia A40

Hardware and deploy

Endpoint name

$1.14/h from your balance while it runs, billed by the hour; stop it from its page. GridPup rents the machine with vLLM, which downloads the model from Hugging Face (usually ready in 10 to 15 minutes), and gives you an https address, keys and a Playground.

Loading…