How much GPU memory does Falcon3-7B-Base need?

About 19.8 GB for conversations of up to 8,192 tokens: 14.91 GB of weights, the cache for one full-length conversation and the runtime's own memory.

What is the cheapest GPU to run Falcon3-7B-Base?

Right now one RTX 3080 at $0.18 an hour; RTX 5090 at $0.67 an hour is the best value for its speed. Prices are live offers from GPU clouds, checked every 15 minutes.

Can Falcon3-7B-Base run on an RTX 4090?

Yes: it needs about 19.8 GB and an RTX 4090 has 24 GB.

How do I deploy Falcon3-7B-Base as an API?

Pick a machine on its GridPup page and press Deploy: GridPup rents the GPU, downloads the model from Hugging Face and serves it at an https address with keys and OpenAI's chat API, billed by the hour.

Models / Falcon3 / Falcon3 7B

tiiuae/Falcon3-7B-Base

Text generation · served with vLLM · ↓ 4.8K downloads a month · View on Hugging Face

tiiuae/Falcon3-7B-Base needs about 19.8 GB of GPU memory for conversations of up to 8,192 tokens. It runs on one RTX 3080 from $0.18/h right now; best value: RTX 5090 at $0.67/h.

Weights: 15 GB
GPU memory needed: about 19.8 GB
Conversation length: 8,192 tokens
Model allows: 32,768 tokens
Memory · disk: 16 GB · 57 GB
Architecture: LlamaForCausalLM
Licence: other

Versions of Falcon3 7B

The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.

Version	Runtime	Weights	GPU memory	Cheapest machine now
Originaltiiuae/Falcon3-7B-Base	vLLM	15 GB	19.8 GB	$0.17/h RTX 3080
Instructtiiuae/Falcon3-7B-Instruct	vLLM	15 GB	19.8 GB	$0.17/h RTX 3080
Base (Prince-1)Prince-1/Falcon3-7B-Base	ONNX Runtime GenAI	15 GB	19.6 GB	$0.16/h Quadro RTX 6000

Hardware and deploy

Endpoint name

$0.67/h from your balance while it runs, billed by the hour; stop it from its page. GridPup rents the machine, which downloads the model from Hugging Face (usually ready in 10 to 15 minutes), and gives you an https address, keys and a Playground.

Questions

How much GPU memory does Falcon3-7B-Base need?: About 19.8 GB for conversations of up to 8,192 tokens: 14.91 GB of weights, the cache for one full-length conversation and the runtime's own memory.
What is the cheapest GPU to run Falcon3-7B-Base?: Right now one RTX 3080 at $0.18 an hour; RTX 5090 at $0.67 an hour is the best value for its speed. Prices are live offers from GPU clouds, checked every 15 minutes.
Can Falcon3-7B-Base run on an RTX 4090?: Yes: it needs about 19.8 GB and an RTX 4090 has 24 GB.
How do I deploy Falcon3-7B-Base as an API?: Pick a machine on its GridPup page and press Deploy: GridPup rents the GPU, downloads the model from Hugging Face and serves it at an https address with keys and OpenAI's chat API, billed by the hour.

Loading…

tiiuae/Falcon3-7B-Base

Text generation · served with vLLM · ↓ 4.8K downloads a month · View on Hugging Face

tiiuae/Falcon3-7B-Base needs about 19.8 GB of GPU memory for conversations of up to 8,192 tokens. It runs on one RTX 3080 from $0.18/h right now; best value: RTX 5090 at $0.67/h.

Weights

15 GB

GPU memory needed

about 19.8 GB

Conversation length

8,192 tokens

Model allows

32,768 tokens

Memory · disk

16 GB · 57 GB

Architecture

LlamaForCausalLM

Licence

other

Versions of Falcon3 7B

The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.

Version	Runtime	Weights	GPU memory	Cheapest machine now
Originaltiiuae/Falcon3-7B-Base	vLLM	15 GB	19.8 GB	$0.17/h RTX 3080
Instructtiiuae/Falcon3-7B-Instruct	vLLM	15 GB	19.8 GB	$0.17/h RTX 3080
Base (Prince-1)Prince-1/Falcon3-7B-Base	ONNX Runtime GenAI	15 GB	19.6 GB	$0.16/h Quadro RTX 6000

Hardware and deploy

Endpoint name

Questions

How much GPU memory does Falcon3-7B-Base need?: About 19.8 GB for conversations of up to 8,192 tokens: 14.91 GB of weights, the cache for one full-length conversation and the runtime's own memory.
What is the cheapest GPU to run Falcon3-7B-Base?: Right now one RTX 3080 at $0.18 an hour; RTX 5090 at $0.67 an hour is the best value for its speed. Prices are live offers from GPU clouds, checked every 15 minutes.
Can Falcon3-7B-Base run on an RTX 4090?: Yes: it needs about 19.8 GB and an RTX 4090 has 24 GB.
How do I deploy Falcon3-7B-Base as an API?: Pick a machine on its GridPup page and press Deploy: GridPup rents the GPU, downloads the model from Hugging Face and serves it at an https address with keys and OpenAI's chat API, billed by the hour.