How much GPU memory does tiny-aya-earth need?

About 10.3 GB for conversations of up to 8,192 tokens: 6.7 GB of weights, the cache for one full-length conversation and the runtime's own memory (an estimate: its settings are behind its licence).

What is the cheapest GPU to run tiny-aya-earth?

Right now one RTX 3060 at $0.11 an hour; RTX 5090 at $0.84 an hour is the best value for its speed. Prices are live offers from GPU clouds, checked every 15 minutes.

Can tiny-aya-earth run on an RTX 4090?

Yes: it needs about 10.3 GB and an RTX 4090 has 24 GB.

Models / tiny-aya / tiny-aya ~3.35B

CohereLabs/tiny-aya-earth

Text generation · served with vLLM · ↓ 6.4K downloads a month · View on Hugging Face

CohereLabs/tiny-aya-earth needs about 10.3 GB of GPU memory for conversations of up to 8,192 tokens (an estimate: its settings are behind its licence). It runs on one RTX 3060 from $0.11/h right now; best value: RTX 5090 at $0.84/h.

The model's publisher asks users to accept its licence (cc-by-nc-4.0). GridPup cannot download this one yet: choose an ungated version below.

Weights: 6.7 GB
GPU memory needed: about 10.3 GB (estimate)
Conversation length: 8,192 tokens
Memory · disk: 16 GB · 48 GB
Architecture: Cohere2ForCausalLM
Licence: cc-by-nc-4.0

Versions of tiny-aya ~3.35B

The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.

Version	Runtime	Weights	GPU memory	Cheapest machine now
globalCohereLabs/tiny-aya-globalLicence to accept	vLLM	6.7 GB	~10.3 GB	$0.11/h RTX 3060
earthCohereLabs/tiny-aya-earthLicence to accept	vLLM	6.7 GB	~10.3 GB	$0.11/h RTX 3060

Hardware and deploy

The publisher has not let GridPup download this licence-gated model yet. An ungated version of it (above) can be deployed.

Questions

How much GPU memory does tiny-aya-earth need?: About 10.3 GB for conversations of up to 8,192 tokens: 6.7 GB of weights, the cache for one full-length conversation and the runtime's own memory (an estimate: its settings are behind its licence).
What is the cheapest GPU to run tiny-aya-earth?: Right now one RTX 3060 at $0.11 an hour; RTX 5090 at $0.84 an hour is the best value for its speed. Prices are live offers from GPU clouds, checked every 15 minutes.
Can tiny-aya-earth run on an RTX 4090?: Yes: it needs about 10.3 GB and an RTX 4090 has 24 GB.

Loading…

CohereLabs/tiny-aya-earth

Text generation · served with vLLM · ↓ 6.4K downloads a month · View on Hugging Face

The model's publisher asks users to accept its licence (cc-by-nc-4.0). GridPup cannot download this one yet: choose an ungated version below.

Weights

6.7 GB

GPU memory needed

about 10.3 GB (estimate)

Conversation length

8,192 tokens

Memory · disk

16 GB · 48 GB

Architecture

Cohere2ForCausalLM

Licence

cc-by-nc-4.0

Versions of tiny-aya ~3.35B

The same model packaged differently: a quantized version needs less GPU memory and runs on a cheaper machine.

Version	Runtime	Weights	GPU memory	Cheapest machine now
globalCohereLabs/tiny-aya-globalLicence to accept	vLLM	6.7 GB	~10.3 GB	$0.11/h RTX 3060
earthCohereLabs/tiny-aya-earthLicence to accept	vLLM	6.7 GB	~10.3 GB	$0.11/h RTX 3060

Hardware and deploy

The publisher has not let GridPup download this licence-gated model yet. An ungated version of it (above) can be deployed.

Questions

How much GPU memory does tiny-aya-earth need?: About 10.3 GB for conversations of up to 8,192 tokens: 6.7 GB of weights, the cache for one full-length conversation and the runtime's own memory (an estimate: its settings are behind its licence).
What is the cheapest GPU to run tiny-aya-earth?: Right now one RTX 3060 at $0.11 an hour; RTX 5090 at $0.84 an hour is the best value for its speed. Prices are live offers from GPU clouds, checked every 15 minutes.
Can tiny-aya-earth run on an RTX 4090?: Yes: it needs about 10.3 GB and an RTX 4090 has 24 GB.