The VRAM tier list, and exactly which AI models your GPU can run
Future Technology
New article published
Hardware
The VRAM tier list, and exactly which AI models your GPU can run
8GB runs a 7B model. 48GB runs a 70B one. Almost every wasted download and failed load in local AI sits in the gap between those two numbers, because local LLM VRAM requirements get quoted as ranges rather than as the hard ceiling they actually are.
Key Takeaways
- 7B models need about 8GB of VRAM, 13B models want 12 to 16GB, and 70B models need 35 to 48GB depending on quantisation.
- The RTX 3090, 4090 and 5090 all sit at 24GB, which makes a used 3090 the best value per gigabyte on the market.
- Apple silicon shares unified memory, so an M5 Max at 64GB runs 70B models that would otherwise need two 24GB cards.
- System RAM is the second wall: 32GB works for small models, 64GB is the real target, 128GB opens up the large ones.
You received this because you subscribe to Future Technology.