The one number that decides what you can run
Forget clock speeds and cores. For local AI, the number that matters is your graphics card's VRAM (its own memory). The model has to fit in it to run fast. If it doesn't fit, the extra spills into normal RAM and runs on the processor, which works but is much slower.
Find it on Windows: press Ctrl + Shift + Esc, click Performance, click GPU 0 (and GPU 1 if you have it), and read Dedicated GPU memory at the bottom. Ignore "shared GPU memory".
Which models fit which card
These are download sizes from the Ollama library (November 2026). Leave a little room on the card for the conversation itself.
| Card (VRAM) | Run this | Size | Notes |
|---|---|---|---|
| GTX 1050 Ti (4 GB) | qwen3.5:4b |
3.3 GB | Fits, small model |
| GTX 1060 6GB | qwen3.5:4b or qwen3.5:9b |
3.3 / 6.6 GB | The 9B slightly spills over; slower but works |
| GTX 1070 / 1080, RTX 2060 / 2070 (8 GB) | qwen3.5:9b |
6.6 GB | Good all-rounder |
| GTX 1080 Ti (11 GB), RTX 2080 Ti (11 GB) | gemma4:12b |
7.7 GB | Reads photos too |
| RTX 3060 12GB | gemma4:12b |
7.7 GB | Our value pick |
| RTX 4060 Ti 16GB, RTX 4080 (16 GB) | gemma4:26b |
16 GB | Mixture of experts: fast for its size |
| RTX 3090 / 4090 (24 GB) | qwen3.6:27b |
18 GB | Among the best home models |
"Mixture of experts" models like gemma4:26b and qwen3.6:35b-a3b only use a small part of themselves for each word. They run faster than their size suggests and cope better when part of them spills into RAM, so they're worth trying if you have lots of system RAM.
Is my old GTX card still supported?
Ollama, the free program most people use to run local models, supports NVIDIA cards with compute capability 5.0 and up. That covers the GTX 750 Ti, the whole GTX 900 and GTX 10 series, and every RTX card. One catch: GTX 900 and 10-series cards need NVIDIA driver 570 or newer. If your old card isn't being used, update the driver first.
AMD Radeon cards work too. Newer cards (RX 6800 and up, RX 7000 and 9000 series) have full support, and Ollama can use many others through Vulkan.
How to check it's really using the GPU
After you run a model, open PowerShell and type:
ollama ps
The PROCESSOR column should say 100% GPU. A split like 30%/70% CPU/GPU means the model is a bit too big for your card; try the next size down. 100% CPU means the GPU isn't being used at all; update your driver and restart.
Quick start in three commands
- Install Ollama from ollama.com (Windows, Mac or Linux).
- Open PowerShell and download a model for your card, for example:
ollama pull gemma4:12b
- Chat with it:
ollama run gemma4:12b
For a proper chat screen with history, file upload and your own document library, add Open WebUI.
Worth buying a used card for this?
If your card has 4 GB or less, a used RTX 3060 12GB is the best value upgrade for local AI: 12 GB of memory at about 170 W. A GTX 1080 Ti (11 GB) is often cheaper but slower and hungrier at about 250 W. A RTX 3090 (24 GB) runs the best home models, but needs a 750 W power supply and draws about 350 W.
Questions
Can I run AI locally without a GPU?
Yes, small models like qwen3.5:4b run on the processor alone. Expect slow answers, often a word or two per second, compared with a full paragraph in a few seconds on a graphics card. Fine for occasional use, frustrating every day.
How much VRAM do I need for a local LLM?
8 GB runs a good 9B model. 12 GB runs strong 12B models with vision. 16 to 24 GB runs the best models you can use at home. Under 6 GB, stick to small 4B models.
Does a GTX 1080 work for AI?
Yes. The GTX 1080 (8 GB) runs 9B models like qwen3.5:9b fully on the card, and the GTX 1080 Ti (11 GB) runs 12B models. Both need NVIDIA driver 570 or newer to work with current Ollama versions.