The one number that decides what you can run

Forget clock speeds and cores. For local AI, the number that matters is your graphics card's VRAM (its own memory). The model has to fit in it to run fast. If it doesn't fit, the extra spills into normal RAM and runs on the processor, which works but is much slower.

Find it on Windows: press Ctrl + Shift + Esc, click Performance, click GPU 0 (and GPU 1 if you have it), and read Dedicated GPU memory at the bottom. Ignore "shared GPU memory".

Which models fit which card

These are download sizes from the Ollama library (November 2026). Leave a little room on the card for the conversation itself.

Card (VRAM) Run this Size Notes
GTX 1050 Ti (4 GB) qwen3.5:4b 3.3 GB Fits, small model
GTX 1060 6GB qwen3.5:4b or qwen3.5:9b 3.3 / 6.6 GB The 9B slightly spills over; slower but works
GTX 1070 / 1080, RTX 2060 / 2070 (8 GB) qwen3.5:9b 6.6 GB Good all-rounder
GTX 1080 Ti (11 GB), RTX 2080 Ti (11 GB) gemma4:12b 7.7 GB Reads photos too
RTX 3060 12GB gemma4:12b 7.7 GB Our value pick
RTX 4060 Ti 16GB, RTX 4080 (16 GB) gemma4:26b 16 GB Mixture of experts: fast for its size
RTX 3090 / 4090 (24 GB) qwen3.6:27b 18 GB Among the best home models

"Mixture of experts" models like gemma4:26b and qwen3.6:35b-a3b only use a small part of themselves for each word. They run faster than their size suggests and cope better when part of them spills into RAM, so they're worth trying if you have lots of system RAM.

Is my old GTX card still supported?

Ollama, the free program most people use to run local models, supports NVIDIA cards with compute capability 5.0 and up. That covers the GTX 750 Ti, the whole GTX 900 and GTX 10 series, and every RTX card. One catch: GTX 900 and 10-series cards need NVIDIA driver 570 or newer. If your old card isn't being used, update the driver first.

AMD Radeon cards work too. Newer cards (RX 6800 and up, RX 7000 and 9000 series) have full support, and Ollama can use many others through Vulkan.

How to check it's really using the GPU

After you run a model, open PowerShell and type:

ollama ps

The PROCESSOR column should say 100% GPU. A split like 30%/70% CPU/GPU means the model is a bit too big for your card; try the next size down. 100% CPU means the GPU isn't being used at all; update your driver and restart.

Quick start in three commands

  1. Install Ollama from ollama.com (Windows, Mac or Linux).
  2. Open PowerShell and download a model for your card, for example:
ollama pull gemma4:12b
  1. Chat with it:
ollama run gemma4:12b

For a proper chat screen with history, file upload and your own document library, add Open WebUI.

Worth buying a used card for this?

If your card has 4 GB or less, a used RTX 3060 12GB is the best value upgrade for local AI: 12 GB of memory at about 170 W. A GTX 1080 Ti (11 GB) is often cheaper but slower and hungrier at about 250 W. A RTX 3090 (24 GB) runs the best home models, but needs a 750 W power supply and draws about 350 W.

Questions

Can I run AI locally without a GPU?

Yes, small models like qwen3.5:4b run on the processor alone. Expect slow answers, often a word or two per second, compared with a full paragraph in a few seconds on a graphics card. Fine for occasional use, frustrating every day.

How much VRAM do I need for a local LLM?

8 GB runs a good 9B model. 12 GB runs strong 12B models with vision. 16 to 24 GB runs the best models you can use at home. Under 6 GB, stick to small 4B models.

Does a GTX 1080 work for AI?

Yes. The GTX 1080 (8 GB) runs 9B models like qwen3.5:9b fully on the card, and the GTX 1080 Ti (11 GB) runs 12B models. Both need NVIDIA driver 570 or newer to work with current Ollama versions.