What "off-grid AI" means

When you use ChatGPT or Gemini, your question travels to a data center and the answer comes back. No internet, no answer, and the company keeps your question.

An off-grid AI is different. The model is a file of a few gigabytes on your own drive. A free program, Ollama, loads it onto your graphics card and runs it. Nothing leaves the room. Pull the network cable and it works the same.

People want this for three reasons:

  • Resilience. It works during outages, storms, and anything worse.
  • Privacy. Medical, legal, money and family questions stay in your house.
  • No subscription. The models and the software are free.

What hardware you need

The part that matters most is the graphics card's memory (VRAM). The model needs to fit in it to run fast.

Your VRAM Example cards A good model to start with
None / under 6 GB Office PC, laptop qwen3.5:4b (3.3 GB), slow on the processor
6–8 GB GTX 1070, GTX 1080, RTX 2060, RTX 4060 qwen3.5:9b (6.6 GB)
10–12 GB GTX 1080 Ti, RTX 3060 12GB, RTX 4070 gemma4:12b (7.7 GB), also reads photos
16 GB RTX 4060 Ti 16GB, RTX 4080 gemma4:26b (16 GB)
24 GB RTX 3090, RTX 4090 qwen3.6:27b (18 GB)

To find your number on Windows: open Task Manager, click Performance, click GPU, and read "Dedicated GPU memory".

Ollama supports NVIDIA cards back to the GTX 900 series (GTX 10-series cards need driver 570 or newer) and many AMD Radeon cards. A Mac with Apple Silicon also works very well.

Plan on 20 to 40 GB of free disk space for a few models, plus more if you add offline Wikipedia.

What it can do with no internet

  • Explain practical tasks step by step: water treatment, food storage, first aid, repairs, gardening, radio, solar.
  • Answer from your own documents. Load your survival manuals into Open WebUI's knowledge feature and it answers from them, with a link to the passage.
  • Read photos (with vision models like Gemma 4): labels, parts, wiring.
  • Do the math you don't want to get wrong when tired: battery runtime, rations, water per person.

What it can't do: know the news, or be trusted blindly on doses and numbers. Local models can make things up. The fix is to have it answer from trusted documents and to keep printed copies of the critical ones.

How much power it uses

This is the question most guides skip. A gaming PC is a real load off-grid:

  • Idle: about 50 to 120 W for the whole PC.
  • While the AI writes an answer: roughly 220 to 300 W with an RTX 3060 or GTX 1080, and 420 to 550 W with an RTX 3090.

The AI only works hard for the seconds it is answering, so real use is mostly idle. A simple rule: hours of runtime = battery watt-hours ÷ average watts, minus about 20% for losses. A 2,000 Wh power station running a PC that averages 120 W gives roughly 13 hours.

The better routine off-grid: keep the PC off, collect questions, then switch it on for a 30 to 60 minute session and write the answers down. Measure your own PC with a plug-in watt meter; your numbers are the ones that count.

The setup, in short

  1. Install Ollama from ollama.com.
  2. In PowerShell, run ollama pull with the model for your card (see the table).
  3. Install Open WebUI (the Desktop app is easiest) for a ChatGPT-style screen.
  4. Set its document reader to a local model (embeddinggemma through Ollama) so the library works offline.
  5. Load your trusted PDFs into a knowledge collection.
  6. Unplug the internet, restart, and test everything before you need it.

Questions

Can AI work without internet?

Yes. Open models like Gemma 4 and Qwen 3.5 run fully on your own computer through free software like Ollama. You need the internet once to download the software and the model files. After that, it works with no connection at all.

What is the best off-grid AI computer?

The best value is a used desktop with an RTX 3060 12GB graphics card: enough memory for strong 12B models at around 170 W for the card. For the smartest models, an RTX 3090 with 24 GB, but it draws about twice the power. Apple Silicon Macs are the most power-efficient option.

Is an off-grid AI safe to use for medical questions?

Use it to find and explain information in trusted medical references faster, not as a doctor. Set it up to answer from documents like Where There Is No Doctor, check the source it shows, and never use a dose or number you can't find in a printed reference.