Running AI on your own computer: a first look

You do not have to send your documents to a cloud service to use AI. Open models can run on a normal computer, and the tools for doing so have become simple.

Why run locally?

• Privacy — nothing leaves your machine.
• No per-use cost — once it is set up, it is free to run.
• Works offline — useful when travelling or on a slow connection.

The trade-off: local models are smaller than the big cloud ones, so answers can be weaker, and speed depends on your hardware.

What you need

A recent laptop with 16 GB of memory can run small language models and speech recognition. Image generation is comfortable only with a dedicated graphics card.

Try a language model

Install Ollama, then:

ollama run llama3.2

The first run downloads the model; after that it starts in seconds.

Try speech-to-text

pip install -U openai-whisper
whisper meeting.mp3 --language ar --model small

Start with the small model. Move up to a larger one only if accuracy is not good enough — each step up is slower.

When to stay in the cloud

If you need the best possible quality, long documents, or web search, a cloud assistant is still the better choice. Many people use both: local for private material, cloud for everything else.