A local chat model, a repeatable task and a baseline you can compare.
01 / Choose a useful first job
A good first use case is rewriting your own short notes into a clear email. You can check whether every date, name and action survives. Start with fictional notes: a model that writes confidently can still invent details. Local inference gives you control over where generation runs; it does not make the answer automatically correct.
This guide uses Qwen3 4B as a modest, established starting point, not a claim that it is the newest or best model. The Ollama listing identifies a Q4_K_M build. Leave memory for the operating system and context instead of choosing a download that fills every available gigabyte.
02 / Install and open your first chat
Install Ollama from its official download page and start the application or service. Open Terminal on macOS/Linux or PowerShell on Windows. These commands download the model and start an interactive chat; the first download needs internet access. Enter /bye when finished.
ollama --version
ollama pull qwen3:4b
ollama run qwen3:4b03 / Give it an acceptance test
Paste the example below into the chat. Check that it preserves Wednesday, 14:00 and the venue, and does not add a price, promise or recipient. Then try your own non-sensitive notes. A useful evaluation checks facts and format, not just whether the writing sounds polished.
Repeat with a shorter and a longer note. Record the delay before the first useful text and the time until the final answer. Run once to warm the model, then repeat the same task three times. Keep the model and prompt unchanged while measuring.
Turn these notes into an email of at most 80 words.
Keep all facts. Do not add missing details.
Notes: Workshop moved to Wednesday, 14:00. Venue: Room B.
Ask participants to reply if they cannot attend.
/no_think04 / Know what stays local
Choose a downloaded local model rather than a cloud model. If you want Ollama’s local-only mode, add disable_ollama_cloud: true to the existing ~/.ollama/server.json settings and restart Ollama. Preserve any other settings. Model downloads still need network access; a browser extension or connected tool can have its own data flow.
Once downloads are complete, disconnect the network and repeat your harmless test. If it fails, identify which component needs connectivity. Check ollama ps while the model runs to inspect processor placement. Unexpected CPU use, memory pressure or a large context can explain slow responses. Start with a smaller model before buying hardware.
05 / Take the next step
Keep a small folder of prompts and expected facts. This becomes your personal regression set when you change models. TokFire Bench measures supported GGUF/llama.cpp or MLX/oMLX workflows separately; an Ollama chat is not automatically a TokFire benchmark. Use the same base model where possible and record the different runtime and quantization instead of merging the scores.