A working MLX chat and a fair plan for comparing memory and response times.
01 / Choose MLX for the right machine
MLX LM provides local language-model generation on Apple silicon. A useful use case is a Mac-based writing or coding assistant that stays available alongside your normal apps. Unified memory is shared by the system and GPU: the RAM printed on the box is not a dedicated model budget.
Start with a modest quantized model and leave room for your editor, browser and context. The community Qwen3-4B-4bit checkpoint is an MLX conversion of Qwen3 4B. It is a different artifact from a GGUF file; do not assume the formats, quantizers or measured speeds are interchangeable.
02 / Create an isolated Python environment
Use a native Apple silicon Python installation compatible with the current MLX LM requirements. In Terminal, uname -m should report arm64. A Rosetta/x86 Python environment can cause incompatible-wheel errors. The commands below create a new folder and virtual environment without changing your system Python.
mkdir -p tokfire-mlx-demo
cd tokfire-mlx-demo
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U mlx-lm
python -m pip show mlx-lm03 / Generate a controlled answer
The first command below downloads the checkpoint from Hugging Face when it is not already cached. Keep this first download out of a speed comparison. After it completes, repeat the same generation and then open an interactive chat. Use mlx_lm.generate --help to check the options of your installed version.
Ask for a three-item checklist and count the items. Test one short prompt, then a longer document you wrote yourself. In Activity Monitor, watch memory pressure and swap while generating. If a run becomes unstable, reduce the model or context; closing one browser tab is not evidence that a much larger model will work reliably.
mlx_lm.generate --model mlx-community/Qwen3-4B-4bit --max-tokens 256 --prompt "Write three checks before sharing a report. /no_think"
mlx_lm.chat --model mlx-community/Qwen3-4B-4bit04 / Understand the current memory tools
MLX LM documents prompt caching, rotating KV caches and adjustable prefill step sizes. Prompt caching reuses computation for a repeated prefix. It can help when you ask several questions about the same document, but a warmed repeated prompt is a different workload from a fresh document.
A smaller prefill step can reduce the peak memory used while reading a long prompt, with a speed trade-off. A bounded rotating KV cache limits retained context and can change answer quality. Change one setting at a time. Label cached and uncached measurements separately, and re-check questions that depend on the beginning of the document.
05 / Compare with your real workflow
Keep the Mac plugged in and use the same power mode for each run. Record macOS, model revision, quantization, MLX LM version and the prompt. Compare at least three warmed runs, then test a cold start separately. A model that is pleasant for short edits may struggle with a long document.
For TokFire’s Mac benchmark, choose the supported oMLX + MLX option and record its own server settings. A direct mlx_lm command and an oMLX serving run are different configurations. Start with one job, then test two and three. Do not turn a single-job result into a promise about several simultaneous agents.