How hardware × model grades work
TokFire fit v1 uses the same interaction guideline across apps 0.7 and 0.8: every task completes, every request decodes at least 30 tok/s, and every request produces visible output within 3 seconds. Missing runtime metrics remain unknown.
A / B additionally require the shipped agent tool workflow to complete without tool errors. C applies to the tested chat/text workload. D means the tested level missed the guideline. U means there is insufficient evidence to assess this hardware/model combination.
Grades describe this workload, context, runtime and tested job count. They are not a universal hardware score, a model quality score or a memory-fit prediction. A chat result cannot prove that agent work is impossible or that only one job will work.
The optional 100–200 tok/s range is a user target, not a promised speed for any cloud subscription. Total throughput across jobs is never substituted for each job’s speed.
Legacy fixed-token workload
Select one to three models. Each model completes a separate round and its server closes before the next starts; models never run concurrently. The runner tokenizes a fixed synthetic text with each model’s tokenizer, then uses exactly 512 or 2,048 input token IDs. Each workload generates 128 tokens with temperature 0, seed 42, EOS ignored and prompt cache disabled. One warm-up is discarded; three measured repetitions are retained. Context is 4,096, KV cache f16, flash attention off, batch and microbatch 512, one slot, GPU layer request 999, CPU threads equal to detected logical cores.
Concurrent jobs
TokFire Bench also provides a separate concurrent profile: one GGUF or MLX model, 1–3 free simultaneous jobs, or up to 20 with Pro. It measures three synchronized rounds and reports each job’s latency and speed separately from aggregate throughput. These results are not mixed with the sequential standard profile.
Windows preview
Windows 10/11 x64 uses llama.cpp and GGUF with 512 input tokens, 128 output tokens and three synchronized measured rounds after warm-up. Each job shares one model server, with 4,096 context tokens per slot. Reports use local-ai-windows-jobs-v1 and include CPU, computer model, GPU names, RAM and Windows version. They are stored separately from the Mac standard comparison table. GPU names do not prove that GPU offloading succeeded. The Windows executable is unsigned and has been cross-compiled and unit-tested on macOS; Windows GUI, login, cancellation and CPU/GPU inference tests remain pending.
Quick trial
The optional MiniCPM trial uses one model, 512 input and 32 output tokens, one warm-up and one measurement. Its local-ai-trial-v1 report stays local and cannot enter the standard comparison database.
What is measured
TTFT is client wall-clock time from starting the loopback HTTP request to receiving the first generated token ID. It includes local HTTP overhead. Prefill and decode rates come from the runtime’s final timing report. Load time is process launch to a healthy server, without clearing the operating system’s file cache. Peak process RSS is sampled every 250 ms across that model session; it is not total unified or Metal memory.
Comparison rules
Exact model bytes and runtime binary are SHA-256 hashed. A comparison group also includes every runtime setting, specification version, and input/output length. Hardware/OS combinations remain separate rows. Each report contributes a median of three runs; community rows average these medians. No global score, quality score, predicted speed or verified badge is generated.
Known limitations
These are community submissions, not attested measurements. Format checks cannot prove the hardware claim or prevent a modified client. Power, temperature, swap, actual layer-offload verification, total GPU memory and sustained thermal testing are not measured yet. Model-load timing is affected by OS caches. Ignoring EOS is a throughput workload, not a quality assessment. Do not infer answer quality or broad application compatibility from speed alone.
Privacy and publication
The benchmark runner makes loopback requests only and saves reports locally. The optional MiniCPM launcher downloads the official model from Hugging Face and verifies its checksum. Browser import does not upload. Saving requires collection consent but no sign-in; publication is a separate opt-in. A private guest credential or optional account identifies the report owner. Published comparisons omit identity but include hardware and hashes. Published comparisons are visible to site visitors. Private reports require the guest credential or account that uploaded them. Clearing guest website data removes access.
Native testing status
The source includes a SwiftUI macOS interface and a real llama-server runner. Python protocol tests run in this development environment. Swift compilation, real Metal inference, GUI lifecycle tests and ad-hoc signing have passed on an M2 Max. The developer DMG is not Developer ID signed or Apple notarized.
Runtime API reference: llama.cpp server