Ollama vs llama.cpp on CPU: same speed, different defaults

Is llama.cpp faster than Ollama on a CPU? On my laptop, with the same model file and the same settings, no: Ollama generated 14.6 tokens/s and llama.cpp 14.7 on Qwen 2.5 1.5B, and 6.8 vs 6.7 on Llama 3.2 3B. The gap people report comes from the defaults, and on this machine the defaults decided whether the 3B model fit in memory at all.

The question comes up constantly, and the numbers floating around disagree. One widely shared post found llama.cpp 26.8% faster on a 1.5B model without naming the hardware; a GitHub issue reported Ollama about 38% slower, comparing it against llama.cpp run with 16 threads, 4 slots and GPU offload. Most comparisons don’t say which context size, thread count or build they used. I wanted a comparison where every setting is written down, on the kind of hardware many people actually have: an Intel i5-7200U laptop (2 cores, 4 threads, 15 W), 19 GB of RAM, no GPU.

This is also the benchmark that crashed my first scheduled run. Today I ran it again, with a memory cap this time.

Does Ollama use llama.cpp?

Yes, and more directly than I expected. Ollama 0.40.3 ships a copy of llama.cpp’s llama-server and starts it as a subprocess for each model. Its own log prints the full command (I shortened the paths):

starting llama-server cmd="lib/ollama/llama-server --model blobs/sha256-1adf... --port 37261
  --host 127.0.0.1 --no-webui --offline -c 4096 -np 1 --log-verbosity 4 --no-log-prefix
  --no-log-timestamps --load-mode none --flash-attn auto -b 512 -ub 512 --context-shift --keep 4"

The source matches: llm/llama_server.go builds those arguments, and the repo pins llama.cpp build b11351 in a LLAMA_CPP_VERSION file. So “Ollama vs llama.cpp” on CPU is really “llama-server with Ollama’s flags, behind a Go proxy, vs llama-server with its own defaults”. That tells you what to measure.

How I tested

Same two GGUF files for both tools (Qwen 2.5 1.5B Instruct and Llama 3.2 3B Instruct, both Q4_K_M), loaded into Ollama with a one-line Modelfile. llama.cpp was built from source with native CPU flags, at build b11554, about 200 builds newer than the b11351 that Ollama bundles.

A Python script with no dependencies sends the same 346-token prompt to both servers, asks for exactly 128 tokens at temperature 0, and reads each server’s own timings:

# Ollama: raw mode, so no chat template is added
post(f"{base}/api/generate", {"model": model, "prompt": prompt, "raw": True,
     "stream": False, "keep_alive": -1, "options": {"num_predict": 128, "temperature": 0}})

# llama-server
post(f"{base}/completion", {"prompt": prompt, "n_predict": 128, "temperature": 0,
     "cache_prompt": False})

Both servers stay loaded the whole time, and the order alternates every round so background noise hits both. Each run starts with a different first line (“Run 3.”) so neither server can reuse a cached prompt. One warm-up, then 5 measured runs per config, and I report the median.

Every llama-server ran inside a systemd scope with a 6 GB limit, so a config that asks for too much memory dies alone instead of taking the laptop down:

systemd-run --user --scope -p MemoryMax=6G -p MemorySwapMax=0 \
  llama-server -m "$gguf" --port 11600 $flags

Is llama.cpp faster than Ollama on CPU?

Not when the settings match. Generation speed, tokens per second, median of 5:

Config	Qwen 1.5B: Ollama	llama.cpp	Llama 3B: Ollama	llama.cpp
Each tool’s defaults	14.65	14.68	6.80	killed at 6 GB
Same context and slots (-c 4096 -np 1)	14.68	13.75	6.80	6.73
Ollama’s exact flags, 2 threads	14.62	14.69	6.81	7.06
Ollama’s exact flags, 4 threads	16.66	16.11	7.71	7.52

Ollama’s defaults and the “same context” row are the same thing for Ollama, since it always launched with -c 4096 -np 1 here. Prompt processing was within 5% in every config but one, at about 46 tokens/s for Qwen and 22 for Llama.

The differences in the table go both ways and stay under 4%, except the 13.75. That one is llama-server with less context than in the “defaults” row a few minutes earlier, which should not slow anything down, yet it ran 6% slower. The same Ollama config measured in two separate sessions also moved by up to 3.6% (7.99 vs 7.71 t/s on Llama with 4 threads). So I treat differences under about 6% as noise.

What Ollama adds is a Go process in front. I measured it as the client’s wall time minus the time the engine reported for prompt and generation: 12 to 15 ms per request for llama-server and 16 to 20 ms for Ollama on Qwen, so 3 to 10 ms extra. Against a 16-second request, that’s nothing. The ollama serve process itself peaked at 68 MB.

The cold start is a real difference. With the model file already in the page cache, from nothing loaded to the first token:

Model	Ollama (3 tries)	llama-server (3 tries)
Qwen 2.5 1.5B	2.68 to 2.74 s	2.16 to 2.26 s
Llama 3.2 3B	5.34 to 6.12 s	4.38 to 4.73 s

Part of that is Ollama’s --load-mode none, which reads the weights into memory instead of mapping the file, and part is its scheduler deciding where the model goes. It only matters if your model gets unloaded between requests. By default Ollama unloads after 5 minutes idle.

Why does llama-server use so much memory?

Because by default it prepares for four users with long conversations. That is the real difference between the two tools on a laptop. Peak memory of the inference process, in MiB:

Setup	Qwen 2.5 1.5B	Llama 3.2 3B
Ollama	1,231	2,768
llama-server, defaults	2,610	killed at the 6 GB cap
llama-server -c 4096 -np 1	1,825	4,056
llama-server with Ollama’s flags	1,232	2,770

With no flags, llama-server takes the context length from the model file (32,768 tokens for Qwen 2.5) and opens 4 parallel slots that share it as one 131,072-token cache. For the 1.5B model that cost 2.6 GB, twice Ollama. For the 3B model, whose cache takes four times more memory per token, the kernel killed it during load:

run-<id>.scope: A process of this unit has been killed by the OOM killer.
run-<id>.scope: Failed with result 'oom-kill'.

Yesterday, without the cap, the same command reached 10.6 GB before the machine ran out. Ollama picked 4,096 tokens on its own, and its log says why: msg="vram-based default context" total_vram="0 B" default_num_ctx=4096. With no GPU, it chooses a small context.

The middle row has a second effect. Even with a matched context, llama-server sat 600 MB to 1.3 GB higher. That’s mmap: llama-server maps the model file, and the mapped pages count toward its RSS. Splitting the number shows it (Qwen: 764 MiB anonymous memory plus 951 MiB file-backed, against Ollama’s 1,108 MiB anonymous). The kernel can drop file-backed pages under pressure, so this costs less than it looks. Pass --load-mode none, as Ollama does, and the two processes land within 2 MiB of each other.

How many threads should I use on a laptop CPU?

Both tools picked 2 threads on this 2-core, 4-thread CPU, and both log it (n_threads = 2 (n_threads_batch = 2) / 4). A sweep with llama-bench shows where that is right and where it isn’t:

Threads	Qwen prompt t/s	Qwen generation t/s	Llama prompt t/s	Llama generation t/s
1	25.6	9.5	12.2	4.9
2	48.5	14.1	22.0	7.7
3	45.3	14.9	21.3	7.9
4	45.6	14.9	21.7	8.3

Prompt processing is compute-bound and peaks at the physical core count. Generation is mostly memory-bound and still picked up 6 to 7% from the extra hyper-threads. In the server benchmark with Ollama’s flags, going from 2 to 4 threads raised generation speed by 6 to 14% (Qwen in Ollama: 14.62 to 16.66 t/s) and made prompts slightly slower. Set it with num_thread in Ollama (request options or a Modelfile PARAMETER) or -t 4 in llama-server. It is the only setting I changed that made either tool faster.

One thing I can’t fully explain: with 4 threads, llama-server on Qwen was 10% faster with Ollama’s flag set (16.11 t/s) than with its own (-c 4096 -np 1 -t 4, 14.69 t/s). On Llama 3B the two flag sets were equal. Ollama’s set differs in mmap and batch size (-b 512 vs the default 2048), and I did not test them one at a time.

So which one should you use on a CPU?

Pick on features and memory behavior, not speed. They run the same engine at the same speed.

• Ollama if you want safe defaults on a small machine: a 4,096-token context when there’s no GPU, one slot, models that load on request and unload when idle, and a model library. You pay 0.4 to 1.7 s more per cold start.
• llama-server if you want control or the newest llama.cpp the day it lands. Always pass -c and -np on a laptop. The defaults are sized for a server with memory to spare, and on mine they got a 1.9 GB model killed.
• Either way, try -t / num_thread at your thread count, not just your core count, and measure generation and prompt speed separately. They peak at different values.

What broke along the way

• The llama.cpp build I cached yesterday failed to start today: cannot open shared object file: libllama-server-impl.so. CMake had baked the original build folder into the binary’s library search path, and I had moved the folder. Setting LD_LIBRARY_PATH only for my llama-server fixed it. Setting it for the whole script would also have leaked into Ollama, which might then have loaded my libraries instead of its own.
• My first memory table was wrong for Ollama on the 3B model. The script unloaded only the current model, so the Qwen runner was still alive, and pgrep | head -1 read its memory instead. I caught it because the two numbers were identical, threw them out, and took the 3B figures from a second script that unloads everything first.
• The cold-start script crashed on its first try because printf "%.2f" rejected 3.17 in a locale that writes decimals with a comma. LC_ALL=C fixed it.
• Caveats: one machine, two models, one prompt length, and the two llama.cpp builds are about 200 builds apart (b11351 in Ollama, b11554 standalone). GPU offload is a different comparison.

Everything is in the project folder: the benchmark script, the four runner scripts, 108 raw runs, the memory and cold-start data, and the server logs. On a Linux laptop, ./setup.sh and ./run_all.sh reproduce the main table in about 25 minutes. Next in this series: which small models are actually usable on a CPU like this one, with speed, memory and a 20-question check for 10 of them. Then: how reliably small models return structured JSON, with and without a json_schema grammar. And last: what happens to that JSON above temperature 0, where llama-server’s default samplers mattered more than the temperature itself.