AI Tool Labs

Builds about Local models

Every number in these posts comes from the same ordinary laptop: a 2-core CPU, 19 GB of RAM and no GPU. They start with which runtime to use, then which small model fits, then what those models can be trusted to do.

Reading order

  1. Ollama vs llama.cpp on CPU: same speed, different defaults: the two runtimes at equal settings, and the defaults that make them look different.
  2. Best small LLM for a CPU-only laptop: 10 models measured: ten small models compared on speed, memory and a 20-question sanity check.
  3. Small LLM JSON output: the grammar fixes syntax, not answers: what a JSON grammar fixes in small-model output, and what it does not.
  4. Does temperature break JSON output? Small LLMs, measured: the same JSON task at temperature 0.8 and 1.5, and the samplers that keep it valid.

  1. Does temperature break JSON output? Small LLMs, measured

    3 small local LLMs, 20 emails to JSON at temperature 0, 0.8 and 1.5. With llama.cpp's default samplers, JSON stayed valid. Without them, 1.5 broke it.

    Shipped

  2. Small LLM JSON output: the grammar fixes syntax, not answers

    6 small local LLMs, 20 emails to JSON, with and without llama.cpp's json_schema grammar. Valid JSON went to 100%; right answers did not move.

    Shipped

  3. Best small LLM for a CPU-only laptop: 10 models measured

    10 small LLMs on a 2-core laptop CPU: speed, RAM, first-token time and a 20-question check. Gemma 4 E2B was the best trade-off.

    Shipped

  4. Ollama vs llama.cpp on CPU: same speed, different defaults

    I benchmarked Ollama and llama.cpp on a 2-core laptop CPU with the same GGUF files. With equal settings the speed matched. The defaults did not.

    Shipped