AI Tool Labs

Builds about Benchmarks

  1. Does Repomix compress mode hurt Claude's answers? 240 runs

    Repomix compress mode cut Claude's input 12-61% on three repos. Claude never guessed wrong, but compress drops Python functions and repeats TypeScript docs.

    Shipped

  2. Does temperature break JSON output? Small LLMs, measured

    3 small local LLMs, 20 emails to JSON at temperature 0, 0.8 and 1.5. With llama.cpp's default samplers, JSON stayed valid. Without them, 1.5 broke it.

    Shipped

  3. How many tokens is my codebase? Claude counts 58% more than GPT

    I counted real Claude tokens for 33 files from public repos. On code, Claude 4.7 and later use 1.58 tokens per GPT token. A CLI estimates it for your repo.

    Shipped

  4. Small LLM JSON output: the grammar fixes syntax, not answers

    6 small local LLMs, 20 emails to JSON, with and without llama.cpp's json_schema grammar. Valid JSON went to 100%; right answers did not move.

    Shipped

  5. Best small LLM for a CPU-only laptop: 10 models measured

    10 small LLMs on a 2-core laptop CPU: speed, RAM, first-token time and a 20-question check. Gemma 4 E2B was the best trade-off.

    Shipped

  6. Ollama vs llama.cpp on CPU: same speed, different defaults

    I benchmarked Ollama and llama.cpp on a 2-core laptop CPU with the same GGUF files. With equal settings the speed matched. The defaults did not.

    Shipped