---
title: "What a Local LLM Can Do in 2026: Gemma 4 on llama.cpp 1.0"
description: "I scored three sizes of Gemma 4 on llama.cpp 1.0 on a MacBook: tool calls, JSON, code, SQL, screenshots. 138 checks, with the failures and the speeds."
canonical: "https://www.trevorlasn.com/blog/what-a-local-llm-can-do-llama-cpp-1-0"
author: "Trevor I. Lasn"
published: "2026-11-15"
last-updated: "2026-11-15"
topics: ["AI", "Open source", "Developer tools"]
---

# What a Local LLM Can Do in 2026: Gemma 4 on llama.cpp 1.0

I scored three sizes of Gemma 4 on llama.cpp 1.0 on a MacBook: tool calls, JSON, code, SQL, screenshots. 138 checks, with the failures and the speeds.

By Trevor I. Lasn · Published 2026-11-15 · https://www.trevorlasn.com/blog/what-a-local-llm-can-do-llama-cpp-1-0

A 14.62 GB file on my MacBook wrote ten JavaScript functions in 51 seconds. I ran each one against assertions I had written before the model saw the task, and all ten passed.

Then I let the same model think before answering and gave it the same ten tasks. It passed five, and the run took 14 minutes.

[llama.cpp](https://github.com/ggml-org/llama.cpp) is the C++ library that runs LLMs on your own hardware. It tagged [v1.0.0](https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0) on 2026-11-13, about three and a half years in.

The changelog has a link. I wanted to know what a model that fits on a laptop can do now, where it breaks, and what each answer costs in seconds and gigabytes.

So I wrote 138 checks with the answers fixed before any run, and pointed them at three sizes of Google's Gemma 4, with thinking off and on. All three models ran 87 of them, and the smallest ran all 138.

Every number below is a count from those runs, and the failures are in here too.

I've had a few MLX models on this machine, but I'd never set up llama.cpp. It's one binary and one command:

```bash
llama-server -hf ggml-org/gemma-4-12B-it-GGUF:Q4_0 -c 32768 -np 1
```

`-hf` pulls the model from Hugging Face on first use, then loads it from the cache. `-c 32768` sets the context window in tokens, and `-np 1` gives the whole window to one request slot.

That last flag matters more than it looks. With `-c 16384 -np 4` the server log prints `n_slots = 4, n_ctx_slot = 4096`: four requests at once, each with a quarter of the window.

Once it prints `listening on http://127.0.0.1:9931` you have an OpenAI-compatible API. Any client that speaks `/v1/chat/completions` works against it, with no key.

This is the machine and the build, because every speed in this post depends on them:

| | |
| --- | --- |
| Laptop | MacBook Pro 14-inch (Mac16,6) |
| Chip | Apple M4 Max: 14 CPU cores (10 performance, 4 efficiency), 32 GPU cores |
| Memory | 36 GB, of which Metal offers llama.cpp 28,753 MiB |
| Disk | 1 TB SSD |
| OS | macOS 26.3.1 |
| llama.cpp | 1.0.0 release binary, build 12359, commit `b1b770d90` |

The laptop wasn't idle. Other jobs kept the one-minute load average between 35 and 90 on those 14 cores for every run, so these speeds come from a busy machine and a quiet one will beat them.

These are the three models. All of them are Apache 2.0, all take images as well as text, and I used the 4-bit `Q4_0` file of each:

| Model | Parameters | Weights | Image projector | Context limit |
| --- | --- | --- | --- | --- |
| Gemma 4 E4B | 4.5B effective, 8B with embeddings | 4.59 GB | 0.56 GB | 128K tokens |
| Gemma 4 12B | 11.95B | 7.22 GB | 0.16 GB | 256K tokens |
| Gemma 4 26B A4B | 25.2B, 3.8B active per token | 14.62 GB | 0.81 GB | 256K tokens |

The 26B A4B is a mixture-of-experts model. It keeps 25.2 billion parameters on disk and runs 3.8 billion of them for each token, which is why the biggest file here turns out to be the fastest.

First download and load took 245 seconds for E4B and 709 seconds for the 26B on my connection. After that, a restart loads any of them from disk in 3 to 10 seconds.

## 138 checks, with the answers fixed first

Every check has one right answer that I wrote down before the run. A script sends the prompt, takes the reply, and scores it with code: it runs the JavaScript, compiles the C, executes the SQL, or compares fields.

| Battery | Checks | How it's scored |
| --- | --- | --- |
| Tool calls | 14 | The exact function and arguments, or no call when no tool fits |
| Structured extraction | 40 fields | Ten sentences into a JSON schema, compared field by field |
| JavaScript | 10 | The code runs under `node --permission` against my assertions |
| C port | 1 | `cc -fsyntax-only -Wall` against the 1.0.0 header, zero warnings |
| Screenshots | 30 facts | Five screenshots, each fact compared with the page it came from |
| Long context | 5 | Questions about an 89.6 KB header pasted into the prompt |
| Agent loop | 6 | The model drives `grep` and `read_file` itself, up to 10 turns |
| SQL | 8 | The query runs on SQLite and its rows must match a reference query |
| Regex | 6 | The pattern runs against strings it must match and must reject |
| Arithmetic and logic | 12 | One right answer each |
| Bug finding | 6 | Name the line that has to change |

All three models ran the first six batteries, 87 checks, in both modes:

The last five batteries, 51 checks, need a second pass per model, and only E4B finished it. By then other jobs had left this laptop 6 to 11 GB of free memory, and loading a 10 GB model into that was getting their compilers killed.

So for the agent loop, SQL, regex, logic and bug finding, the 12B and 26B numbers are missing below. Missing means not run, not zero.

Every request used temperature 0, so a rerun repeats. I ran E4B's ten JavaScript tasks twice with thinking on and got the same 10,487 tokens both times.

Tool calls first, because they decide whether a model can do anything beyond chat. I described three functions to the model: `npm_package(name)`, `github_repo(owner, repo)` and `convert_currency(amount, from, to)`.

Then I asked 14 questions. Nine need a call, one of those needs two, and five need none. This is the whole reply to "What is the latest version of zod on npm?":

```json
{
  "role": "assistant",
  "content": "",
  "tool_calls": [{
    "type": "function",
    "function": { "name": "npm_package", "arguments": "{\"name\":\"zod\"}" },
    "id": "l4ftTaXdBZYOWIRGwFad84FUPcHOjL6E"
  }]
}
```

The model can't know the version, and it didn't guess one. It stopped with `finish_reason: "tool_calls"` and waited.

I pulled the real numbers from the npm registry and sent them back as the tool result. The answer: "The latest version of **zod** on npm is **4.6.7**, and it received **460,607,664** downloads last week." Both figures match the registry.

E4B and the 26B scored 14 of 14, in both modes. The 12B scored 13 of 14 in both modes, and it missed the same question each time.

I asked for "the latest release of the Astro repo, withastro/astro". The 12B sent this:

```json
{ "name": "github_repo", "arguments": "{\"owner\":\"astro\",\"repo\":\"astro\"}" }
```

The owner was in the question, and it wrote a different one. A real API would have answered 404, or worse, returned somebody else's repository.

The five no-tool questions went the right way everywhere. "What is 17 times 23?" got an answer with no call, and for "What's the weather in Tallinn today?" no function fits, so every model declined:

> I'm sorry, but I don't have access to real-time weather data.

None of them tried to bend `github_repo` into a weather service. And for "Compare the star counts of facebook/react and vuejs/core", all three sent two calls in one reply.

Next, structured extraction. I wrote ten sentences that each describe a release, and asked for four fields with a JSON schema in `response_format`:

| Sentence | Expected |
| --- | --- |
| Vite went to 8.0.0 on Aug 21, 2026. The migration guide lists a dozen breaking changes. | `vite`, `8.0.0`, `2026-08-21`, `12` |
| llama.cpp tagged 1.0.0 on 13.11.2026 and it removes a deprecated API, which is one breaking change. | `llama.cpp`, `1.0.0`, `2026-11-13`, `1` |
| Two days before Christmas Day 2025, pnpm 11.0.0 arrived with four breaking changes. | `pnpm`, `11.0.0`, `2025-12-23`, `4` |

All six runs turned "a dozen" into `12`, read `13.11.2026` as the 13th of November, and worked out that two days before Christmas is `2025-12-23`.

All 60 replies parsed as JSON. That part isn't the model's doing: llama-server turns the schema into a grammar and only samples tokens that fit it, so a reply can't come out malformed.

The values are still the model's. E4B and the 26B scored 40 of 40 with thinking off, and the other four runs scored 39, all on this sentence:

```text
Released 2026/09/30: typescript@6.1.0-beta (3 breaking changes, see notes).
```

<br />

```json
{ "package": "typescript@6.1.0-beta", "version": "6.1.0-beta", "date": "2026-09-30", "breaking_changes": 3 }
```

The version landed in the right field and stayed glued to the package name too. A grammar guarantees the shape of the JSON, and nothing guarantees what goes in it.

Then code. Ten tasks, each with assertions the model never sees: a slugifier, a semver comparison, interval merging, Roman numerals, a CSV line parser, an LRU cache, a topological sort, a binary search with a hang to fix, a duration formatter and a word wrapper.

With thinking off:

| Task | E4B | 12B | 26B A4B |
| --- | --- | --- | --- |
| `slugify` | pass | pass | pass |
| `compareSemver` | pass | pass | pass |
| `mergeIntervals` | **fail** | pass | pass |
| `toRoman` | pass | pass | pass |
| `parseCsvLine` | pass | pass | pass |
| `LRUCache` | pass | pass | pass |
| `topoSort` | pass | pass | pass |
| `binarySearch` (fix a hang) | pass | pass | pass |
| `formatDuration` | pass | **fail** | pass |
| `wordWrap` | pass | pass | pass |

The 12B's `formatDuration` declares a constant and then reassigns it:

```js
const seconds = Math.floor(ms / 1000);
const d = Math.floor(seconds / (24 * 3600));
seconds %= (24 * 3600);
```

<br />

```text
TypeError: Assignment to constant variable.
```

I changed that one `const` to `let` and all seven assertions passed. A linter would have caught it in a second, but the model had no linter and didn't run its own code.

E4B failed on a name. I asked for `mergeIntervals`, and it wrote `mergeInterval`, so my test hit `ReferenceError: mergeIntervals is not defined`. With the name fixed, that code passes too.

One pass deserves an asterisk. The 26B's `compareSemver` never strips build metadata, so `"0+build5"` parses to `NaN`, and every comparison against `NaN` is false:

```js
compareSemver("1.0.0+build5", "1.0.0"); // 0, which my assertion wanted
compareSemver("1.0.1+build5", "1.0.0"); // 0, which is wrong
```

It passed my nine assertions by accident. I count it as a pass because the answer key was fixed before the run, and I'm showing it because ten green ticks would have hidden it.

Now the same ten tasks with thinking on. Gemma 4 can write a private scratchpad before the reply, and llama-server turns that on by default (`--reasoning auto`):

Thinking made every model worse at this. The 26B dropped from ten to five, and all five failures look the same: `finish_reason: "length"`, an empty answer, and between 8,611 and 11,944 characters of reasoning.

The reply limit was 4,096 tokens, and thinking tokens count against it. On half the tasks the model spent the whole budget planning and never wrote the function.

llama-server has a flag for this. `--reasoning-budget N` caps the thinking at N tokens, and `--reasoning off` removes it:

```bash
llama-server -hf ggml-org/gemma-4-E4B-it-GGUF:Q4_0 --reasoning-budget 1024
```

On E4B a 1,024-token budget finished the ten tasks in 207.7 seconds instead of 254.7, with the same 8 passes. Raising the reply limit to 16,384 changed nothing for E4B either, because E4B never hit the limit: its two failures are the wrong name and a semver comparison that returns `1` where it should return `-1`.

The budget run for the 26B would show whether its five lost tasks come back. It's one of the runs I couldn't fit in memory.

The C port is the job I cared about most. I diffed `llama.h` between the `v0.9.0` and `v1.0.0` tags: 1.0.0 deletes 36 deprecated functions.

I wrote a 15-line C file that calls six of them. Against the 0.9.0 header it compiles with six deprecation warnings, and against the 1.0.0 header it fails with 8 errors.

Each model got the file and the six warnings, with one instruction: make it compile against 1.0.0. These are three of the six lines the 12B changed:

```diff
-    llama_token bos = llama_token_bos(vocab);
-    int32_t n_vocab = llama_n_vocab(vocab);
-    size_t state_size = llama_get_state_size(ctx);
+    llama_token bos = llama_vocab_bos(vocab);
+    int32_t n_vocab = llama_vocab_n_tokens(vocab);
+    size_t state_size = llama_state_get_size(ctx);
```

I didn't read the answers first. The script handed each one to the compiler:

```bash
cc -std=c11 -fsyntax-only -Wall -I inc-v1.0.0 src/model.c
```

Exit code 0 and no warnings, in all six runs. It's a mechanical job, since each warning names its replacement, and it's the kind of chore I'd rather hand off.

The next four batteries ran on E4B only, and they're where thinking pays.

For SQL I gave the model a three-table SQLite schema (packages, releases, downloads) and eight questions. The script runs each query on a real database and compares the rows with a reference query.

Without thinking, E4B got 5 of 8. Asked for the percentage of releases with zero breaking changes, rounded to one decimal, it returned this:

```sql
SELECT CAST(SUM(CASE WHEN breaking_changes = 0 THEN 1 ELSE 0 END) AS REAL) * 100 / COUNT(*) FROM releases;
```

<br />

```text
45.4545454545455
```

The number is right and the rounding I asked for is missing. A second query wrote `breaking_changes = 1` where the question said "at least one", and a third returned every release of each package where I'd asked for the latest. With thinking on, all eight passed.

Regex went worst. Six patterns, each run against strings it must match and strings it must reject, and E4B passed 2 without thinking and 3 with it.

Three of the six no-thinking answers weren't regular expressions at all: Node threw `SyntaxError: Invalid regular expression` on them. Its CSS hex color pattern parsed, and then missed `#fff`:

```js
/^#{3}([0-9a-fA-F]{3}|0-9a-fA-F]{6})$|^#[0-9a-fA-F]{6}$/
```

`#{3}` means three hash characters. I wouldn't ship a regex from a 4.59 GB model without running it first.

The twelve arithmetic and logic questions each have one answer. E4B got 7 without thinking and 9 with it, and one miss survived both modes:

> A build takes 75 seconds. How many full builds fit in 2 hours?

It answered 16, and the answer is 96. With thinking on it also counted 8 letter r's in "strawberry raspberry", which has 6.

Bug finding shows a numbered snippet and a symptom, and asks for the line that has to change. E4B named 5 of 6 without thinking and 6 of 6 with it.

Two of my answer keys were too strict, and I'm saying so because it changes the score. For a `strcpy` into an 8-byte buffer I'd keyed the `strcpy` line, and E4B named the buffer declaration, which fixes it too. I accept both, and under my original key the scores would be 4 and 5.

Screenshots next. Every Gemma 4 file comes with a projector that turns an image into tokens, so the same endpoint takes a PNG.

The first two screenshots hold 17 facts: a bar chart with a title and four bars, and a table with three rows of four cells.

The 12B read every title, label and number in both modes. E4B without thinking dropped the tallest bar from the chart, the one labelled "Yes" with 207, and cut the decimals off the table: 53.8 became 53 and 71.1 became 71.

The 26B without thinking read every cell and then swapped two of them, giving 46 as the slowest speed and 43.6 as the fastest. Thinking fixed both models' tables.

Three harder screenshots went to E4B only: a code block, a diff and the top of a page, 13 facts. It read 5 without thinking and 8 with it.

The misses are small-text misses. `256GB` came back as `256B`, the date `2026-11-15` as `2021-11-15`, and in the diff `llama_n_vocab` shrank to `llama_vocab`.

The server log explains part of it. By default an image gets at most 512 tokens, however big it is:

```text
load_model: cap image_max_tokens (original=1120) to 512 (n_ubatch = 512)
load_model: increase n_ubatch (-ub) to increase vision token budget
```

I restarted E4B with `-ub 2048` and ran all five screenshots again. The scores didn't move: 12 of 17 and 5 of 13. On this model the image token budget wasn't the limit.

The last battery pastes a whole file into the prompt. `llama.h` from the 1.0.0 tag is 89.6 KB, which the tokenizer turns into 25,280 tokens.

I asked five questions that each have one function name as the answer, such as "Which function copies rows from host memory into the h_nextn device buffer?" The 12B found all five, and E4B and the 26B found four.

Both of those missed the same one. "Which function returns the number of NextN layers of a model?" has the answer `llama_model_n_layer_nextn`, and both replied `llama_model_target_layer_ids_n`.

The header explains why. The wrong function carries a comment and the right one doesn't:

```c
LLAMA_API int32_t llama_model_n_layer_nextn(const struct llama_model * model);

// returns the number of extracted layers from target model
LLAMA_API uint32_t llama_model_target_layer_ids_n(const struct llama_model * model);
```

They matched my question against the comment text. The 12B read the name.

Pasting the file is one way to answer questions about it. The other is to let the model look things up, so the agent loop gives it three tools over a folder with both headers and two C files: `list_files`, `grep`, and `read_file`, which returns 120 lines per call.

Six questions, up to ten turns each, and no hints about which tool to use. E4B answered 3 of 6 without thinking and 5 of 6 with it.

The failures come from how it used the tools. Asked how many lines of the 0.9.0 header contain `DEPRECATED`, it sent this:

```json
{ "name": "grep", "arguments": "{\"count_only\":true,\"pattern\":\"DEPRECATED\"}" }
```

No `path`, so the count covered all four files and came back 43. The header has 40.

Asked whether `llama_n_vocab` is still declared in the 1.0.0 header, it read the first 120 lines of a 1,737-line file and answered "Yes". The function was deleted, and one `grep` would have shown that.

With thinking on it sent the `grep` with a path both times and got both right. Its one remaining miss needed two hops: find the free call in `old.c`, then find that function's deprecation note. After seven turns it answered `llama_free` where the note says `llama_model_free`.

## What it costs, and where it stops

Thinking is the first cost. This is how many tokens each run generated for the same ten JavaScript tasks:

The 26B wrote 15 times more tokens with thinking on, and tokens are time on a laptop:

The 12B took 21 minutes and 48 seconds to pass six tasks it had passed nine of in 2 minutes and 16 seconds.

So thinking isn't a setting to switch off and forget. On E4B it cost time on code and bought correct answers nearly everywhere else:

The price: the eight SQL questions took 18.9 seconds without thinking and 119.4 with it. I'd leave thinking on for queries and lookups, and turn it off or cap it for code.

Speed is the second cost. Across every reply in a run, weighted by tokens, E4B generated 40 tokens per second, the 12B 18 to 21, and the 26B A4B 38 to 45.

The 26B A4B writes two and a half times faster than the 12B from a file twice the size. That's the 3.8 billion active parameters at work: per token it does less arithmetic than the 12B does.

Single replies swing with the load on the machine. The 26B's slowest reply ran at 26 tokens per second and its fastest at 82.

Every model reads faster than it writes, and how much faster depends on the model. This is the time each one took to process the 25,280-token header before its first answer:

The 12B needs two minutes and 18 seconds before it says a word about that file. The follow-up questions then came back in under 1.5 seconds each.

That's the prompt cache. llama-server compares each prompt with the previous one and only evaluates the part it hasn't seen, so my four follow-ups cost 25 to 33 new tokens each instead of 25,280.

The reading cost grows in a straight line with the prompt. I sent E4B five slices of the header, each as a fresh prompt:

That's about 1,000 tokens per second from 778 tokens to 22,614, and generation held between 37 and 49 tokens per second behind every one of them. A long prompt costs you the wait up front and nothing after.

One server can also answer several requests at once. I restarted the 12B with four slots and sent the same question 1, 2 and 4 times in parallel:

Four requests together got twice the throughput of one. Each single request slowed from 21 tokens per second to between 12 and 15, so batch work gains and a chat window loses.

Memory is the third cost, and it decides which of these you can run at all.

`footprint` counts the memory the process owns. The weights are also mapped from disk, 6.8 GB for the 12B and 13.7 GB for the 26B, and macOS can drop those pages and read them back.

The footprint alone didn't stop me. This laptop runs compilers and other agents all day, and out of 36 GB it had 6 to 11 GB to spare when I wanted the second pass.

E4B fit every time. The 12B and the 26B raised macOS memory pressure to the warning level within 90 seconds of loading, and another job's watchdog started killing compiles, so I stopped them.

If the laptop is also your work machine, budget for that. A 16 GB Mac runs E4B next to an editor and a browser, and I wouldn't plan on more.

The fourth wall is what the weights know. With no tools attached, I asked each model for the latest llama.cpp release and told it to say so if it didn't know. The 12B, in an earlier run:

> I do not know because **llama.cpp has not officially released a version 1.0.0.**

It said that while running on llama.cpp 1.0.0. In the scored runs all three declined to name a version, which is the right answer, and the fix is the tool call from the first battery.

I'd also planned to run gpt-oss-20b and [Qwen3.8-27B](https://huggingface.co/ggml-org/Qwen3.8-27B-GGUF) through the same checks. Those are 12.1 GB and 19.4 GB files, and for the same memory reason neither one ran.

The last wall is size, and for that I'd read the 1.0.0 pull requests. llama.cpp makes every PR disclose LLM use, and Georgi Gerganov, who started the project, fills his in with the model and the hardware.

He has 17 PRs in this release, and 14 disclose an LLM. All 14 name an open model in llama.cpp's own GGUF format:

| Model | PRs | Hardware | Generation speed |
| --- | --- | --- | --- |
| Qwen3.8-Flash-Next | 6 | MacBook M5 Max 128GB, RTX 5090 32GB | 53.8 to 71.1 tok/s |
| DeepSeek-V4.1-Flash-1017 | 5 | Mac Studio M5 Ultra 256GB | 38.7 to 62.0 tok/s |
| GLM-5.3-Flash | 3 | Mac Studio M5 Ultra 256GB | 43.6 to 46.0 tok/s |

One of them is [#30565](https://github.com/ggml-org/llama.cpp/pull/30565), "llama : prune". That's the PR that deleted the 36 functions I had Gemma port away from. This is its disclosure:

```markdown
- AI usage disclosure: YES
  - Harness: `dsh:llama.cpp`
  - Hardware: `Mac Studio, M5 Ultra, 256GB`
  - Model: ggml-org/DeepSeek-V4.1-Flash-1017-GGUF
  - Stats: `◷ 1.8h · ⚙ 8.4m · ↑ 965k · ↓ 233k · ⇈ 399.3 tok/s · ⇊ 59.2 tok/s · ≡ 764 · ⌨ 51`
```

I read that stats line as a 1.8 hour session: 965k tokens in, 233k out, at 59.2 tokens per second. The harness is DeepSeek Harness, which I [wrote about last week](https://www.trevorlasn.com/blog/deepseek-harness-keeps-uploading-your-sessions-after-you-turn-telemetry-off).

So the public API of llama.cpp 1.0 was trimmed by a local model running on llama.cpp. My laptop did the 15-line version of that job, and his desktop did it across the library.

That's the map as I measured it. A 4.59 GB model handles tool calls, extraction and mechanical code changes, and needs thinking turned on before I'd trust its SQL or its lookups.

A 14.62 GB model writes ten working functions in under a minute. Hours of agent work on a C++ codebase run locally too, if you own 128 GB of memory.

I use Claude Code every day, and none of these three replaces it. But if I had a folder of screenshots or release notes to turn into JSON, I'd run it here before I paid per token.