Skip to content
OriginalSoftware

llama.cpp hits 1.0: the engine under local AI just promised not to break you

After three and a half years of build numbers, llama.cpp ships version 1.0.0: a stability promise for the public C API, a cleanup of deprecated functions, and smaller memory footprints for long conversations.

A linocut print of a stone arch bridge under construction, its keystone block hanging from copper ropes above the unfinished gap
The keystone, about to lock the arch: version 1.0 is the contract that holds the structure together.
On this page

In brief

  • llama.cpp and ggml shipped v1.0.0 on November 13, 2026; breaking changes to the public C API now require a major-version bump.
  • The release removes 35 deprecated functions from llama.h and promotes the NextN speculative-decoding API into the main header.
  • FP8 KV caches and a host-memory MoE expert cache shrink the memory footprint for long conversations and large mixture-of-experts models.
  • The promise covers only the public C API and ABI boundaries — not model outputs, GGUF files, CLI flags, or the server's HTTP API.
  • The project supports more than 150 architecture variants and ships through eight distribution channels, from Homebrew to XCFramework.

On November 13, llama.cpp shipped version 1.0.0, and its underlying tensor library ggml did the same. The headline is not a feature, a speed record, or a new model. It is a promise: from now on, a breaking change to the public C API arrives only with a major-version bump.

That sounds like bookkeeping. For the software underneath local AI, it is the most consequential thing the project has done since it started. llama.cpp is the C/C++ inference engine that runs large language models on hardware from consumer laptops to enterprise servers, and it sits underneath a long tail of tools — Ollama, LM Studio, desktop apps, editors, agents — that link against its libraries or embed them outright. When a library that many things depend on changes its interface without warning, the breakage ripples outward to software that never mentions llama.cpp by name.

Three and a half years of build numbers

The project began in 2023, when Georgi Gerganov set out to run Meta's LLaMA model in pure C and C++, without Python and without requiring a GPU. Three and a half years later, the Hugging Face announcement says "many downstream applications already depend on our work" — and the versioning had never caught up with that fact.

Until now, builds were tagged with raw commit numbers: b1234, b12359, and the like. A number like that tells you when a build was cut and nothing about whether your code will still compile against it. The 1.0.0 tag, which corresponds to build b12359, replaces that guessing game with a contract.

What the promise actually covers

The rules apply to all ggml-org projects using semantic versioning, and they are deliberately simple. A MAJOR bump means a breaking change to the public C API — the headers under ./include. A MINOR bump means a backward-compatible feature, a backend update, new model support, or an API addition. A PATCH bump means a bug fix with no API change.

Two libraries carry the promise. libllama is the inference engine, with its public headers in llama.cpp/include/; libggml is the tensor library underneath, with its headers in ggml/include/. Many projects link one or both directly, so a change in either can break code that never touches the other.

The practical effect is for the people who ship software on top of the libraries. App developers can declare a compatible range — 1.x — instead of pinning a single commit and hoping. Packagers on Debian, Homebrew, or conda-forge can hold back a breaking update instead of guessing whether a release is safe to ship. And a future 2.0 becomes a visible signal to schedule migration work, rather than a surprise discovered at compile time. The announcement frames it as a responsibility: "Every new integration means more developers depending on our library interfaces and therefore having a flexible but stable API is paramount."

A technical ink cutaway of a three-layer machine: a copper band binds the gear bed and engine block below while a row of small windows floats apart above
The promise binds the two library layers; what sits on top — flags, scripts, server responses — is outside it.

The cleanup 1.0 starts with

Version 1.0.0 uses the major milestone to clear out the old API in one go. The release removes 35 deprecated functions from llama.h, along with the warmup setter llama_set_warmup and the defragmentation-threshold option defrag_thold and its --defrag-thold flag. The NextN speculative-decoding API moves from llama-ext.h into the main llama.h header, and llama_set_embd_state_src is renamed llama_set_ctx_state_src. ggml 1.0.0 does the same kind of pruning, dropping its deprecated public API and the legacy DIAG_MASK operations.

If your code calls any of the removed functions, it will not build against 1.0.0. That is the point of a major version, and it is the first thing to check before upgrading. The breakage is intentional and, for once, labelled.

What else shipped in the tag

The release is not only a contract. The notes since v0.9.0 list real changes, and the ones that matter most are about memory — the constraint that decides which models fit on which machine.

FP8 key-value caches arrive through the f8_e4m3 type, with flags -ctk and -ctv to select it. The KV cache is the conversation memory a model keeps while generating, and storing it in 8-bit floating point instead of 16-bit roughly halves its footprint.

A host-memory cache for mixture-of-experts models lets rarely used experts live in ordinary system RAM instead of on the GPU, with a configurable share of the work for the CPU when an expert has to be fetched. External PLE shards can now be loaded from separate GGUF files.

Three new text architectures arrive — Kimi-K3.1-Pro, Ripple-65B-Preview, and xing4_0 — alongside drama-3 text-to-speech and vision support for MageVL-Cowork and Kimi-K3.1. llama-server gets security hardening against server-side request forgery, CORS-origin abuse, and DNS rebinding, plus an OpenAI-compatible /v1/audio/speech endpoint and cancellation of MCP tool calls when a client disconnects. The web UI is redesigned, with chat groups, a live-preview markdown editor, and a mobile-friendly layout.

One thing the release does not do is claim speed. Neither the announcement nor the release notes give before-and-after performance figures, so treat any claim of a speed-up from the version number alone as unverified.

What 1.0 does not promise

Read the promise narrowly, because its narrowness is the honest part. The versioning rules cover the public C API and, in the team's wording, API and ABI boundaries. They say nothing about model outputs staying identical between versions, about every GGUF file loading forever, or about command-line flags and the server's HTTP responses staying fixed.

That gap is normal for a first stable release, and it tells you what to test. If you pin a version and script llama-server flags, run your own smoke test after each upgrade. If you ship a compiled integration, the semver guarantee is now your protection; if you drive the tools from a shell script, it is not. Load the GGUF files you rely on and run a fixed prompt set before you roll it out.

Why a version number matters

Local inference has grown into a stack. A model file in GGUF format sits on top of ggml kernels. A runtime such as llama.cpp loads the file. Apps, editors, agents, and chat front ends sit on top of the runtime. When the runtime changes an API without a signal, every layer above it can break — and the breakage lands on users who never chose llama.cpp at all.

The scale of what the promise covers is part of the story. The project supports more than 150 architecture variants and all mainstream speculative decoding methods, and ships through eight distribution channels, from pre-release artifacts to Docker, Debian, Homebrew, conda-forge, MacPorts, Winget, and XCFramework.

None of that makes 1.0 a finish line. The minor-version rules explicitly allow new features, backends, and models, and the project will keep moving at the same pace. What changes is how the breaking ones are announced. After three and a half years of build numbers, the engine under local AI has a version number that means something — and the next time it breaks your code, it will at least have the decency to tell you first.

More from grep.tips

All articles