Small local models are often sold like magic. I am more interested in whether the setup is boring in the best possible way. Liquid AI’s new LFM2.5-2.6B is interesting because the official local route really is short: install a current runner, pull a roughly 1.67 GB Q4_K_M file, and start chatting.
Released on 4 August 2026, LFM2.5-2.6B is a 2.6-billion-parameter text model aimed at tool use, data extraction, RAG and long-running agent workflows. Liquid AI lists a 128K context window and native tool calling, but it also gives an unusually useful warning: this is not its recommended model for agentic coding or knowledge-heavy work. That honesty matters.
Verification note: This guide was checked against Liquid AI’s release post, documentation and official Hugging Face model cards on 9 August 2026. The commands below are documentation-verified, not a claim that I benchmarked every operating system or piece of hardware. Speed figures are Liquid AI’s measurements, not independent tests.
The quick answer
If you already have Ollama, run this official Hugging Face model directly:
ollama run hf.co/LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_MIf you prefer llama.cpp, use the Windows or macOS/Linux instructions below. The Q4_K_M download is about 1.67 GB and is the sensible starting point for most laptops. A long context window still consumes extra memory, so start at 4K or 8K context rather than assuming 128K will be free.
What you need
- A 64-bit Windows, macOS or Linux computer
- At least 4 GB of free RAM for a comfortable Q4 starting point; more is useful for longer context and other applications
- About 3 GB of free disk space for the model, runner and working room
- An internet connection for the first download
- No dedicated GPU is required for the GGUF route, although compatible GPU offload can improve speed
Those RAM and disk figures are practical starting estimates, not guarantees. The model file is only part of memory use: the context cache, runner and operating system all need room too.
Choose the right model file
| Format | Approximate size | Best use |
|---|---|---|
| Q4_0 | 1.59 GB | Smallest official 4-bit option |
| Q4_K_M | 1.67 GB | Best first choice for most people |
| Q5_K_M | 1.94 GB | A little more quality, a little more memory |
| Q8_0 | 2.87 GB | Higher fidelity if memory is not tight |
| BF16/F16 | 5.4 GB | Fine-tuning or testing where full precision matters |
Liquid AI’s documentation recommends Q4_K_M as the general balance of size and quality. That is the version used throughout this guide.
Option 1: run LFM2.5-2.6B with Ollama
This is the shortest path if you want a terminal chat rather than a configurable server.
Step 1: install or update Ollama
Download Ollama from the official Ollama download page. If it is already installed, update it before continuing so the Hugging Face GGUF syntax is supported.
Step 2: pull and run the model
ollama run hf.co/LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_MOllama downloads the official quantized model on the first run. Once the prompt appears, try a task that suits a small agent model:
Turn this messy note into JSON with fields for owner, due_date and next_action:"Sam will renew the SSL certificate next Tuesday and confirm it in the IT channel."I would not begin by asking it to write an entire application or recall obscure facts. The model card specifically steers users toward tool use, extraction, RAG and long-context workflows instead.
Option 2: run it with llama.cpp
llama.cpp gives you more control and can expose an OpenAI-compatible local endpoint. That makes it useful if you want to connect another application later. If that is your goal, my llama.cpp MCP setup guide explains the broader local-server workflow.
Windows
Open Windows Terminal or PowerShell and install llama.cpp through WinGet:
winget install llama.cppClose and reopen the terminal, then start an interactive chat:
llama cli -hf LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_MmacOS or Linux
Liquid AI’s official model card points to llama.cpp’s installer:
curl -LsSf https://llama.app/install.sh | shllama cli -hf LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_MIf you prefer not to pipe an installer into a shell, download a release from the official llama.cpp releases page, inspect it, and use the included binary instead.
Start a local API and web interface
llama serve -hf LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_MLeave that terminal running and open http://127.0.0.1:8080. Applications that support an OpenAI-compatible endpoint can normally use http://127.0.0.1:8080/v1. Keep the server bound locally unless you understand authentication, firewalling and the risks of exposing an agent endpoint.
Option 3: use Transformers for Python experiments
Use the native checkpoint when you need direct access from Python. Liquid AI says the model requires transformers>=5.0.0. The full BF16 weights need considerably more memory than Q4_K_M.
python -m venv .venv# Windows.venvScriptsactivate# macOS or Linuxsource .venv/bin/activatepython -m pip install --upgrade pippip install "transformers>=5.0.0" torch accelerateSave this as chat_lfm.py:
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamermodel_id = "LiquidAI/LFM2.5-2.6B"model = AutoModelForCausalLM.from_pretrained( model_id, device_map="auto", dtype="bfloat16",)tokenizer = AutoTokenizer.from_pretrained(model_id)streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)inputs = tokenizer.apply_chat_template( [{"role": "user", "content": "Extract the action items from this note: Patch the server Friday, then email Alex."}], add_generation_prompt=True, return_tensors="pt", tokenize=True,)["input_ids"].to(model.device)model.generate( inputs, do_sample=True, temperature=0.1, top_k=50, repetition_penalty=1.1, max_new_tokens=300, streamer=streamer,)
python chat_lfm.pyIf your CPU or GPU does not support BF16, use the GGUF route rather than blindly changing precision settings. PyTorch installation also varies by operating system and accelerator, so use the selector on the official PyTorch installation page.
How much of the 128K context should you use?
A model supporting 128K tokens does not mean every laptop should start there. Context uses memory, and very long prompts can make a small model slower or less focused. Start with 4K to 8K for ordinary chat, then increase only when a real document or tool trace requires it. For llama.cpp, add a conservative context value such as:
llama cli -hf LiquidAI/LFM2.5-2.6B-GGUF:Q4_K_M -c 8192 --temp 0.1 --top-k 50 --repeat-penalty 1.1Troubleshooting
“llama” is not recognised
Close and reopen your terminal after installation. On Windows, confirm the package installed with winget list llama.cpp. If you downloaded a ZIP instead, run the executable from that extracted folder or add it to your PATH.
The download restarts or fails
Check that you have several gigabytes of free disk space and that security software is not blocking the runner. Retry on a stable connection. Download only from Liquid AI’s official Hugging Face organisation or the runner’s verified model integration.
It is much slower than the headline numbers
Liquid AI’s 220 tokens/s and 113 tokens/s figures were measured on an Apple M5 Max and AMD Ryzen AI Max+ 395. They are vendor benchmarks on specific hardware. Your CPU, quantization, context length, prompt size, thermal limits and GPU offload all change the result.
The response starts with thinking text
The official model card describes LFM2.5-2.6B as an always-thinking model and its chat template inserts a <think> tag. Use a current runner that understands the supplied chat template. Do not manually invent a different template unless you have a specific integration reason.
Tool calls are inconsistent
Keep tool names unambiguous, schemas small and parameter descriptions explicit. Liquid AI documents Python-style tool calls by default, with JSON available when requested in the system prompt. A 2.6B model can still choose the wrong tool, so require confirmation before destructive actions and validate every argument in your application.
Is LFM2.5-2.6B worth installing?
Yes, if you want a compact local model for extraction, private document workflows, structured output or an inexpensive tool-calling experiment. The Q4_K_M build is small enough that trying it is low drama.
No, if your main goal is difficult coding, authoritative factual answers or a replacement for a much larger frontier model. Small models are most useful when the task is narrow and the guardrails are clear. That is less glamorous than “an agent that does everything,” but it is far more likely to work.
Frequently asked questions
Does LFM2.5-2.6B need a GPU?
No. The official GGUF build is designed for local CPU inference. A compatible GPU can accelerate some or all layers, but it is not required to begin.
Can it run on 4 GB of RAM?
The 1.67 GB Q4_K_M file can fit within a small memory budget, but the operating system, runner and context cache need additional RAM. Four gigabytes is a tight lower bound; 8 GB or more is a more comfortable practical target.
Is it completely private?
Inference can stay local after the files are downloaded. Privacy still depends on the surrounding application, logs, plugins and any tools the model calls. A local model connected to a cloud search tool is not a fully offline workflow.
Can I use it in a commercial project?
The model is distributed under Liquid AI’s LFM1.0 licence. Read the current licence and your organisation’s obligations before production or commercial deployment; this guide is not legal advice.
What should I try next?
Once the local server works, connect it to a small, reversible tool such as a read-only notes search or structured-data extractor. If you want a broader local API option, compare it with my LocalAI Windows guide. Avoid giving any local agent unrestricted shell, email or file-delete access.