Connect a local GGUF model to filesystem tools in llama.cpp’s built-in Web UI. This guide covers Windows, macOS and Linux, using one disposable folder so you can check tool access before working with real files.
You will start llama-server, load an MCP configuration and verify a read-only task. MCP is optional for local chat. Its filesystem tools can also write files, so keep the first setup away from Documents, work repositories and credentials.
Lachie’s take: What I like is that this adds useful tool access without requiring a large agent framework. What deserves caution is the permission boundary: the setup is only as safe as the folders and commands you expose.
Tested during editorial review: Commands and behaviour were checked against the merged llama.cpp change, current llama-server documentation and official MCP filesystem-server instructions. Follow the validation steps for your operating system before granting access to real files.
What changed in llama.cpp?
If your tools are image workflows rather than filesystem utilities, the Comfy MCP setup guide shows how Claude Code, Cursor and other local clients can control a carefully scoped ComfyUI installation.
The merged implementation adds stdio-based MCP support to llama-server. It reads a Cursor-compatible JSON configuration, starts the configured MCP processes, discovers their tools and makes those tools available through llama-server. MCP tools are exposed as <server>_<tool>, for example filesystem_read_text_file.
Think of llama.cpp as the brain and MCP as a labelled tool drawer. The model can be shown the tools it needs without being handed the keys to the entire workshop. I like this direction because it adds a practical bridge between “the model knows what to do” and “the model can use a tool to do it.”
One important distinction: this is an MCP client inside llama-server. It does not magically turn every OpenAI-compatible application into an MCP client. The simplest place to try it is llama.cpp’s own Web UI at http://127.0.0.1:8080.
If your priority is a reusable OpenAI-compatible endpoint rather than llama.cpp’s built-in Web UI, our LocalAI on Windows with Docker guide covers that alternative.
This backend MCP support is still new and upstream describes it as experimental. I would not use it with untrusted servers or untrusted users; this is a controlled local-workflow feature, not something to casually expose on the internet.
Before you start
The setup has a few moving parts, but none of them is especially mysterious. Think of llama.cpp as the engine, the model as the driver and the MCP server as the carefully limited set of tools in the boot. Check each part before connecting them.
- A current official llama.cpp build with
--mcp-servers-configin its help output. The original walkthrough usedb10142; the v0.4.0 server documentation was rechecked on 5 September 2026. - A GGUF instruct model that can use tools reliably.
- Node.js and
npxfor the filesystem example. - A terminal or PowerShell window.
- One non-sensitive test directory that the MCP server is allowed to access.
If you are new to local models, first read our local Llama installation guide. Its model choices are older, but it explains the basic local-model workflow. You can also browse the newer AI guides hub.
Step 1: Update llama.cpp
Download a suitable binary from the official llama.cpp releases, or build it from source. Open a terminal in the folder containing the server executable. Check that exact binary:
Windows PowerShell, from the binary folder:
.\llama-server.exe --version
.\llama-server.exe --helpmacOS or Linux, from the binary folder:
./llama-server --version
./llama-server --helpPowerShell needs the .\ prefix to run an executable from the current folder when it is not on PATH. Using the explicit path also avoids picking up an older installed copy. Check that --help lists --mcp-servers-config before continuing.
Before replacing a working build, follow the llama.cpp 0.3.0 safe upgrade guide to preserve the current version, migrate changed flags and keep a tested rollback path.
Step 2: Create an allowlisted MCP test folder
Now comes the useful (and potentially dangerous) part. Start with an empty test folder. If the model misunderstands an instruction, the most interesting thing it should be able to damage is a disposable text file. Do not point your first test at Documents, a cloud-drive root, a work repository or anything containing credentials.
Windows PowerShell:
New-Item -ItemType Directory -Path "$env:USERPROFILE\mcp-test"macOS or Linux:
mkdir -p "$HOME/mcp-test"Step 3: Create the MCP configuration
Create a file named mcp.json. This small JSON file is where you decide which MCP process may start and which folder it can see. Replace the example path with the exact allowlisted test-folder path on your computer; this is one place where copying a path almost correctly is not good enough.
Windows configuration:
{
"mcpServers": {
"filesystem": {
"command": "cmd",
"args": [
"/c",
"npx",
"-y",
"@modelcontextprotocol/server-filesystem",
"C:/Users/YOUR_NAME/mcp-test"
]
}
}
}The official server instructions use cmd /c on Windows so that npx.cmd launches correctly. JSON also requires each backslash in a Windows path to be doubled.
macOS or Linux configuration:
{
"mcpServers": {
"filesystem": {
"command": "npx",
"args": [
"-y",
"@modelcontextprotocol/server-filesystem",
"/absolute/path/to/mcp-test"
]
}
}
}The folder allowlist limits what the filesystem MCP tools expose to the model, but it is not operating-system isolation. The spawned Node process still runs with your normal user-account permissions.
Use a disposable folder containing no credentials or irreplaceable files. The server’s write_file, edit_file and move_file tools can make destructive changes. If you need stronger isolation, run the experiment inside a container or virtual machine.
Step 4: Start llama-server with MCP
This is the point where the pieces meet. Use your own GGUF model and configuration paths. I keep --jinja explicit here so the template behaviour is visible, although it is already enabled by default in the release used for this guide. It is not sufficient on its own: the model still needs reliable tool calling and a compatible chat template. The --host 127.0.0.1 setting keeps the first experiment on your own machine, which is where I would leave it until the setup is working and properly secured.
Windows:
.\llama-server.exe -m "C:/Models/your-model.gguf" --jinja --mcp-servers-config "C:/path/to/mcp.json" --host 127.0.0.1macOS or Linux:
./llama-server -m "/path/to/your-model.gguf" --jinja --mcp-servers-config "/path/to/mcp.json" --host 127.0.0.1By default, llama-server listens on port 8080. Wait until the model and MCP server finish loading, then open http://127.0.0.1:8080 in your browser.
Step 5: Test the MCP tool safely
Check each layer separately. First open http://127.0.0.1:8080/health: a ready server returns HTTP 200 with {"status":"ok"}. This confirms model readiness, not MCP access. Next inspect http://127.0.0.1:8080/tools for the registered filesystem tools.
- Put one harmless text file in
mcp-test. - Open llama.cpp’s Web UI and ask it to call
filesystem_list_allowed_directories. Confirm that only your intended test directory is listed before asking it to read a file. - Start with a read-only prompt: “List the files in the allowed directory and summarise the text file. Do not create, change, move, or delete anything.”
- Inspect the actual tool call, file path and returned text. Compare the response with the harmless file you created; a plausible summary alone is not proof of a tool call. A read-only prompt does not disable write-capable tools, and the UI may not ask for approval before a write.
- Only test file creation after read access works and only inside the allowlisted test folder.
A small model may describe a tool call without producing the structured call llama-server expects. That is a model or chat-template limitation, not necessarily an MCP connection failure. Try a tool-capable instruct model and keep --jinja enabled.
Security rules worth keeping
Local does not automatically mean safe. The model may be running on your computer, but every MCP server is still a new permission boundary. I would treat the folder list and command list with the same care as a service account: give it the smallest job that works.
- Use trusted MCP servers only. Each server is a child process running with the permissions of your user account.
- Allow the smallest possible directory. Never begin with an entire drive, home folder or user profile.
- Keep llama-server on localhost. CORS settings are not authentication. Do not change the host to
0.0.0.0unless you have deliberately secured the network boundary. - Prefer a config file. Inline JSON containing API keys may appear in the operating system’s process list.
- Protect and exclude the config. Restrict its file permissions and never commit a secret-bearing MCP config; add it to
.gitignore. - Do not rely on an approval prompt. A filesystem tool may be able to write, edit or move files without the Web UI asking first. Use disposable data, backups and least privilege.
- Pin what you trust.
npx -ycan fetch the newest package. For a repeatable setup, review and pin a specific package version.
Troubleshooting llama.cpp MCP setup
If the first attempt fails, resist the urge to grant broader permissions or reinstall everything. An MCP problem is usually a path, process, configuration or tool-template issue. Work through those separately and keep the allowlist narrow while you diagnose it.
Unknown option: –mcp-servers-config
Check the version and help output using the explicit executable path from Step 1. A second copy on PATH can hide the build you just downloaded. If that exact binary lacks the option, choose a current official release; adding flags for other tool features will not provide stdio MCP support.
MCP config reports no servers
Validate the JSON and make sure the top-level property is exactly mcpServers. Curly quotes copied from a document editor will also break JSON.
npx does not start on Windows
Run node --version and npx --version first. If either command fails, install Node.js and open a fresh terminal so the updated PATH is available. On Windows, keep the cmd plus /c configuration shown above.
The filesystem server exits immediately
A child process stopping after startup is not necessarily a failure: llama-server discovers tools, stops the process and restarts it when a tool is called. If discovery or the tool call fails, check the log, executable path and allowed folder. This example passes an existing directory explicitly; do not assume the client supplies one through MCP Roots.
Tools appear, but the model never uses them
First check the startup log for MCP warmup, then open http://127.0.0.1:8080/tools as a diagnostic. That endpoint is internal and may change, but it should reveal whether tools were registered; the Web UI may group MCP tools with its built-in tools. If the tools exist but are never called, the likely problem is the model or chat template. Some models need an explicit --chat-template or --chat-template-file. Test with a direct request such as “Use the filesystem tool to list the allowed directory.”
Frequently asked questions
Does llama.cpp now support every MCP server?
The new integration targets MCP servers that communicate over stdio and can be launched from the JSON configuration. It does not mean every transport, server, or downstream client combination has been verified.
Can I use several MCP servers?
Yes. Add more named entries under mcpServers. Start with one, verify it, and add others individually so failures and permissions remain easy to diagnose.
Is an internet connection required?
Local inference and an already-installed local MCP server can run without a cloud model. This example uses npx -y, which needs Node/npm and normally downloads the latest package on its first run. For reproducibility, review and pin a specific version. Individual MCP servers may also call online services, so check each server’s documentation and code.
Is MCP access automatically safe because the model is local?
No. Local processing can improve privacy, but a tool can still read or change local data within the permissions you grant. Treat MCP configuration as privileged access.
Sources and next steps
Source review, 5 September 2026: Rechecked the v0.4.0 release, server documentation and PowerShell executable-path rules. This walkthrough uses MCP tools only: it does not need --tools all or --agent. The --ui-mcp-proxy option serves a separate browser-side use case.
- Merged llama.cpp stdio MCP implementation
- Official llama-server documentation
- Official filesystem MCP server instructions
- llama.cpp release b10142
- llama.cpp security guidance
My take: native MCP support makes llama.cpp considerably more useful than a local chat box. It creates a path from “the model knows what to do” to “the model can use a tool to do it,” without automatically moving the whole workflow into a cloud service.
That does not make local tool use harmless. Treat every MCP server as a new permission boundary: expose the smallest possible working directory, test with disposable data, read the server’s documentation and expand access only when there is a clear reason.
For a first experiment, deliberately boring is good. Let the model list files in the test folder, read one harmless text file and stop. Once that works, add permissions one folder and one tool at a time.
Prefer a packaged desktop front end? Our Unsloth Desktop setup guide shows how to download and run a local GGUF, then launch Codex or Claude Code against it without assembling the full stack by hand.
If you also run models in Ollama, use the Promptfoo regression-test guide to compare their outputs before switching models. It does not test MCP tool permissions.