How to Run Superwhisper S1-mini Locally with llama.cpp

Checked scope: I checked these instructions against the S1-mini by Superwhisper v1 model card, its GGUF repository and llama.cpp documentation on 21 August 2026. I have not installed or benchmarked the model. Performance figures in this guide come from the publisher and are labelled as such.

This guide shows how to run Superwhisper S1-mini locally with llama.cpp, send it a transcript and fix the prompt settings that commonly cause blank output.

This is Superwhisper’s S1-mini transcript normaliser, not Fish Audio’s OpenAudio S1-mini text-to-speech model.

S1-mini by Superwhisper cleans the rough text produced by a speech-recognition engine. It removes fillers and repeated words, resolves clear self-corrections, adds punctuation and writes spoken dates, times, numbers, currency and email addresses in a readable form.

It does not listen to audio. Put it after Whisper, Parakeet, NeMo-Speech.cpp or another ASR engine:

audio -> speech recognition -> raw transcript -> S1-mini -> clean text

The smallest official GGUF is a 462 MiB Q4_K_M file. Version 1 handles English only and is designed for about 1,000 input tokens at a time. Those limits make it suitable for dictation, short notes and transcript chunks rather than an entire meeting in one request.

What you need

• A 64-bit Windows, macOS or Linux computer.
• About 1 GB of free space for llama.cpp, the quantised model and cache headroom.
• An internet connection for the first model download.
• A raw English transcript. S1-mini cannot accept an audio file.
• A recent llama.cpp build with Hugging Face download and Jinja chat-template support.

If you still need the speech-recognition stage, start with the NeMo-Speech.cpp local transcription guide. S1-mini begins where that tool’s raw text ends.

Install llama.cpp

Windows

Open PowerShell and install the current package through Windows Package Manager:

winget install llama.cpp

Close and reopen PowerShell, then check that the server command is available:

llama-server --version

If you need a hardware-specific prebuilt archive, choose the matching package from the official llama.cpp release page and keep all files from that archive together.

macOS

brew install llama.cpp
llama-server --version

Linux

Homebrew on Linux can use the same package:

brew install llama.cpp

The llama.cpp project also offers an installer at llama.app. Review an installer script before piping it into a shell, or use the prebuilt Linux archives and source-build instructions linked from the official repository.

Start S1-mini on your computer

Run this command in PowerShell, Terminal or a shell. It downloads the publisher’s Q4_K_M GGUF on the first run and serves it only on your computer:

llama-server -hf superwhisper/s1-mini-GGUF:Q4_K_M --jinja --chat-template-kwargs '{"enable_thinking":false}' --temp 0 --host 127.0.0.1 --port 8080

Keep that window open. Wait until llama.cpp reports that it is listening on port 8080. On Windows, a manually downloaded build may call the file llama-server.exe.

Four settings in that command matter:

• --jinja applies the model’s chat template.
• enable_thinking:false selects the non-reasoning template used during training.
• --temp 0 keeps the text transformation deterministic.
• --host 127.0.0.1 avoids exposing an unauthenticated server to the local network.

Do not replace the thinking setting with --reasoning-budget 0. The S1-mini model card says that creates a different assistant prefix and reduces output quality.

Send the first cleanup request

S1-mini was trained with a fixed system message. Open the official S1-mini model card, copy the system message from its example and replace the clearly marked value below. Keep the copied wording unchanged.

Open a second PowerShell window:

$body = @{
  messages = @(
    @{
      role = "system"
      content = "PASTE_THE_EXACT_S1_MINI_SYSTEM_MESSAGE_FROM_THE_MODEL_CARD"
    },
    @{
      role = "user"
      content = "[Styling: semi-formal] [Structure: prose] [Context: general]`nso um i need to send the the report by friday no wait make that thursday"
    }
  )
  temperature = 0
} | ConvertTo-Json -Depth 5

$response = Invoke-RestMethod `
  -Uri "http://127.0.0.1:8080/v1/chat/completions" `
  -Method Post `
  -ContentType "application/json" `
  -Body $body

$response.choices[0].message.content

The result should keep the intended change to Thursday, remove the false start and return only cleaned text. Exact punctuation can depend on the model build, so check meaning as well as appearance.

macOS or Linux request

The same request works with curl. Replace the marked system-message value first. The JSON is enclosed in single quotes so the system message and control line are passed unchanged.

curl http://127.0.0.1:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "system", "content": "PASTE_THE_EXACT_S1_MINI_SYSTEM_MESSAGE_FROM_THE_MODEL_CARD"},
      {"role": "user", "content": "[Styling: semi-formal] [Structure: prose] [Context: general]\nso um i need to send the the report by friday no wait make that thursday"}
    ],
    "temperature": 0
  }'

Choose the output style

Every transcript must begin with one control line. Keep the spelling, brackets and order unchanged:

[Styling: semi-formal] [Structure: prose] [Context: general]
your raw transcript starts here

Use only the trained values:

semi-formal, prose and general are a sensible first test. Change one setting at a time and compare the result against the raw transcript.

Use S1-mini with a transcription workflow

Keep the two stages separate while testing. First save the ASR output as plain text. Then send that text to S1-mini. This makes it clear whether a missing name, number or sentence came from speech recognition or cleanup.

1. Transcribe a short recording with your existing ASR tool.
2. Keep an untouched copy of the raw transcript.
3. Add the control line and send one chunk to S1-mini.
4. Compare names, figures, dates and commitments with the recording.
5. Only automate the pipeline after several representative tests.

The llmfit guide can help estimate what other local models fit your hardware, but S1-mini itself is small enough that CPU use is the normal starting point.

Optional Python setup with Transformers

The original BF16 weights require Python and PyTorch. Use this route if you are building an application or need direct access to the tokenizer. For a basic local service, the GGUF route above is easier.

python -m venv .venv

Activate it on Windows:

.\.venv\Scripts\Activate.ps1

Or on macOS and Linux:

source .venv/bin/activate

Install a Transformers version with Qwen3 support:

python -m pip install --upgrade pip
pip install "transformers>=4.51.0" torch

Use the complete Python example from the official model card. Keep enable_thinking=False, greedy decoding and the exact system and control messages. Add device_map="auto" only if you have installed accelerate and want automatic GPU placement.

Fix common S1-mini problems

The output is blank

First check the input. A transcript containing only fillers such as “um” is meant to return an empty string. For real text, blank output usually means the thinking template was left on. Restart llama.cpp with both --jinja and --chat-template-kwargs '{"enable_thinking":false}'.

Also confirm that the system message was copied exactly from the official model card and that the user message starts with a valid control line.

Transformers reports KeyError: 'qwen3'

Your Transformers package is too old:

pip install --upgrade "transformers>=4.51.0"

Restart Python after the upgrade.

The result is garbled or invents text

Check for a changed system message, a misspelled control value or a temperature above zero. S1-mini is trained for one fixed transformation format and is not a general instruction-following model. Do not add a second request such as “explain your changes”.

A long meeting loses details

Keep each pass under roughly 1,000 input tokens. Split at sentence or speaker boundaries and process chunks in order. Do not cut through an email address, date, list or self-correction. Rejoin the cleaned chunks afterwards and review the joins.

It does not transcribe an audio file

That is expected. Use Whisper, Parakeet, NeMo-Speech.cpp or another ASR engine first, then pass its plain-text output to S1-mini.

Port 8080 is already in use

Stop the other service or choose another local port:

llama-server -hf superwhisper/s1-mini-GGUF:Q4_K_M --jinja --chat-template-kwargs '{"enable_thinking":false}' --temp 0 --host 127.0.0.1 --port 8081

Change the request URL to match.

Non-English text is changed badly

S1-mini v1 is English-only. Keep other languages out of the cleanup stage or choose a model documented for that language. Names and mixed-language passages still need human checking.

Privacy and licence checks

After the files are downloaded, llama.cpp can process the supplied text locally. That does not make the complete workflow private by itself. A cloud speech-recognition service may already have received the audio, and exposing port 8080 beyond 127.0.0.1 can let other devices send requests to the server.

The publisher describes the model as Apache 2.0 with an additional naming requirement. Any use must retain the exact product and publisher naming specified in the model licence. If you plan to redistribute the weights or bundle them in paid software, read the current licence text and confirm that your packaging and notices comply.

S1-mini questions

Is S1-mini a speech-to-text model?

No. It cleans text produced by a separate speech-to-text model.

How much memory does S1-mini need?

The official Q4_K_M file is 462 MiB. Runtime memory is higher than the file size and depends on context length, llama.cpp build and backend, so there is no universal RAM figure.

Can S1-mini run without a GPU?

Yes. The publisher says the quantised build runs on a laptop CPU. No universal speed is promised, so test your own hardware with representative transcript lengths.

Can it create lists and emails?

Yes. Use Structure: lists for a clear enumeration and Context: email for greeting, body and sign-off layout. These settings change formatting; they do not give the model permission to invent content.

How accurate is it?

Superwhisper reports 94.8% token accuracy on 7,519 held-out English cases using greedy decoding on the Q4_K_M build. That is a publisher test, not an independent result, and it does not guarantee accuracy on names, technical terms or your microphone’s ASR errors.

Does it work in Ollama or LM Studio?

The publisher says the GGUF builds can work in llama.cpp-based tools, including Ollama and LM Studio. The important part is preserving the trained chat template, exact system message, control line and empty thinking block. A generic Qwen chat profile may return blank output.

Sources

• S1-mini by Superwhisper v1 model card and Transformers quickstart
• Official S1-mini GGUF builds and llama.cpp notes
• Official S1-mini transcript-cleanup demo
• llama.cpp installation documentation
• llama.cpp server documentation