How to Install NVIDIA NeMo-Speech.cpp for Local Transcription

NeMo-Speech.cpp is the kind of local AI tool I pay attention to: it solves a specific job without dragging a Python inference stack into the runtime. NVIDIA’s native C++ project turns supported GGUF speech models into a command-line transcriber, subtitle generator, browser playground and documented OpenAI-compatible audio API subset. The simplest first test is the English Nemotron Q8 model and a WAV file.

The important catch is maturity. As of 14 August 2026, the project’s GitHub Releases page says there are no releases, so the public installation path builds from the repository checkout rather than installing a tagged binary package. This is not yet a one-click desktop app. Expect the commands to evolve, recheck the official documentation before a production deployment and pin a tested commit if you need a repeatable build.

Windows users need Visual Studio 2022 Build Tools with the Desktop development with C++ workload. Windows, macOS and Linux source builds also need Git, CMake 3.26 or newer, Ninja and the appropriate compiler and GPU toolkit.

What NeMo-Speech.cpp does

NeMo-Speech.cpp is built on ggml and runs quantised GGUF speech models without a Python inference stack. NVIDIA documents four useful ways to use it:

• transcribe audio from the command line;
• generate SRT, VTT or JSON output;
• run a local HTTP server and browser playground;
• connect software to its OpenAI-compatible audio API subset or realtime WebSocket endpoint.

If you only want a friendly model manager for local LLMs, this is more technical than Unsloth Desktop. NeMo-Speech.cpp makes sense when the job is specifically local speech recognition, subtitles or a speech API that you control.

What you need

You also need Python only for the Hugging Face command-line downloader used below. The runtime itself is native C++.

Choose a speech model

For a first test, I would use Nemotron Speech Streaming EN 0.6B. Its official model repository provides a ready-to-run 700 MB Q8 GGUF file and targets English audio. NVIDIA documents Q8_0 as the portable default. The project’s converter can create q4_k, q5_k and q6_k variants for lower storage and memory use, but those smaller quantisations are not additional ready-to-download files in the model repository used below.

If you need more languages, NVIDIA also documents Nemotron 3.5 ASR Streaming 0.6B for more than 40 language-locales. Parakeet TDT 0.6B v3 covers 25 European languages but is an offline model rather than a streaming one. Start with one model, prove the workflow, then add another only if its language or latency trade-off fits your use case.

1. Clone the repository

Open PowerShell, Terminal or your Linux shell and run:

git clone https://github.com/NVIDIA/NeMo-Speech.cpp.git
cd NeMo-Speech.cpp

2. Install NeMo-Speech.cpp

Windows

From PowerShell in the repository folder:

.\scripts\install.ps1 -Source

The installer performs its own version check, installs to %LOCALAPPDATA%\Programs\NeMoSpeech by default and updates the current user’s PATH. Open a new PowerShell window after it finishes, then confirm:

nemo-speech --version

Automatic selection chooses CUDA whenever nvidia-smi.exe is present, but compiling that backend also requires nvcc.exe from a supported CUDA Toolkit. If you have the NVIDIA driver but not the CUDA Toolkit, request a CPU build instead:

.\scripts\install.ps1 -Source -Backend cpu

To explicitly request a CUDA build after confirming nvcc.exe is available, use:

.\scripts\install.ps1 -Source -Backend cuda

macOS or Linux

scripts/install.sh --source
export PATH="$HOME/.local/bin:$PATH"
nemo-speech --version

The installer chooses Metal on Apple Silicon, CUDA when nvidia-smi is detected, and CPU otherwise. A CUDA source build also requires nvcc from the CUDA Toolkit. You can force the most compatible build with scripts/install.sh --source --backend cpu.

3. Download the English GGUF model

Install or update the Hugging Face downloader:

python -m pip install -U huggingface_hub

Then download the official Q8 model into a local models folder:

hf download nvidia/nemotron-speech-streaming-en-0.6b nemotron-speech-streaming-en-0.6b.q8_0.gguf --local-dir models

This command names the exact file, which avoids pulling every asset in the model repository.

4. Run the included transcription test

nemo-speech transcribe test_files/asr/wav/test/jfk.wav --model models/nemotron-speech-streaming-en-0.6b.q8_0.gguf

If that works, replace the sample path with your own WAV file:

nemo-speech transcribe recording.wav --model models/nemotron-speech-streaming-en-0.6b.q8_0.gguf

The CLI documentation lists mono or stereo PCM16 and float32 WAV files from 8 kHz to 96 kHz. It automatically downmixes and resamples supported WAV input.

5. Create subtitles or timestamped JSON

For an SRT subtitle file:

nemo-speech transcribe recording.wav --model models/nemotron-speech-streaming-en-0.6b.q8_0.gguf --format srt --output recording.srt

For WebVTT, change srt to vtt. For structured output with word timestamps:

nemo-speech transcribe recording.wav --model models/nemotron-speech-streaming-en-0.6b.q8_0.gguf --json --word-times

You can also process a folder recursively and control parallel work:

nemo-speech transcribe recordings/ --model models/nemotron-speech-streaming-en-0.6b.q8_0.gguf --recursive --output-dir transcripts --concurrency 4

Start with a low concurrency value. A higher number is not automatically faster if memory or GPU capacity is already tight.

6. Launch the local browser playground

nemo-speech serve --asr-model models/nemotron-speech-streaming-en-0.6b.q8_0.gguf --open

The documented default address is http://127.0.0.1:8080. The listener has no API key by default, but keeping it bound to 127.0.0.1 limits network access to your computer. Do not casually change the host to 0.0.0.0. For remote access, NVIDIA documents an API key, TLS and an explicit CORS origin, and recommends putting the secret in NEMO_SPEECH_HTTP_API_KEY instead of a command-line argument. NVIDIA identifies NIM as the supported production deployment path.

If you are building a broader local AI stack, compare this speech-specific server with LocalAI on Windows with Docker. Developers experimenting with local tool use may also want the llama.cpp MCP setup guide.

Optional: use the multilingual streaming model

Download the official Nemotron 3.5 ASR Q8 GGUF:

hf download nvidia/nemotron-3.5-asr-streaming-0.6b nemotron-3.5-asr-streaming-0.6b.q8_0.gguf --local-dir models

Then point the same transcribe or serve command at that filename. Check NVIDIA’s current model documentation for the supported locale codes and language-selection flags before building them into an automated workflow.

Troubleshooting

nemo-speech is not recognised

Open a new terminal after installation so the PATH change is loaded. On macOS or Linux, run export PATH="$HOME/.local/bin:$PATH" in the current shell and add it to your shell profile if the command then works.

The build cannot find CMake, Ninja or a compiler

Install every prerequisite listed for your operating system and confirm each command is available before rerunning the installer. Windows specifically requires the Visual Studio 2022 Build Tools for the documented source build.

The wrong compute backend was selected

Run nemo-speech doctor to see the compiled backends and detected devices. Runtime commands accept device selectors such as cpu, metal, cuda:0 and vulkan:0 through --device. The installers use backend names without a device number: cpu, metal, cuda or vulkan.

The model file cannot be found

Run the command from the repository folder or replace the relative model path with an absolute path. Check the filename carefully: the commands above use .q8_0.gguf.

My MP3 or video file is rejected

The documented CLI input is WAV. Convert the source to PCM WAV first, then retry. Avoid assuming that a media container is supported simply because another transcription tool accepts it.

Frequently asked questions

Does NeMo-Speech.cpp send audio to NVIDIA?

The runtime and GGUF model run locally in the documented workflow. The initial repository clone, dependency installation and model download require internet access. If privacy is critical, review the source, bind the server locally and monitor network activity in your own environment.

Can I use it to transcribe meetings or calls?

Technically, it can transcribe supported WAV recordings, but local processing does not remove consent, workplace-policy or retention obligations. Only process audio you are authorised to record and transcribe, and protect or delete the resulting files according to your requirements.

Is NeMo-Speech.cpp open source?

NVIDIA-authored runtime code is released under Apache 2.0, while bundled third-party components retain their own terms. Model licensing is separate. The English Nemotron model card uses the NVIDIA Open Model License, so read the runtime notices and the specific model card before commercial deployment or redistribution.

Can it transcribe in real time?

Yes, when used with a supported streaming model. NVIDIA documents a realtime WebSocket server as well as file transcription. Parakeet TDT 0.6B v3 is documented as offline-only, so model choice matters.

Should I use Q8 or a smaller quantisation?

Use Q8_0 first if your hardware can hold it. It is NVIDIA’s recommended portable default. Move to q6, q5 or q4 only when memory or storage is the limiting factor, and validate transcription quality on audio that resembles your real workload.

My recommendation

Use NeMo-Speech.cpp if you want a scriptable local speech runtime, subtitle output or a private HTTP endpoint and you are comfortable building a C++ project. Start with the English Q8 model and the included WAV test. If you only need occasional desktop dictation, a packaged app will be easier to maintain.

Primary sources

• NVIDIA NeMo-Speech.cpp repository
• Official installation guide
• Official CLI reference
• Official ASR model guide
• Official server and security configuration guide
• GitHub Releases page
• Nemotron Speech Streaming EN 0.6B model card
• Nemotron 3.5 ASR Streaming 0.6B model card