North Micro Vision is a compact 2.4-billion-parameter vision-language model that you can run on your own computer with Hugging Face Transformers. It accepts text and one or more images, then returns text for tasks such as image captioning, document questions, OCR, chart analysis and visual grounding.
The attractive part is not that it replaces a large hosted assistant. Cohere Labs released North Micro Vision Instruct on 12 August 2026 under Apache 2.0. Its checkpoint is about 4.63 GiB, and Cohere documents a standard Transformers path for local inference.
What North Micro Vision can and cannot do
| Good fit | Poor fit |
|---|---|
| Captioning and visual question answering | Open-ended general chat |
| OCR, documents, charts and figures | Hard mathematical reasoning |
| Multiple images in one prompt | Code generation |
| Object grounding and bounding boxes | Tool calling or autonomous agents |
| Multilingual image understanding | Very long, untested multimodal prompts |
Cohere describes the model as a compact foundation for prototyping and customisation. It combines a 2B language model with a 400M native-resolution vision encoder. The language backbone has a 128K-token context window, but its validated multimodal training context is 8K tokens. Do not turn the larger text-only number into a promise about long image conversations.
If your main goal is a general local chatbot, compare this with my Qwen3.8-27B local guide. For an easier graphical model manager, see how to install Unsloth Desktop.
Requirements
- The commands below target Windows with Python 3.11. The Python API may also run in supported Linux or macOS PyTorch environments, but Cohere does not publish a macOS compatibility or performance guarantee.
- Reserve about 12–15 GB of free space for the 4.63 GiB checkpoint, PyTorch, Transformers and package/download caches.
- A recent NVIDIA GPU is the most practical route. CPU loading is possible through PyTorch, but generation may be slow.
- Enough available VRAM or system memory for the bfloat16 checkpoint, image tokens and runtime overhead. File size is not the complete memory requirement.
- A stable connection for the first model download.
The official checkpoint is bfloat16. Cohere does not publish a minimum-VRAM promise in the model card, so treat any exact number from a third-party blog as hardware-specific. Native-resolution images also use more memory as their dimensions grow.
Step 1: create a clean Windows environment
Open PowerShell in a new project folder:
mkdir north-micro-vision
cd north-micro-vision
py -3.11 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install --upgrade pipIf PowerShell blocks the activation script, allow locally created scripts only for the current PowerShell session, then activate the environment again:
Set-ExecutionPolicy -Scope Process RemoteSignedThis setting ends when you close the current PowerShell window. If your device is managed by an employer, follow its security policy instead of overriding it.
Step 2: install PyTorch for your hardware
PyTorch packages depend on the operating system and accelerator. Use the command generated by the official PyTorch installation selector. For a CPU-only smoke test, the basic package is:
python -m pip install torch torchvision --index-url https://download.pytorch.org/whl/cpuFor an NVIDIA GPU, select the CUDA build that matches a supported PyTorch combination. Do not choose a CUDA wheel only because its version number resembles the driver version.
Check what PyTorch can see:
python -c "import torch; print('PyTorch:', torch.__version__); print('CUDA:', torch.cuda.is_available()); print('GPU:', torch.cuda.get_device_name(0) if torch.cuda.is_available() else 'CPU')"Step 3: install the required Transformers build
Cohere’s release requires support currently present in the Transformers 5.16.0 development branch. Transformers 5.15.0 was the latest stable release when checked, so the official model card instructs users to install Transformers from the main GitHub repository:
python -m pip install accelerate pillow
python -m pip install "git+https://github.com/huggingface/transformers.git"The model repository does not ship custom Python model code or declare an auto_map entry, so this guide intentionally does not use trust_remote_code=True. Installing Transformers from a Git development branch still executes package code from that branch, so keep it inside the isolated virtual environment and move to the released 5.16.0 package when it becomes available.
Once version 5.16.0 is published, replace the source installation with the pinned release:
python -m pip install accelerate pillow "transformers==5.16.0"Pinning a released version is easier to reproduce than following a changing development branch. Check the official release first rather than assuming 5.16.0 is available.
Step 4: create the first image test
Create a file named run_north_micro.py with this documentation-based example:
import torch
from transformers import AutoModelForImageTextToText, AutoProcessor
model_id = "CohereLabs/North-Micro-Vision-Instruct"
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
model_id,
dtype="auto",
device_map="auto",
)
image_url = "https://cdn-uploads.huggingface.co/production/uploads/66d732effe6684fc16b12c28/Io_5OCmftsmH-n158ZtPs.png"
messages = [{
"role": "user",
"content": [
{"type": "image", "url": image_url},
{"type": "text", "text": "Describe this image in three factual bullet points."},
],
}]
inputs = processor.apply_chat_template(
messages,
tokenize=True,
add_generation_prompt=True,
return_tensors="pt",
return_dict=True,
).to(model.device)
with torch.inference_mode():
outputs = model.generate(
**inputs,
max_new_tokens=160,
do_sample=False,
)
generated_ids = [
output_ids[len(input_ids):]
for input_ids, output_ids in zip(inputs.input_ids, outputs)
]
response = processor.batch_decode(
generated_ids,
skip_special_tokens=True,
clean_up_tokenization_spaces=False,
)[0]
print(response)Run it:
python run_north_micro.pyThe first run downloads the model and tokenizer into the Hugging Face cache. A successful answer proves that Python, PyTorch, Transformers and the model package can work together. It does not yet prove OCR accuracy on your own documents.
Step 5: analyse a local image
After the official URL test works, place a harmless image named sample.png in the project folder. Avoid real invoices, identity documents or confidential screenshots until you have confirmed that every input remains local.
Replace the URL with a Pillow image:
from PIL import Image
local_image = Image.open("sample.png").convert("RGB")
messages = [{
"role": "user",
"content": [
{"type": "image", "image": local_image},
{"type": "text", "text": "Transcribe the visible text. Preserve line breaks and mark uncertain words with [unclear]."},
],
}]The weights and inference run locally after download, but privacy still depends on your code. The URL example fetches a remote image. A local Pillow object avoids that fetch, while package downloads, telemetry settings and any surrounding application still deserve review.
Useful prompts for OCR, charts and grounding
Document extraction
Return JSON with these keys only: document_type, date, total, currency and supplier. Use null when a value is not visible.Validate the returned JSON before passing it to another system. A vision model can misread digits or invent a missing field.
Chart analysis
State the chart title, both axis labels, the highest labelled value and two trends. Do not infer values that are not visible.Bounding boxes
Locate the red warning icon and return one bounding box as [x1, y1, x2, y2].North Micro Vision reports coordinates on a normalised 0–1000 scale. Convert them to pixels as follows:
x1_px = x1 / 1000 * image_width
y1_px = y1 / 1000 * image_height
x2_px = x2 / 1000 * image_width
y2_px = y2 / 1000 * image_heightOptional: deterministic versus sampled output
The first script uses do_sample=False so repeated tests are easier to compare. Cohere’s model card recommends these sampling values for a more varied response:
outputs = model.generate(
**inputs,
max_new_tokens=160,
do_sample=True,
temperature=0.7,
top_p=0.8,
top_k=20,
)Use deterministic output for extraction tests. Sampling may be more natural for descriptions, but it can also make structured results less repeatable.
Troubleshooting
The model type is unknown or an import is missing
Check the installed Transformers version:
python -c "import transformers; print(transformers.__version__)"If 5.16.0 is not available from PyPI yet, reinstall the current source build inside the active virtual environment. Confirm that python and pip point to .venv before changing packages.
CUDA runs out of memory
Close other GPU applications and begin with one smaller image. Native resolution is useful, but a large scan creates many image tokens and increases memory use. Resize a test copy rather than overwriting the original:
from PIL import Image
img = Image.open("sample.png").convert("RGB")
img.thumbnail((1600, 1600))
img.save("sample-test.png")If automatic device placement still fails, use CPU inference for a smoke test or choose a smaller/quantised community conversion only after reviewing its publisher and licence. Do not imply that a third-party quant is Cohere’s official checkpoint.
Generation is very slow
Confirm that torch.cuda.is_available() is true on an NVIDIA setup and that the model is not fully assigned to the CPU. Reduce image dimensions and max_new_tokens. Flash Attention 2 is optional and should be installed only on a supported CUDA system using Cohere’s documented command.
The OCR answer contains invented text
Ask for uncertain words to be marked, use do_sample=False, crop irrelevant areas and compare important fields with a conventional OCR engine. Do not automate payments, identity checks or compliance decisions from one unverified model response.
A system prompt makes results worse
Cohere says the model was not trained with system prompts and does not recommend them, even though the chat template accepts the role. Put the task instruction in the user message.
vLLM will not load the model
Hugging Face currently displays a generic generated vLLM command, but Cohere’s model card and release article still say public vLLM support is coming soon. The vLLM repository did not contain a CohereCompass implementation when checked. Use Transformers until an official vLLM release or model-specific recipe confirms support.
Frequently asked questions
Is North Micro Vision open source?
The weights and model card are published under Apache 2.0. “Open source” can mean different things for model training data and the complete development process, so the precise claim is that the released model artefacts use the Apache 2.0 licence.
How large is the download?
The official manifest lists model.safetensors at 4,969,765,248 bytes, about 4.63 GiB, plus a roughly 18.6 MiB tokenizer and small configuration files. Leave extra space for packages and cache operations.
Does it run offline?
Inference can run without a hosted API after the model and dependencies are cached and your script uses local images. The first setup downloads packages and model files. Test offline operation before relying on it in a disconnected environment.
Can it read more than one image?
Yes. The official model card lists interleaved text and images and multi-image understanding. More and larger images consume more of the validated 8K multimodal context and increase memory use.
Can I use it as an agent?
Not directly. Cohere explicitly says tool calling and agentic workflows are unsupported. Use it as a vision component whose output your application validates.
How does it compare with LFM2.5-VL-3B?
Cohere’s published benchmark table did not test LFM2.5-VL-3B; it tested LFM2.5-VL-1.6B. Do not infer a direct 3B comparison from that table. Test both models on the same representative documents and compare accuracy, latency and memory on the same hardware.
What to do next
Start with the official image and deterministic output. Then try one non-sensitive local document, verify each extracted field manually and record the package versions. Only after that should you build a batch process or consider fine-tuning.
If you need to expose a local model through an OpenAI-compatible service, read my LocalAI Windows and Docker guide. For coding-agent integrations, my llama.cpp MCP guide explains the surrounding local-server pattern, although it does not add unsupported tool calling to North Micro Vision.