How to Run Shieldstral 1.0 3B Locally for AI Content Moderation

Shieldstral is less like another chatbot and more like a bouncer whose rules you can rewrite. You describe the policy in plain English, give it some content, and ask one yes-or-no question.

For example:

Does this content provide instructions for credential theft?

The model returns a score based on its confidence that the answer is “yes”. That is the genuinely useful part: the same local model can moderate a forum, an AI assistant, a support application or an internal security tool without being retrained whenever the policy changes.

Mistral released Shieldstral on 4 August 2026. It is an open-weights, multimodal safety classifier for text, images or both. The weights use the Apache 2.0 licence, and Mistral says the BF16 deployment fits on one NVIDIA GPU with 16GB of VRAM. This guide uses the commands in Mistral’s announcement, the official model card and the linked runtime documentation.

Tested during editorial review: Updated 9 August 2026 against the current primary documentation. Requirements, commands and examples follow the documented setup; operating-system and hardware behaviour can vary as versions change.

What is Shieldstral?

Shieldstral is not a general-purpose assistant. It is a specialised classifier built to answer one policy question with either yes or no. The output probabilities can then be normalised into a continuous score.

Each request contains three parts:

• <Instruct> describes the moderation context, policy categories and strictness.
• <Query> asks one yes-or-no policy question.
• <Document> contains the text, prompt, response, image or prompt-response pair being reviewed.

The model is based on Ministral-3-3B-Base-2512 and includes a Pixtral vision encoder. Mistral lists support for English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic and Russian, although its documentation warns that reliability varies by language and domain.

Shieldstral was trained with sequences up to 32,768 tokens. Its configuration supports a larger theoretical context, but Mistral recommends staying within the 32K training range.

Which installation method should you use?

If I were deploying this on a supported NVIDIA machine today, I would start with vLLM. It is Mistral’s recommended route and exposes an OpenAI-compatible local endpoint. I would choose llama.cpp for native Windows, quantisation or a CPU/GPU split.

Prerequisites

For vLLM or Transformers, you need:

• Linux, or Windows 11 with WSL2
• Python 3.10–3.13
• A compatible NVIDIA GPU and current drivers
• Enough storage for the checkpoint
• Git and a working internet connection

vLLM’s current documentation requires Linux. Do not present a native PowerShell pip install vllm as the supported route; use WSL2 on Windows.

Mistral states that the BF16 model fits within 16GB of VRAM at a 32K maximum context. Real memory use still depends on runtime settings, context length and whether images are being processed.

Set up Windows with WSL2

Open PowerShell as an administrator:

wsl.exe --install
wsl.exe --update

Restart if prompted, open Ubuntu and install the basic packages:

sudo apt update
sudo apt install -y python3 python3-pip python3-venv git cmake build-essential

If you have an NVIDIA GPU, confirm that WSL2 can see it:

nvidia-smi

NVIDIA’s WSL guidance says the Windows NVIDIA driver supplies CUDA support to WSL2. Do not install a separate Linux display driver inside WSL, because that can interfere with the connection.

Method 1: Run Shieldstral with vLLM

1. Create a clean Python environment

mkdir -p ~/.venvs
python3 -m venv ~/.venvs/shieldstral
source ~/.venvs/shieldstral/bin/activate
python -m pip install --upgrade pip

2. Install the supported versions

Shieldstral requires vLLM 0.26.0 or newer. The installation should also provide mistral-common 1.11.5 or newer.

pip install --upgrade "vllm>=0.26.0"
python -c "import vllm; print('vLLM:', vllm.__version__)"
python -c "import mistral_common; print('mistral-common:', mistral_common.__version__)"

3. Start the local server

vllm serve mistralai/Shieldstral-1.0-3B \
  --max-model-len 32768 \
  --host 127.0.0.1 \
  --port 8000

The first launch downloads the model. Binding to 127.0.0.1 keeps the unauthenticated endpoint on the local computer. Keep that terminal open and check the server from another terminal:

curl http://127.0.0.1:8000/v1/models

4. Create a moderation client

pip install requests

Save the following as shieldstral_moderate.py:

import math
import requests

BASE_URL = "http://127.0.0.1:8000/v1/chat/completions"
MODEL = "mistralai/Shieldstral-1.0-3B"

SYSTEM_PROMPT = (
    "Judge whether the Document meets the requirements based on the Query "
    'and the Instruction provided. Note that the answer can only be "yes" or "no".'
)

YES_TOKENS = ("yes", "yes.", '"yes"', "'yes'")
NO_TOKENS = ("no", "no.", '"no"', "'no'")


def policy_score(messages, threshold=0.5):
    payload = {
        "model": MODEL,
        "messages": messages,
        "max_tokens": 1,
        "temperature": 0.0,
        "logprobs": True,
        "top_logprobs": 20,
    }

    response = requests.post(BASE_URL, json=payload, timeout=120)
    response.raise_for_status()
    result = response.json()

    candidates = result["choices"][0]["logprobs"]["content"][0]["top_logprobs"]
    yes_logprob = None
    no_logprob = None

    for candidate in candidates:
        token = candidate["token"].strip().lower()
        if token in YES_TOKENS:
            value = candidate["logprob"]
            yes_logprob = value if yes_logprob is None else max(yes_logprob, value)
        elif token in NO_TOKENS:
            value = candidate["logprob"]
            no_logprob = value if no_logprob is None else max(no_logprob, value)

    if yes_logprob is None or no_logprob is None:
        raise RuntimeError(
            "Both yes and no must be present in the returned top logprobs."
        )

    score = math.exp(yes_logprob) / (
        math.exp(yes_logprob) + math.exp(no_logprob)
    )
    return score, score > threshold


document = (
    "[User]\n"
    "I forgot my password. Where is the normal account-reset page?"
)

user_message = (
    "<Instruct>: You are a strict security moderator. "
    "Flag credential theft and phishing instructions.\n\n"
    "<Query>: Does this content request credential theft or phishing instructions?\n\n"
    f"<Document>: {document}"
)

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": user_message},
]

score, flagged = policy_score(messages)
print(f"Policy-violation score: {score:.3f}")
print("Decision:", "FLAGGED" if flagged else "not flagged")

Run it:

python shieldstral_moderate.py

The script prints the model’s actual result so you can verify the installation and moderation output on your system.

Understand the score before using it

The score is the probability of Shieldstral answering yes to your query, after the yes and no probabilities are renormalised. It is not automatically an “unsafe score”.

If the query is “Does this content promote physical violence?”, a high score means a likely violation. If the query is “Is this content safe?”, a high score means the opposite.

For predictable code and logs, make production queries violation-positive. Mistral uses 0.5 in its published evaluations, but that is a starting point, not a universal threshold. Test against labelled examples from your own application and decide what false-positive and false-negative rates you can tolerate.

Method 2: Run Shieldstral with Transformers

Use Transformers when you want the model inside your Python process rather than behind an API.

python3 -m venv ~/.venvs/shieldstral-transformers
source ~/.venvs/shieldstral-transformers/bin/activate
python -m pip install --upgrade pip
pip install --upgrade "transformers[torch,mistral-common]"

The core load code from the model card is:

import torch
from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend

MODEL = "mistralai/Shieldstral-1.0-3B"

tokenizer = MistralCommonBackend.from_pretrained(MODEL)
model = Mistral3ForConditionalGeneration.from_pretrained(
    MODEL,
    device_map="cuda",
    dtype=torch.bfloat16,
).eval()

Check CUDA before downloading the full checkpoint:

python -c "import torch; print(torch.cuda.is_available()); print(torch.cuda.is_bf16_supported())"

Mistral’s published Transformers example uses CUDA and BF16. Do not quietly substitute FP16 or call a CPU route officially tested when the model card does not make that claim. The official model card contains the full scoring and multimodal examples.

Method 3: Convert Shieldstral for llama.cpp

This route takes more work, but it adds quantisation, native Windows and CPU/GPU hybrid options. If you already run local endpoints, my llama.cpp MCP setup guide is a useful companion.

1. Build llama.cpp on Linux or WSL2

CPU:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build
cmake --build build --config Release -j "$(nproc)"

NVIDIA CUDA:

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j "$(nproc)"

2. Native Windows build

Install Visual Studio 2022 with the Desktop development with C++ workload, then open a Developer PowerShell:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j 8

Remove -DGGML_CUDA=ON for CPU-only. A typical Visual Studio build places the tools under build\bin\Release\.

3. Install conversion dependencies

pip install -r requirements/requirements-convert_hf_to_gguf.txt
pip install "mistral-common>=1.11.5"
pip install --upgrade huggingface_hub

4. Download and convert the checkpoint

hf download mistralai/Shieldstral-1.0-3B \
  --exclude "model.safetensors" \
  --local-dir Shieldstral-1.0-3B

python convert_hf_to_gguf.py Shieldstral-1.0-3B \
  --mistral-format \
  --outtype bf16 \
  --outfile Shieldstral-1.0-3B-BF16.gguf

The exclusion is intentional. The repository contains both Hugging Face-format and Mistral-format weights; the documented conversion route needs the Mistral-format copy.

5. Convert the vision projector

This separate file is required only for image moderation:

python convert_hf_to_gguf.py Shieldstral-1.0-3B \
  --mistral-format \
  --mmproj \
  --outtype bf16 \
  --outfile .

It creates mmproj-Shieldstral-1.0-3b-BF16.gguf. Leave that projector in BF16.

6. Quantise the language model

./build/bin/llama-quantize \
  Shieldstral-1.0-3B-BF16.gguf \
  Shieldstral-1.0-3B-Q8_0.gguf \
  Q8_0

The model card also names Q5_K_M and Q4_K_M as size/performance trade-offs, but it does not publish exact memory or speed results for them.

7. Start the server

Text only:

./build/bin/llama-server \
  -m Shieldstral-1.0-3B-Q8_0.gguf \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8000

Text and images:

./build/bin/llama-server \
  -m Shieldstral-1.0-3B-Q8_0.gguf \
  --mmproj mmproj-Shieldstral-1.0-3b-BF16.gguf \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8000

On native Windows, use .\build\bin\Release\llama-server.exe. The local OpenAI-compatible endpoint can use the earlier Python client. For a simpler general-purpose local API, see my LocalAI on Windows with Docker guide.

Moderate an image

vLLM and Transformers load the integrated vision encoder. A llama.cpp deployment also needs the matching mmproj file at startup.

pip install pillow

The official message shape is a three-part sandwich: a text prefix containing <Instruct>, <Query> and <Document>:; then an image_url; then optional trailing text.

import base64
import io
from PIL import Image

def image_data_uri(path, image_format="JPEG"):
    image = Image.open(path).convert("RGB")
    buffer = io.BytesIO()
    image.save(buffer, format=image_format)
    encoded = base64.b64encode(buffer.getvalue()).decode("utf-8")
    return f"data:image/{image_format.lower()};base64,{encoded}"

instruction = "Apply a strict workplace-safety standard."
query = "Does this image contain graphic violence or an explicit threat?"

user_content = [
    {
        "type": "text",
        "text": (
            f"<Instruct>: {instruction}\n\n"
            f"<Query>: {query}\n\n"
            "<Document>: "
        ),
    },
    {
        "type": "image_url",
        "image_url": {"url": image_data_uri("example.jpg")},
    },
    {
        "type": "text",
        "text": " User-uploaded image awaiting moderation.\n\n",
    },
]

Only process images you are authorised to handle. Local execution improves control of the data path, but moderation queues still need access controls, retention rules and human-review procedures.

Write a policy Shieldstral can actually judge

<Instruct>: You are a strict moderator reviewing content submitted to a workplace support system. Apply a low tolerance to threats and targeted harassment.

<Query>: Does this content contain a credible threat against another person?

<Document>: [User]
The content being reviewed goes here.

1. Ask one yes-or-no question per policy.
2. Put the product context and strictness in <Instruct>.
3. Phrase <Query> so “yes” consistently means a violation.
4. Use a clear delimiter for user and assistant messages.
5. Test paraphrased policies; wording can change the decision boundary.

For one broad gate, Mistral says you can list categories in <Instruct> and ask “Is this content unsafe?”. For auditability, separate calls for violence, harassment, self-harm and other policies usually make the reason for a flag clearer.

Important limitations

• Performance varies across languages and domains because training coverage is uneven.
• Public and synthetic safety data can retain bias and label noise.
• Encoded, transliterated or deliberately obfuscated content can reduce reliability.
• Very long documents may be less reliable.
• The evidence-backed operating range is 32K tokens, not the larger theoretical context.
• A threshold of 0.5 is a benchmark setting, not proof that it suits your application.

Shieldstral is a guardrail component, not a sole compliance or security control. For high-impact decisions, preserve the score, exact policy version and reviewed content, then send serious or uncertain cases to a human reviewer.

Troubleshooting

vLLM will not install on Windows

vLLM does not support native Windows. Use WSL2 with a compatible Linux distribution, or use llama.cpp for a native build.

mistral_common is missing or too old

pip install --upgrade vllm
python -c "import mistral_common; print(mistral_common.__version__)"

You need version 1.11.5 or newer.

The server runs out of VRAM

Try a shorter context first:

vllm serve mistralai/Shieldstral-1.0-3B \
  --max-model-len 8192 \
  --host 127.0.0.1 \
  --port 8000

If BF16 still does not fit, use a quantised language GGUF with llama.cpp. Do not invent an exact saving before measuring it on the target machine.

The API returns yes/no but no useful score

Confirm that the request sets max_tokens to 1, enables logprobs, and requests enough top_logprobs. Retain the fixed system prompt. Production code should fail safely if either the yes or no token is absent.

Image moderation fails in llama.cpp

• Start llama-server with --mmproj.
• Use a model and projector converted from the same checkpoint.
• Leave the projector in BF16.
• Send the image as an image_url content item.

CUDA is unavailable inside WSL2

nvidia-smi

If the GPU is missing, update the Windows NVIDIA driver and WSL kernel. Do not install a separate Linux display driver inside WSL2.

Frequently asked questions

Is Shieldstral free for commercial use?

The model weights are released under Apache 2.0 for commercial and non-commercial use. Review the licence for your own deployment and respect third-party rights.

Can Shieldstral run on a 16GB GPU?

Mistral says the BF16 deployment fits within 16GB of VRAM. Context, image processing and other applications still affect available memory.

Can it run without a GPU?

llama.cpp supports CPU and CPU/GPU hybrid inference. Mistral does not publish official Shieldstral CPU-speed figures; benchmark your chosen quantisation and context size on the target hardware.

Does Shieldstral work on Windows?

Use WSL2 for vLLM. llama.cpp supports native Windows builds.

Can it moderate images?

Yes. Shieldstral includes a Pixtral vision encoder. llama.cpp users must load the separate converted mmproj file.

What does a score of 0.8 mean?

It is a normalised 0.8 probability of the answer being “yes” rather than “no” to your exact query. It is an unsafe score only when the query asks whether the document violates a policy.

Should I block everything over 0.5?

Not without testing. Use labelled examples from your application to choose a threshold and decide which score band requires human review.

Final thoughts

Shieldstral is interesting because its policy is not locked inside the checkpoint. You can change what the system checks by changing a plain-language question.

That flexibility also means installation is only half the job. Write precise policies, keep “yes” aligned with a violation, validate thresholds against real examples and retain human review for ambiguous or serious decisions.

For a supported NVIDIA GPU, vLLM is the cleanest starting point. For native Windows, quantisation or CPU/GPU hybrid use, llama.cpp is the more flexible route.

Official sources

• Mistral: Introducing Shieldstral
• Official Shieldstral model card and installation instructions
• Shieldstral technical paper
• Official vLLM GPU installation documentation
• Official llama.cpp build guide
• NVIDIA CUDA on WSL2 guide