Search a PDF by describing the page you need: a network diagram, a spending table or a chart with a particular trend. NeoMME-Retriever ranks page images against your question. This guide builds a small local Python example that returns page numbers, so you can open the source and check the evidence.
Start with three pages you know well. You will compare two scoring methods, inspect their rankings and try a question the document cannot answer. That gives you a useful first check before indexing a folder of reports.
What NeoMME does with a PDF page
H Company announced NeoMME on 3 September 2026. Its retrieval models encode questions and document-page images. Rendering a page as an image preserves the visible layout supplied to the model, including charts and tables; an OCR transcript is not required for this search path.
The result is a ranking, not an extracted spreadsheet or a written answer. If you need invoice amounts as structured fields, use an extraction tool. If you need to locate the page containing the invoice breakdown, retrieval is the relevant step. For a separate image-reading task, see the North Micro Vision OCR and image-analysis guide.
Before you start
- Python 3.10 or newer, pip and Git. Use a separate environment for this project.
- A local PDF you are allowed to process. Pick a short, non-sensitive file for the first run.
- Internet access for the initial package and model downloads, plus space for their cache.
- The
Hcompany/NeoMME-260M-Retrievercheckpoint, not the base encoder or a single-head fine-tuning variant.
The 260M retriever model card lists 263 million parameters and a default longest image side of 2,048 pixels. This example caps rendered pages at 1,024 pixels and uses CPU execution to keep the setup straightforward. Those are starter choices, not a minimum-hardware specification. Small text may need a higher-resolution pass.
1. Create an isolated Python environment
Create and open a new project folder, then run:
python -m venv .venvOn Windows PowerShell, activate it with:
.\.venv\Scripts\Activate.ps1On macOS or Linux, use python3 for environment creation if that is your installed command, then activate it with:
source .venv/bin/activateIf PowerShell blocks activation, use .\.venv\Scripts\python.exe in place of python in the remaining commands. You do not need to change the machine’s execution policy. Python’s environment documentation explains both approaches.
2. Install the retrieval and PDF packages
The launch example uses Transformers from its main branch. The command below pins that route to commit c93057d, checked on 6 September, which contains the required NeoMME classes. The vision and chat-template extras supply its image-processing and template dependencies. It also requires Sentence Transformers 6 or newer for MeanMaxSim scoring. Keep this source build separate from other projects; the installation documentation explains source installs.
python -m pip install --upgrade pip
python -m pip install "torch>=2.5" pillow pypdfium2 "sentence-transformers>=6.0.0" "transformers[vision,chat_template] @ git+https://github.com/huggingface/transformers.git@c93057d4835cd31752bb56f59989dd27696eb45b"Check that the required classes and scoring function import, then save the installed package versions:
python -c "from transformers import NeoMMEForRetrieval, NeoMMEProcessor; from sentence_transformers.util import mean_maxsim; print('NeoMME imports are ready')"
python -m pip freeze > requirements-snapshot.txtThis import check does not download the model or prove retrieval quality. It catches a missing class or incompatible environment before the larger download.
3. Save the local PDF search script
Save the following as search_pdf.py. It renders the first three pages with pypdfium2 and applies the documented NeoMME retrieval API. The PDF stays local; this script does not call a hosted OCR or answer-generation service.
import argparse
from pathlib import Path
import pypdfium2 as pdfium
import torch
from sentence_transformers.util import cos_sim, mean_maxsim
from transformers import NeoMMEForRetrieval, NeoMMEProcessor
parser = argparse.ArgumentParser()
parser.add_argument("pdf", type=Path)
parser.add_argument("query")
parser.add_argument("--pages", type=int, default=3)
parser.add_argument("--max-side", type=int, default=1024)
args = parser.parse_args()
if args.pages < 1 or args.max_side < 32:
parser.error("Use a positive page count and max-side of at least 32.")
images = []
pdf = pdfium.PdfDocument(str(args.pdf))
for index in range(min(args.pages, len(pdf))):
page = pdf[index]
scale = args.max_side / max(page.get_size())
bitmap = page.render(scale=scale)
images.append(bitmap.to_pil().convert("RGB").copy())
bitmap.close()
page.close()
pdf.close()
if not images:
raise SystemExit("The PDF contains no pages.")
model_id = "Hcompany/NeoMME-260M-Retriever"
processor = NeoMMEProcessor.from_pretrained(model_id)
model = NeoMMEForRetrieval.from_pretrained(model_id).eval()
def prepare(items, task):
return processor.apply_chat_template(
items,
task=task,
tokenize=True,
return_dict=True,
return_tensors="pt",
processor_kwargs={"padding": "longest"},
)
page_messages = [
[{"role": "user", "content": [{"type": "image", "image": image}]}]
for image in images
]
query_messages = [
[{"role": "user", "content": [{"type": "text", "text": args.query}]}]
]
pages = prepare(page_messages, "document")
query = prepare(query_messages, "query")
with torch.inference_mode():
page_vectors = model(**pages)
query_vectors = model(**query)
late = mean_maxsim(
query_vectors.embeddings,
page_vectors.embeddings,
a_mask=query["attention_mask"],
b_mask=pages["attention_mask"],
)
dense = cos_sim(
query_vectors.dense_embeddings,
page_vectors.dense_embeddings,
)
print(f"Searched the first {len(images)} pages of {args.pdf.name}")
for label, scores in (("Late interaction", late), ("Dense", dense)):
print(f"\n{label}:")
for index in scores[0].argsort(descending=True).tolist():
print(f" Page {index + 1}: {scores[0, index].item():.4f}")The page numbers refer to positions in the PDF, including covers. They may differ from printed page labels. The script rebuilds embeddings on each run; it is a small validation example, not a persistent search service.
4. Ask a question and inspect the ranked page
Place your PDF beside the script. Replace this filename and question with ones relevant to its first three pages:
python search_pdf.py "sample-report.pdf" "Which page shows the network diagram?"Open the PDF at each method’s highest-ranked page. Record whether it contains the evidence you wanted. Scores rank these candidate pages; they are not confidence percentages. Do not compare a dense score directly with a late-interaction score as though both share a calibrated scale.
5. Use a small evaluation set before adding more documents
Here is an editorial test plan you can adapt. Make a three-page sample containing a network diagram, a monthly-cost table and a project timeline. Write the correct page number for each question before running the model:
- “Where are the firewall and switches connected?” checks a diagram-focused query.
- “Where can I compare monthly hosting costs?” checks table retrieval without asking the model to calculate a total.
- “Which page shows the rollout dates?” checks a paraphrase rather than an exact heading.
- “What is the office lease expiry date?” should be marked absent if your sample has no lease information.
The script will still rank pages for the absent question. Treat that as a reminder to inspect evidence, not proof that the top page answers it. Log the query, expected page, both top-ranked pages and any failure. Keep the PDF revision, rendering size and model/package versions with your notes.
Troubleshooting
NeoMME or MeanMaxSim will not import
Run python -m pip show transformers sentence-transformers in the same environment as the script. Check that the source installation completed and Sentence Transformers is at least version 6. An old environment or a local file named transformers.py can load different code from the intended package.
Memory fills or the first run is slow
Separate package/model downloads from inference when diagnosing the wait. For a smaller check, add --pages 1 --max-side 768 to the command. This reduces the example’s input workload; it is not a speed guarantee. Keep readable details when reducing resolution, and do not scale to a large library until this small run completes.
The wrong page ranks first
Check whether the right page was among the first three searched. Inspect the original for tiny text, rotation or poor scan quality. Keep the separate query and document task labels in the script. Try a precise evidence-seeking question, then record the miss; do not hide it by keeping only successful prompts.
Questions before using this in a bigger system
Can it work offline?
Download the required files first. Then add local_files_only=True to both from_pretrained calls to load cached files. Hugging Face also documents HF_HUB_OFFLINE=1 for preventing Hub HTTP calls. Keep document processing local if you later add an answer-generation step.
Does this replace OCR or a document database?
No. This example locates candidate pages. Field extraction, arithmetic, version tracking and access controls need their own handling. For a larger index, preserve the source filename, document revision and page number with every stored embedding. Replacing a PDF should trigger an explicit index update.
How do I turn a retrieved page into an answer?
A visual language model can receive selected page images after retrieval. That is a separate stage with its own privacy and accuracy checks. Build reliable page selection first; then use the document Q&A guide for broader system planning. Keep page references visible so a reader can verify the answer.
Official-source review, 6 September 2026: Model selection, retrieval calls, scoring and rendering APIs were checked against the sources below. The three-page evaluation is an editorial method for assessing your own documents; it contains no claimed benchmark result.