Run Local PDF OCR with Xinference 3.2 and DeepDoc

Short answer: run Xinference 3.2.0 on a loopback-only Docker port, launch the built-in DeepDoc image model, then upload a PDF to /v1/images/ocr with model=DeepDoc and kwargs={"task":"parse"}. The response contains ordered document elements rather than only a flat OCR string.

Xinference 3.2.0 added PDF input to its OCR API and a built-in DeepDoc model. The full parse task can combine OCR, layout, reading order, paragraph merging, tables and figures across an ordinary multi-page PDF. It is a useful local document-processing route when you do not want to send each file to a hosted OCR API, provided the model and server remain on infrastructure you control.

What arrived in Xinference 3.2.0

Unlike a retrieval framework, this first stage extracts structure from the document. You could later feed reviewed text into a system such as the site’s LlamaIndex document-chat workflow or compare it with the Quivr installation guide. Keep extraction and question answering as separate, testable steps.

Prerequisites and hardware choices

• A supported Docker installation with enough free disk space for the image, model files and temporary document data.
• A non-sensitive test PDF you are allowed to process.
• For the official GPU image, a compatible NVIDIA stack. Xinference’s 3.x Docker documentation specifies CUDA 13 and an NVIDIA driver of 580 or newer.
• For a simpler compatibility test, use the official CPU-suffixed image shown below and expect lower throughput.
• On Windows, run the multipart request with curl.exe, not PowerShell’s legacy curl alias.

Do not choose a GPU image solely from the graphics card marketing name. Check the installed driver, container GPU access and Xinference’s current image notes. The instructions make no promise that a particular PDF size will fit your RAM or VRAM.

1. Start Xinference on loopback

docker run -d --name xinference-deepdoc -p 127.0.0.1:9997:9997 -e XINFERENCE_HOME=/data -v xinference-deepdoc-data:/data xprobe/xinference:v3.2.0-cpu xinference-local -H 0.0.0.0

The service listens on all interfaces inside the container, while Docker publishes it only to 127.0.0.1 on the host. The named volume keeps downloaded model data when the container is replaced. The official GPU route uses --gpus all with xprobe/xinference:v3.2.0 after the documented NVIDIA prerequisites are met.

docker ps --filter name=xinference-deepdoc
docker logs --tail 100 xinference-deepdoc

Open http://127.0.0.1:9997 for the web interface or http://127.0.0.1:9997/docs for the local API documentation. Do not expose port 9997 to a LAN or the internet without configuring Xinference authentication, TLS and network access controls.

2. Launch DeepDoc

In the web interface, choose the image-model section, select DeepDoc and launch it. The equivalent standard Xinference CLI form is:

docker exec xinference-deepdoc xinference launch --model-name DeepDoc --model-type image

Run xinference launch --help inside your pinned image if a later build changes CLI flags. First launch can download model weights and create a model-specific environment, so it is not an offline step. Wait for the launch to finish and list running models:

docker exec xinference-deepdoc xinference list

Confirm the actual model UID displayed by Xinference. The examples below use DeepDoc, matching the documented request, but a customised launch can assign a different UID.

3. Parse a PDF through the verified API

Use a small, non-sensitive sample.pdf for the first request. In the same folder, create a plain UTF-8 file named deepdoc-options.json containing only:

{"task":"parse"}

Reading the JSON field from a file avoids nested-quote differences between Windows PowerShell and newer PowerShell. From that folder, run this single-line Windows command. It writes or replaces deepdoc-result.json and prints the HTTP status:

curl.exe -sS -X POST http://127.0.0.1:9997/v1/images/ocr -F "model=DeepDoc" -F "kwargs=<deepdoc-options.json" -F "image=@sample.pdf;type=application/pdf" -o deepdoc-result.json -w "%{http_code}"

On macOS or Linux, use curl instead of curl.exe; the same quoted file-reading form works. The < tells curl to read the field’s content from the JSON file, while @ uploads the PDF as a file. If authentication is configured, add the real bearer token using the syntax of your shell, for example, -H "Authorization: Bearer $env:XINFERENCE_TOKEN" in PowerShell. Do not add a fake Authorization header to an unauthenticated loopback test.

The exact field names matter:

• model=DeepDoc selects the launched model.
• kwargs={"task":"parse"} requests whole-document parsing.
• image=@sample.pdf uploads the file even though it is a PDF.
• /v1/images/ocr is the OCR endpoint used by the merged 3.2 implementation.

Do not pass pages or dpi with task=parse. The implementation rejects those options for full parsing because it consumes the whole document and uses cross-page coordinates. The documented PDF limit is 200 pages, with additional resource and pixel budgets that can reject problematic input earlier.

4. Validate the result before using it

Check the printed HTTP status before interpreting the output: a non-2xx status means the file may contain an error response, even if curl wrote it successfully. Open deepdoc-result.json in an editor that does not execute embedded HTML. The verified response shape includes a top-level task and an elements array similar to this abbreviated example:

{
  "task": "parse",
  "elements": [
    {
      "type": "table",
      "text": "<table>...</table>",
      "image_base64": "...",
      "metadata": { "page_number": 2 }
    }
  ]
}

The default image scope can include crops for tables and figures. Treat text as untrusted extracted content: table HTML should be sanitised before rendering in a browser, and base64 images should not be logged unnecessarily. A useful acceptance test checks all of the following:

• the HTTP request completes without an error object;
• task equals parse and elements is a non-empty array;
• page numbers map to the correct source pages;
• headings and paragraphs are in usable reading order;
• table cells match the visible table rather than merely looking plausible;
• critical names, dates and numbers are manually compared with the PDF.

OCR is an extraction aid, not evidence. Scans, rotated pages, handwriting, faint text and complex tables can fail silently. Keep the original document and a traceable review step before search indexing, automation or publication.

Network, credential and document safety

After the model and container assets are cached, the specific loopback DeepDoc request does not require a hosted OCR provider. That statement does not cover operating-system telemetry, container registries, later updates or other models configured in the same Xinference instance.

Troubleshooting

Stop or remove the service

Terminate the model using the UID shown by xinference list:

docker exec xinference-deepdoc xinference terminate --model-uid DeepDoc
docker stop xinference-deepdoc
docker rm xinference-deepdoc

The named volume remains after the container is removed. That is useful for a rollback, but it can retain model files and service data. Inspect and back up what you need before intentionally deleting it with docker volume rm xinference-deepdoc-data. Volume deletion is destructive and is not part of a routine stop.

Frequently asked questions

Can DeepDoc parse only selected pages?

Not through the 3.2 full parse task using pages or dpi; those options are rejected. Split a permitted copy of the PDF first if you need a selected-page workflow, while preserving the original and page mapping.

Does it need an NVIDIA GPU?

No for the documented CPU image route, but CPU processing can be slower. The official 3.x GPU image has specific CUDA and driver requirements; verify them before downloading the larger image.

Is a local PDF automatically private?

No. Privacy depends on the endpoint, bind address, host security, logs and output handling. The loopback example keeps the request on one host, but a remote endpoint uploads the entire file to that server.

Can I send the output straight to an LLM?

Technically yes, but first validate reading order and critical values, remove unnecessary personal data, and understand the LLM provider’s data path. Chunking unreviewed OCR can preserve extraction errors while making them harder to trace.

Official sources

Sources reviewed: 10 September 2026. Reviewed against the official Xinference 3.2.0 release, DeepDoc request contract and Docker documentation. The pinned release includes the full parse task and follow-up multi-page PDF fix. Record the versions and validate the workflow in your own environment.

Documentation links

• Xinference 3.2.0 release notes
• Merged DeepDoc PDF parsing API implementation and request contract
• Official Xinference installation guide
• Using Xinference, listing and terminating models
• Official Docker image and GPU requirements
• Xinference repository and Apache-2.0 licence
• curl manual: form fields, file input and HTTP status output

Bottom line: Xinference 3.2 makes DeepDoc’s structured PDF parsing available through one local API call, but the path is new and OCR can be confidently wrong. Keep the endpoint on loopback, use the exact task=parse multipart contract, review every critical field and preserve the source document.