# WeVisDoc: turning a page image into structured Markdown with a Qwen3-VL model

> WeVisDoc is an end-to-end document parser that takes a picture of a page and returns Markdown with LaTeX formulas and HTML tables, released by Tencent in a 2B and a 4B variant fine-tuned from Qwen3-VL-Instruct models. The project ships model weights, a vLLM serving script and a client, and the headline evaluation numbers come with a caveat about how the baselines were measured.

**Tencent/WeVisDoc** — WeVisDoc is an end-to-end document parser for page images. Fine-tuned from Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct, it turns a page into structured Markdown, with LaTeX formulas and HTML tables.

- Repository: https://github.com/Tencent/WeVisDoc
- Stars: 368 · Forks: 17
- Language: Python
- License: Apache-2.0
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/tencent-wevisdoc

## A page image in, Markdown with formulas and tables out

WeVisDoc is an end-to-end document parser for page images. The input is a picture of a page, typically a scan or a rendered document, and the output is structured Markdown in which formulas are expressed as LaTeX and tables are expressed as HTML. The point of the project is that a document image is turned into something a text pipeline can consume without a separate layout analysis stage and a separate table recogniser.

The models are fine-tuned from Qwen3-VL-Instruct checkpoints, with the repository offering two sizes, Tencent/WeVisDoc-2B and Tencent/WeVisDoc-4B on the Hugging Face hub, derived from Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct respectively. The 2B and 4B labels are parameter counts, so the choice is between a model that fits on a single consumer GPU and one that does not.

The project is published by Tencent under the Apache-2.0 licence, with the code in the repository, a project page at tencent.github.io/WeVisDoc, and a technical report on arXiv under identifier 2609.20423. The repository is not archived, the last push was on 2026-09-18, and it publishes no GitHub releases, which means the model artefacts and the serving code are versioned separately on the hub. The documentation is bilingual, with an English README and a Simplified Chinese one, and the code is Python with a wevisdoc/ package, a scripts/ directory, a demos/ directory and three requirements files.

## The evaluation tables, and the caveat attached to the baselines

The project publishes two benchmark tables, and the more useful thing about them is the note underneath. On OmniDocBench v1.6, WeVisDoc-4B reports an Overall score of 95.38 and WeVisDoc-2B reports 95.06, and the table also lists the component metrics TextEdit, FormulaCDM, TableTEDS, TableTEDS_S and ROEdit, so you can see where the score comes from rather than a single number. On PureDocBench the 4B model has a mean Overall of 75.54 across the Clean, Digital Degraded and Real Degraded tracks, and the 2B model has 73.86.

The caveat is stated explicitly. The tables include end-to-end document parsing specialists only, the WeVisDoc results are means over three inference runs, and an asterisk marks baseline results obtained with the project's own evaluation pipeline, while unmarked baseline results are taken from the corresponding papers. That is the line to read twice, because a comparison across rows mixes numbers measured under one pipeline with numbers copied from other teams' write-ups, and the difference in measurement conditions is not visible in the table itself.

The other thing the tables show is how wide the field is. The OmniDocBench list runs from Nanonets-OCR2 at 3B parameters through models of 4B, 7B and even a 0.1B model, with Overall scores from the low eighties to the mid nineties, and several entries sit within a point of each other. A lead measured against a mixed-provenance baseline is worth checking against your own pages before you build on it.

## Running the quick start: a venv, a server and a demo script

The quick start needs Python 3.10 or newer and a GPU, because the parser is served through vLLM. The dependency install is isolated in a virtual environment:

```bash
python -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements-vllm.txt
```

The service is started in one terminal:

```bash
bash scripts/serve_vllm.sh Tencent/WeVisDoc-2B
```

The model identifier is the only thing that changes between the two sizes, and the documentation says to use Tencent/WeVisDoc-4B instead to run the 4B version. With the server up, a second terminal processes the bundled images:

```bash
source .venv/bin/activate
bash scripts/run_demo.sh
```

The predictions are written to outputs/predictions/, so the demo output is easy to inspect and, more importantly, easy to diff against your own document set. The demos/ directory in the repository holds the demo material.

The three requirements files separate the concerns. requirements-vllm.txt is what the serving path needs, requirements.txt holds the client side dependencies, and requirements-local.txt sits alongside them for a local backend. The client requirement is deliberately small, openai>=1.40.0,<3, with a comment that says to install GPU dependencies for the selected backend, which tells you the client talks to the parser through an OpenAI compatible API rather than a bespoke one.

## Serving knobs: context budget, tensor parallelism and the health check

The vLLM launcher passes extra arguments through to vLLM and requires vLLM 0.11.1 or newer. The environment variables it reads are the operational surface of the project, and they are documented as a table with defaults.

WEVISDOC_MODEL_PATH and MODEL_PATH set the model ID or checkpoint when no positional argument is given, with the positional argument taking precedence. SERVED_MODEL_NAME defaults to wevisdoc and is also read by the client, which is the alias you will use in API calls. HOST and PORT default to 127.0.0.1 and 8000, so the service binds to loopback unless you change it. TENSOR_PARALLEL_SIZE defaults to 1, MAX_MODEL_LEN to 32768 as a total budget across text, image and output tokens, GPU_MEMORY_UTILIZATION to 0.9, MAX_NUM_SEQS to 8 for concurrent sequences, and OMP_NUM_THREADS to 1 for CPU preprocessing.

A two GPU configuration with a larger context and a health check looks like this:

```bash
CUDA_VISIBLE_DEVICES=0,1 TENSOR_PARALLEL_SIZE=2 MAX_MODEL_LEN=65536 \
  bash scripts/serve_vllm.sh Tencent/WeVisDoc-4B --dtype bfloat16
curl --fail http://127.0.0.1:8000/health
```

Two details are easy to miss. The context budget is shared between the image and the output, so a page image consumes part of MAX_MODEL_LEN before the model writes anything, and doubling that budget to 65536 halves the batch you can serve at the same time. And to expose the service on the network you set HOST=0.0.0.0, which matters because the default is loopback and the client, in another terminal or another host, has to reach it.

## What the repository contains beyond the serving script

The tree is small and the pieces map onto the workflow. The wevisdoc/ directory holds the Python package with the client, the scripts/ directory holds the two shell entry points used in the quick start, serve_vllm.sh and run_demo.sh, and demos/ holds the demo material. The assets/ directory carries the figure used in the project documentation.

The structure says something about the design decision the project made. Rather than shipping a training or fine-tuning pipeline, the repository publishes the fine-tuned weights and the serving and client code, so the intended deployment is a model server rather than a research codebase. The technical report on arXiv covers the training side, and the Hugging Face pages for the 2B and 4B models are where the weights live. If your requirement is to fine-tune the parser on your own document types, this repository is not that starting point; the base Qwen3-VL-Instruct checkpoints are, and the report describes how WeVisDoc itself was produced.

The client dependency list is short enough to restate, since it determines how much of your codebase has to change. The openai client at 1.40.0 or newer and below 3 is the only hard requirement in requirements.txt, which means the parser is consumed as an API endpoint with an OpenAI compatible shape, not as a library you import into a data pipeline. That keeps the model server and the extraction code decoupled, and it also means the extraction code has to be written against a client that knows nothing about page layout.

## Where a general purpose OCR model is a different kind of tool

The comparison table is a useful map of the field, and it separates models by size and by approach in a way that helps you pick a different tool when WeVisDoc is not the right one. The 2B WeVisDoc model and the 1B and 0.1B entries in both tables show what a small model can do, and several of them sit within a point or two of the WeVisDoc scores on OmniDocBench while scoring materially lower on PureDocBench, which is the benchmark that deliberately degrades the input to test tolerance of real scans.

The practical alternative to a specialised parser is a general purpose vision language model used directly, with a prompt asking for Markdown. The difference in approach is exactly what the fine-tuning removes: a general model has to infer the document structure from its instructions, while a parser trained on document images has learned the layout conventions and the LaTeX conventions of the domain. If your input is not document-like, or if you need reasoning about the content as well as its structure, a general model is the better tool, and the fine-tuned parser is the wrong one.

There is also a deployment alternative worth weighing. The project ships a vLLM path and a local requirements file, and the served model is an ordinary OpenAI compatible endpoint, so a team that already runs a model server can host it there and treat WeVisDoc as one more model. That avoids a second GPU allocation if the parsing job is small relative to the rest of your inference work.

## Apache-2.0, a bilingual README and no tagged releases

The licence is Apache-2.0, which matters for a model project because it covers the code in the repository, and the model weights on the hub carry their own terms on the Hugging Face model pages. Check the model card rather than assuming the repository licence covers the checkpoints, since a fine-tuned model derived from a base model can be subject to the base model's terms as well.

On maintenance, the repository is not archived and the last push was on 2026-09-18, so the code is being worked on. There are no GitHub releases, though, which means the code in the repository has no version tag and the model identifiers on the hub are the only stable handles. For a deployment, that implies pinning the model revision you use, since a hub repository can be updated in place.

The project also keeps two READMEs, one in English and one in Simplified Chinese, which tells you that a Chinese speaking audience is a first class one for this release. The technical report is on arXiv under identifier 2609.20423, and the project page is hosted at tencent.github.io/WeVisDoc. Together with the Apache-2.0 licence and the absence of tagged releases, that is a coherent picture: a research release with published artefacts, an evaluation and a paper, distributed for use rather than as a supported product.

## Conclusion

WeVisDoc fits a team that ingests scanned or exported page images and wants Markdown with LaTeX and HTML tables rather than plain text, and that has a GPU available to serve a vision language model. It is the wrong tool for a text-only pipeline, for a CPU only machine, and for a case where you need a licensed and versioned release, since the repository publishes no GitHub releases and the artefact you deploy is a model checkpoint from the Hugging Face hub. Before adopting it, read the evaluation tables with the baseline caveat in mind, since some comparison numbers were taken from the original papers rather than re-run under one pipeline, then check the vLLM version requirement of 0.11.1 or newer, the memory and context settings you can afford, and the Apache-2.0 terms against your own model use policy.

## FAQ

### What is WeVisDoc and what does it output from a page image?

WeVisDoc is an end-to-end document parser that turns a page image into structured Markdown, with formulas expressed as LaTeX and tables expressed as HTML. The 2B and 4B models are fine-tuned from Qwen3-VL-2B-Instruct and Qwen3-VL-4B-Instruct and are published as Tencent/WeVisDoc-2B and Tencent/WeVisDoc-4B on the Hugging Face hub.

### How do I run WeVisDoc locally?

Python 3.10 or newer is required. Create a virtual environment, install requirements-vllm.txt, then run bash scripts/serve_vllm.sh Tencent/WeVisDoc-2B to start the service and bash scripts/run_demo.sh in a second terminal to process the bundled images, with predictions written to outputs/predictions/.

### Which vLLM version does WeVisDoc need and how do I scale it to two GPUs?

The launcher requires vLLM 0.11.1 or newer and passes extra arguments through to it. Two GPUs are configured with CUDA_VISIBLE_DEVICES=0,1, TENSOR_PARALLEL_SIZE=2 and a raised MAX_MODEL_LEN, and the health endpoint can be checked with curl --fail against http://127.0.0.1:8000/health.

### What licence is WeVisDoc released under?

The repository is licensed under Apache-2.0, and the technical report is published on arXiv under identifier 2609.20423. The model checkpoints on the Hugging Face hub carry their own terms, which are stated on the model pages rather than in the repository licence.

## Sources

- [Issues](https://github.com/Tencent/WeVisDoc/issues)
- [License: Apache-2.0](https://github.com/Tencent/WeVisDoc/blob/main/LICENSE)
- [README](https://github.com/Tencent/WeVisDoc/blob/main/README.md)
- [Tencent/WeVisDoc on GitHub](https://github.com/Tencent/WeVisDoc)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/tencent-wevisdoc