# Hulu-Med: an open medical vision-language model across text, 2D, 3D and video

> Hulu-Med is a family of openly released medical vision-language models from Zhejiang University, covering text, 2D images, 3D volumes and surgical video, with training code, weights and an evaluation harness all published. It suits medical AI researchers who want a starting point they can inspect and retrain.

**ZJUI-AI4H/Hulu-Med** — A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding

- Repository: https://github.com/ZJUI-AI4H/Hulu-Med
- Stars: 1,031 · Forks: 35
- Language: Python
- License: Apache-2.0
- Published: 2026-10-20 · Updated: 2026-10-20 · Language: en
- Canonical page: https://hysenlabs.com/projects/zjui-ai4h-hulu-med

## Transparency as the product, not a feature

The framing of the project is unusual and it is worth reading the word carefully. Hulu-Med is described as a transparent medical vision-language model that unifies understanding across diverse modalities including medical text, two and three dimensional images and videos, and the transparency claim is spelled out in the key features as a complete open-source pipeline including data curation, training code and model weights. The stated motivation is a focus on transparency and accessibility, and the data claim is that the model was trained entirely on public data. The performance claim is that it reaches state-of-the-art performance on 30 medical benchmarks while competing with proprietary systems, and the feature list words it as outperforming leading open source models. Transparency here means you can read how the corpus was assembled and rerun the training, which is a stronger property than releasing weights alone, and it is the reason a research group would pick this over a model with better benchmark numbers and no published pipeline.

## The corpus, and what the modalities cover

The training data description is the most concrete part of the documentation and it is worth taking in the numbers. The corpus spans 16.7 million samples across 12 anatomical systems and 14 imaging modalities, and the systems are listed individually, covering multi-system, skin and integumentary, respiratory, cellular and tissue level, digestive, nervous, cardiovascular, musculoskeletal, reproductive, urinary, whole body, endocrine, immune and lymphatic, and hematologic. The 14 modalities are CT, MRI, X-ray, ultrasound, PET, OCT, endoscopy, microscopy, histopathology, fundus, dermoscopy, angiography, digital photograph and medical chart, so the coverage of imaging is broad rather than centred on radiology. The downstream tasks listed include medical dialogue, anomaly detection, prognosis prediction, treatment planning, surgical skill assessment, education, medical report generation, surgical phase recognition and medical computation. Two of those are unusual for a general model, surgical skill assessment and surgical phase recognition, and they follow from the video coverage, since those are questions you cannot answer from a still image. The stated training cost is modest by the standards of frontier models, with only 4,000 to 40,000 GPU hours required for the 7B to 32B variants.

## Six published sizes, and the base model matters more than the parameter count

The model zoo is a table of six variants and the parameter counts are not evenly spread, which is the interesting part. Hulu-Med-4B has 4B parameters, is based on Qwen3-VL-4B and costs about 1,000 GPU hours. Hulu-Med-7B is based on Qwen2.5-7B at about 4,000 GPU hours. Hulu-Med-14B is based on Qwen3-14B at about 8,000 GPU hours. Hulu-Med-30A3 is a 30B model on the Qwen3-VL-30A3B base at about 3,200 GPU hours, which is notably cheaper than the 14B, so the training cost does not track the size. Hulu-Med-32B is based on Qwen2.5-32B at roughly 40,000 GPU hours, the highest figure in the table. And Hulu-Med-235A22 is a 235B model on the Qwen3-VL-235A22B base at about 10,000 GPU hours, the largest variant by a wide margin and among the cheapest per parameter. A late addition of Hulu-Med-Flash-Preview-27B is described as having shorter chain of thought reasoning for faster throughput, with a reported HealthBench score of 64.5 and 43.7 on the hard set. All the variants are published on both the Hugging Face and ModelScope model hubs, and a Gradio demo is available.

## Three inference paths, and which one you want

The repository layout shows three separate notebooks, and they reflect three different ways of running the model. There is a notebook for the vLLM route, one for the general inference path and one specific to Qwen3, plus a MedUniEval directory holding the evaluation harness and a source directory. The vLLM integration came later, in November 2025, with the announcement that the model is now compatible with the latest vLLM for faster inference and tensor parallel support. The Transformers integration came a few days earlier, with Hugging Face compatible models released for simplified loading and inference, and the notice that the models on the main branch can be loaded directly with the standard causal language model class, with the weights downloaded automatically. For users in regions with limited access, there is a documented mirror environment variable to set:

```bash
export HF_ENDPOINT=https://hf-mirror.com
```

Choosing between them is a practical decision. The Transformers path is the one to start with, because it is the least machinery and it works on a single machine, while the vLLM path is the one to move to when you need the throughput, and the Qwen3 notebook is presumably there because the chain of thought handling differs for that base model family. The requirements file gives a strong hint about the target environment, since it pins PyTorch built against CUDA 11.8 wheels and a comment saying it matches the README install step.

## MedUniEval, and why an evaluation harness matters more than a score

The evaluation code was released separately in November 2025 as MedUniEval, built on MedEvalKit, and it is described as designed for the comprehensive evaluation of medical vision-language models across various modalities, including text, two and three dimensional data and video. The claim is that more benchmarks are coming, and some processed evaluation data are available through the ModelScope model repository. Shipping an evaluation harness alongside a model is a deliberate choice and it is the one that tells you the most about how a group works. A model with a benchmark number tells you what the authors measured; a model with a harness tells you how they measured it and lets you measure something else the same way, which is what a research group actually needs when it wants to compare its own fine-tune. The multimodal scope is the interesting part, because a benchmark that covers text, 2D, 3D and video in one framework is unusual, and the fact that the model is trained on public data and evaluated with a public harness means the whole chain is checkable. The project also links a paper on arXiv and a citation file is present in the repository, so the work is expected to be referenced.

## What the model is not, and what to read before you use it

Two things are worth being clear about before you plan an evaluation. First, the parameter scale sets a hard floor on deployment. The smallest published variant is 4B parameters, so even the low end is a serious piece of hardware, and the 235B variant is a cluster job. Second, the licence story has two layers, and the repository licence is only one of them. The code and the repository are Apache-2.0, but the weights are hosted as separate artefacts on Hugging Face and ModelScope under their own terms, and the training data, while described as entirely public, comes from sources with their own licences that a commercial deployment would have to satisfy. The repository is not archived and the last push was on 2026-08-30, so the code is being worked on, and the news section shows a steady release cadence through 2026 with the Flash preview and the 30A3 and 235A22 variants arriving in February and April. There are no GitHub releases, so the model hubs and the paper are where versions are tracked, and the demo is described as open.

## Conclusion

Hulu-Med is a good fit for a medical AI group that wants a multimodal baseline they can actually audit, because the pipeline is described as fully open with data curation, training code and weights, and because the model covers modalities that most open medical models do not, including 3D volumes and surgical video. It is the wrong choice if you need a small model you can run on a laptop, since the smallest published variant is 4B parameters, and it is the wrong choice if you need a model with a permissive licence for commercial use, since the repository is Apache-2.0 but the weights and the training data come with their own terms that you should read. Before using it, read the model zoo rather than the headline, because the variants differ sharply in base model, parameter count and training cost, and check the inference path you need, since there are separate notebooks for the vLLM route, the Transformers route and a Qwen3-specific one. The project is Apache-2.0 licensed, the last push was on 2026-08-30, and the repository publishes no GitHub releases.

## FAQ

### What is Hulu-Med and what modalities does it cover?

It is a transparent medical vision-language model that unifies understanding across medical text, two and three dimensional images and videos. The key features describe it as processing medical text, 2D images, 3D volumes and surgical videos, and as a fully open pipeline with data curation, training code and model weights.

### What sizes of Hulu-Med are published?

Six variants, from Hulu-Med-4B on a Qwen3-VL-4B base through the 7B, 14B, 30A3 and 32B models to Hulu-Med-235A22, plus a Flash-Preview-27B variant with shorter chain of thought reasoning. The training cost ranges from about 1,000 GPU hours for the 4B model to about 40,000 for the 32B one.

### How do I run inference with Hulu-Med?

The repository ships three notebooks, one for the vLLM route, one for general inference and one for Qwen3. The Transformers integration loads the weights directly with the standard causal language model class and downloads them automatically, and users in regions with limited access can set the Hugging Face mirror endpoint environment variable to hf-mirror.com.

### What is MedUniEval in the Hulu-Med project?

It is the evaluation code released alongside the model, built on MedEvalKit, designed for comprehensive evaluation of medical vision-language models across modalities including text, 2D, 3D and video. The documentation says more benchmarks are coming and some processed evaluation data are available through the ModelScope model repository.

### How much medical data was Hulu-Med trained on?

The corpus spans 16.7 million samples across 12 anatomical systems and 14 imaging modalities, with the modalities including CT, MRI, X-ray, ultrasound, PET, OCT, endoscopy, microscopy, histopathology, fundus, dermoscopy, angiography, digital photograph and medical chart, and the model was trained entirely on public data. The project is Apache-2.0 licensed.

## Sources

- [Issues](https://github.com/ZJUI-AI4H/Hulu-Med/issues)
- [License: Apache-2.0](https://github.com/ZJUI-AI4H/Hulu-Med/blob/main/LICENSE)
- [README](https://github.com/ZJUI-AI4H/Hulu-Med/blob/main/README.md)
- [ZJUI-AI4H/Hulu-Med on GitHub](https://github.com/ZJUI-AI4H/Hulu-Med)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/zjui-ai4h-hulu-med