Model or dataset
Tencent/WeMM-Embedding avatar
Tencent/WeMM-Embedding

WeMM-Embedding: transformers pinned to 5.2.0, torch left floating, audio scored zero

WeMM-Embedding is a family of universal multimodal embedding models by the WeChat Vision Team at Tencent, supporting multimodal understanding and retrieval.

1,819 stars124 forksPythonNOASSERTION

At a glance

What is it?
WeMM-Embedding is a family of open weight multimodal embedding models from Tencent's WeChat Vision team, with three sizes, Matryoshka dimension ladders and serving paths for vLLM and SGLang. Read the tables carefully: the model does not support audio, the average column mixes two metrics, and the evaluation harness is a vendored copy of someone else's pipeline.
Who is it for?
WeMM-Embedding is a credible open release for text, image, video and visual document retrieval, and the honest details are in the repository: pinned inference stack, a stated MRL recipe, and a serving path that patches an installed package. Two numbers deserve skepticism before you quote them.
Can I use it commercially?
Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
Is it still maintained?
Yes. The repository last received commits 17 days ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 15, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • Tencent/WeMM-Embedding1,819 ★ · +219
Star history
Repository2026-09-162026-11-15
Tencent/WeMM-Embedding1.6k1.8k

219 Stars in the last 60 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

transformers is pinned to 5.2.0 and torch is not pinned at all

The whole installation is one command, `pip install -r requirements.txt`, and that file is five lines long:

code
torch
transformers==5.2.0
qwen-vl-utils[decord]==0.0.14
sentence-transformers==5.7.0
accelerate>=1.1.0

The README explains the exact transformers pin with a warning that newer versions may differ in preprocessing behavior, which is an admission that the output depends on the library version. Yet `torch` sits on the first line with no constraint at all, and `accelerate` is a floor rather than a pin. So the dependency that actually runs the model is the one left open. Two other entries are worth noting: the video path pulls a decoder through the `decord` extra on `qwen-vl-utils`, pinned to a 0.0.x release, and the model family from Tencent depends on a Qwen utility package for preprocessing. Neither the requirements file nor the README mentions model weights, which are not in this repository at all; they are separate repositories on Hugging Face, one per size.

Audio is unsupported, and eleven audio tasks still count toward the total

The model zoo section states plainly that audio input is not currently supported. The MMEB-v3 table then reports an Audio column, and every WeMM-Embedding row in it reads 0.0. The reason is written down as a rule rather than a footnote: V3-All includes the 78 MMEB-v2 tasks, 53 text tasks, 47 agent tasks, 11 audio tasks and MCMR, and unsupported tasks are assigned a score of zero. So the aggregate figure that leads the table, 59.3 for the 9B model, is computed over a task set where a fifth of the categories contribute nothing for this model. The comparison rows make it concrete. Qwen3-VL-Embedding at 8B scores 53.5 with 0.0 on audio, while Omni-Embed-Nemotron at 3B reaches 43.5 with 36.5 on audio, E5-Omni posts 30.8 and 43.0 on audio across its two sizes, and Tianmu-Emb-Uni at 8B reaches 38.9. Read the Audio column before quoting the average.

The average column averages two different metrics

Neither benchmark table reports a single measurement. On MMEB-v2 the README states that image and video tasks use Hit@1 while visual-document tasks use NDCG@5, and both are folded into the AVG column. On MMEB-v3 it is the other way around: text results use NDCG@5, while agent, MCMR and audio results use Hit@1, and those four columns plus text produce V3-All. Hit@1 is a top-1 retrieval hit rate and NDCG@5 is a rank-discounted score over five results, so the arithmetic in the AVG column mixes two scales. That does not make the comparison meaningless, but it does mean a gap of a few points between two rows is not a statement about a single metric. The same caveat applies to the ranking claims: the 2B model at 256 dimensions is said to retain 98.7% of its full-dimensional image and video performance, which is an image and video statement and says nothing about visual documents or the audio column.

Extraction depends on a dedicated token and the right chat template

How an embedding is pulled out of the model is two sentences, and both are load bearing. Embeddings are obtained from the last-layer hidden state at the dedicated `<embedding>` token position, followed by L2 normalization. That means the token has to be in the input, the position has to survive preprocessing, and normalization happens once at the end. The serving command for vLLM passes the template that guarantees it, `--chat-template "$MODEL_PATH/embedding_chat_template.jinja"`, so the template file is expected to sit inside the model directory you point at. If that file is missing or the wrong one, nothing in the pipeline raises an error about pooling; you get a different vector. Matryoshka truncation is then a slice and a second normalization:

python
embedding = torch.nn.functional.normalize(embedding[..., :d], dim=-1)

The examples reach the same code through two libraries. The transformers example takes a local path, and the SentenceTransformer example also accepts a Hugging Face model id because `SentenceTransformer` loads the model directly. Omitting `--dimension` returns the full embedding.

Serving on SGLang starts by patching the installed package

Two serving stacks are documented, and both come with a version attached: vLLM 0.27.0 and SGLang 0.5.9 are the tested versions. The vLLM path is a single `vllm serve` with the pooling runner and the chat template. The SGLang path is different in kind. Before the server starts you run `python scripts/patch_sglang_video.py`, then launch with `--is-embedding` and `--enable-precise-embedding-interpolation`. So the SGLang route is not configuration, it is a script that modifies the installed library to handle video, which means it has to be re-applied whenever the environment is rebuilt and it is tied to the exact SGLang release it was written against. One-command wrappers exist for both, in `scripts/serve_vllm.sh` and `scripts/serve_sglang.sh`. Neither serving path mentions batching, hardware or concurrency settings, so those choices are left to the operator.

The three dimension ladders do not share a top dimension

Each model advertises its own Matryoshka ladder, and the top of each ladder is different. The 2B model lists 64, 128, 256, 512, 1024 and 2048. The 4B model lists the same prefix and then 2560, not 2048. The 9B model lists 64 through 2048 and then 4096. So the only dimension every model supports is the shared prefix, and the maximum is per model, which means a hardcoded dimension in a pipeline will not port between sizes. The examples in the file all pass `--dimension 2048` against the 2B model, which is that model's maximum, not a shared one. The performance claim for truncation is also given for one model and one dimension only: the 2B at 256 dimensions on MMEB-v2. Nothing in the table states what the 4B retains at 256, or what the 9B retains at 4096, so anyone planning a storage or index-size budget has one data point to reason from.

Reproducing the table means running a vendored copy of another pipeline

The evaluation code lives in `mmeb_v3_eval/` and the README is explicit about what it is: the official TIGER-AI-Lab VLM2Vec pipeline with a minimal diff. The listed differences are multi-node multi-GPU inference via `torchrun --nnodes=N`, a `wemm_embedding` backbone implementing the project's own preprocessing and batched inference, dataset instructions aligned with the released model, and 64-frame video sampling. Running it looks like this:

bash
cd mmeb_v3_eval
DATA_ROOT=/path/to/MMEB-V3 bash scripts/download_data.sh
MODEL_PATH=/path/to/WeMM-Embedding-2B DATA_BASEDIR=/path/to/MMEB-V3 \
OUTPUT_DIR=exps/wemm_embedding bash scripts/run_eval.sh

The dataset lives outside the repository and is fetched by a script. Note also the entry further up the page: PeerQA and DeepPlanning scores and the aggregates that depend on them were updated on 2026-09-16 using task-wide global retrieval, with a link to issue number nine. A repository that records a rescoring date and the methodological change behind it is being straight with you about what moved, and it means the numbers in the table are not the numbers from the first release of the evaluation.

No releases, no license in the metadata, and no training code here

The repository is small and inference shaped. At the top level there is the README, a Chinese version, a LICENSE file, `assets/`, `docs/`, two example scripts, `mmeb_v3_eval/`, `requirements.txt` and `scripts/`. There is no training code, no model definition and no weights, because the model is a set of Hugging Face repositories under the tencent organisation, linked from the model zoo table. The repository has no GitHub releases, so the version of anything you get is whatever the branches and the model repositories currently hold, and the last push to main is dated 2026-09-20. One inconsistency is worth flagging rather than resolving: the license field on the repository is empty, while a LICENSE file is present in the root of the tree and linked from the README, which does not name a license in its visible text. Check the file itself before you build a compliance story on it.

Editorial conclusion

WeMM-Embedding is a credible open release for text, image, video and visual document retrieval, and the honest details are in the repository: pinned inference stack, a stated MRL recipe, and a serving path that patches an installed package. Two numbers deserve skepticism before you quote them. The headline averages include eleven audio tasks the model does not support, scored zero by rule, and the average column mixes Hit@1 with NDCG@5, so the total is not a single metric. Before deploying, pin torch yourself, because the manifest leaves it unpinned while insisting that preprocessing behavior depends on the transformers version, and check that your model directory contains the `embedding_chat_template.jinja` file the vLLM command depends on.

Frequently asked questions

Does WeMM-Embedding support audio?

No. The model zoo section states that audio input is not currently supported, and on MMEB-v3 unsupported tasks are assigned a score of zero, so every WeMM-Embedding row in the Audio column reads 0.0 while models such as Omni-Embed-Nemotron and E5-Omni post real audio scores.

Which Transformers version does WeMM-Embedding expect?

The README recommends `transformers==5.2.0` for inference and reproducibility because newer versions may differ in preprocessing behavior, and requirements.txt pins that exact version. `torch` is listed in the same file with no version constraint at all, and accelerate has a floor of 1.1.0 rather than a pin.

What are the Matryoshka dimensions for the three WeMM-Embedding sizes?

The 2B model lists 64, 128, 256, 512, 1024 and 2048. The 4B model lists the same prefix and then 2560. The 9B model lists 64 through 2048 and then 4096, so only the prefix is common to all three sizes.

How does WeMM-Embedding serve on vLLM and SGLang?

vLLM is launched with the pooling runner and a chat template read from the model directory. The SGLang path additionally requires running `python scripts/patch_sglang_video.py` first, then launching with `--is-embedding` and `--enable-precise-embedding-interpolation`. The tested versions are vLLM 0.27.0 and SGLang 0.5.9.

Where do I get the WeMM-Embedding model weights?

Not from this repository, which ships no weights and no training code. The model zoo table links one Hugging Face repository per size, under the tencent organisation, for the 2B, 4B and 9B models. Installation here is just `pip install -r requirements.txt`.

Official sources

  1. Issues
  2. README
  3. Tencent/WeMM-Embedding on GitHub
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/tencent-wemm-embedding.svg)](https://hysenlabs.com/projects/tencent-wemm-embedding)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/tencent-wemm-embedding.svg)](https://hysenlabs.com/projects/tencent-wemm-embedding)

Was this useful?