CUHK-X's data lives behind a DUA, so the repo ships only baselines

Seven aligned streams, and none of them in the repository

The dataset's organising idea is stated in one line: one recording, seven time-aligned streams. The streams are colour video, infrared, thermal, depth, millimetre-wave radar, skeleton and inertial measurement. That combination is unusual because most human activity recognition data is single-modality video, and the argument in the readme is that existing datasets leave a gap precisely where synchronised sensors with detailed annotations for reasoning tasks would help. The scale is given as 64,267 samples across those seven modalities. But the repository is not the dataset. The top-level listing contains a small-model baseline directory, a large-model benchmark directory, a documentation directory and a build script directory, plus a changelog, a citation file and a licence. The dataset files are distributed separately under a data use agreement, and access has to be requested on a portal. So the clone gives you the evaluation code and nothing to evaluate until the agreement is signed.

Two tracks with opposite constraints

The challenge this dataset anchors has two tracks, and reading them side by side is the fastest way to understand what the benchmark is measuring. The small-model track is 40-class cross-subject recognition across depth, inertial, radar, skeleton, infrared and thermal streams, and its constraints are severe: classic convolutional, recurrent or transformer architectures only, a size limit of 100 megabytes, and no large pretrained backbones, no closed APIs and no language models. The large-model track is visual question answering for action understanding and for reasoning, across the same non-RGB modalities, with no parameter limit and explicit permission to use large vision-language models, closed APIs and language-model pseudo-labelling. Both tracks are built on this dataset, and the baselines in this repository are named as the reference implementations for each. RGB is excluded from the challenge entirely, which is why the modality lists in the track table omit the colour stream the dataset itself contains.

The cross-subject split is eighteen users against four

Here is the number to sit with before designing an experiment. Both tracks train on users 1 through 9 and users 16 through 24, and are scored cross-subject on Kaggle test users 10, 11, 25 and 26. That is eighteen training subjects against four test subjects, and the numbering implies the recording campaign covered at least twenty-six users with blocks held out in the middle and at the end. Four test subjects is a small sample for a claim about generalisation across people, and the readme does not say how many samples those four users contribute, so the effective test size in seconds rather than subjects is unknown from the public documentation. Finalists are additionally evaluated on organiser-held private data, which addresses the size concern for the competition but not for anyone reusing the split. If you are training on this data for a paper, the honest framing is that you have four held-out people and should report results per subject rather than pooled.

Only part of a finalist's score comes from accuracy

The scoring breakdown is published, and it is not a leaderboard. Accuracy counts for half: the Kaggle private leaderboard contributes twenty percent and organiser-held private-data evaluation thirty percent. The other half is spread across five judged criteria, namely reproducibility at ten percent, the technical report at twenty percent, the presentation at ten percent and model efficiency at ten percent. So a solution that scores worse on the private data can still win, and a solution that wins can lose on reproducibility. The format amplifies that: each team gives a five-minute talk followed by five minutes of questions, the two tracks run on separate half-day slots at the conference venue in Shanghai, remote teams participate by video call, and the finalists are required to open-source their solutions under a permissive licence within thirty days of the finals. That last obligation is the most useful commitment in the readme, because it means the winning code becomes reference material rather than a leaderboard entry that vanishes.

The size limit is a deployment constraint, not a benchmark one

Isolate the small-model track's constraints and they describe an embedded device rather than a competition. A hundred-megabyte ceiling on the model, no pretrained backbone of any size, no closed API and no language model. In practice that rules out the route almost all modern human activity recognition work takes, which is to fine-tune a large pretrained vision backbone and accept the hundred-million-parameter-plus footprint. What remains is training from scratch on the modality of your choice, which is why the small-model baselines cover the modalities where a compact model is plausible: inertial, radar, skeleton, and colour video for the dataset work outside the challenge. It also explains why the challenge bills itself as the first international human activity recognition competition with no RGB, since excluding the most expensive modality is what makes a from-scratch budget realistic in the first place. If you are benchmarking, the small-model track measures a different question from the large-model track and the two leaderboards are not comparable.

Four vision-language backbones and five task types

The large-model benchmark directory is described as the code for a named set of tasks covering action understanding and action reasoning. Five are listed. Action selection is the closest to conventional recognition. Captioning asks for a description. Emotion analysis asks for affect. Sequential reordering is the synthetic one: given frames out of order, recover the sequence. And next-action prediction asks what happens next. That last two are the reasoning half of the benchmark and they are the ones that require the dataset to carry annotation effort, since a clip has to be annotated with a correct ordering and a correct continuation for either question to have an answer. Four model families are wired in: a Qwen-family vision-language model, InternVL, Video-LLaVA and Video-Chat. Having four rather than one matters for a benchmark, because it turns model choice into a variable you can measure instead of a decision you have to defend.

The browser explorer shows the label set without an agreement

The most underused asset in this repository is not the baselines, it is the explorer. The readme describes it as streaming real clips in the browser with synchronised playback across depth, infrared and thermal, which is the only way to see whether those three streams are genuinely time-aligned or merely time-stamped. It also carries a forty-action atlas organised by the seven categories the paper uses, which means you can read the label taxonomy in full before you sign anything, and a playable next-action reasoning quiz built on real ground truth, which lets you try the reasoning task yourself with no data access at all. That is a genuinely open way to evaluate whether this benchmark is the right shape for your work. The documentation directory also holds the project site with separate pages for the dataset description, benchmarks and results, and publications, plus a script directory whose stated job is building the explorer's assets and the repository banner.

One tagged release and an unidentified licence

Two facts about the repository as an artefact. It has exactly one tagged release, version one point zero, published on 2026-09-07, described as the first tagged release of the baseline code, the project website and the explorer; the repository was pushed to again on 2026-09-20. A citation file was added in the same release so that the hosting platform offers a cite-this-repository button, which is a courtesy more data and benchmark repositories should copy. The licence is the murkier point. The repository metadata reports no licence it can identify, while a licence file sits at the root of the tree, and the challenge requires finalist solutions to be released under a permissive licence rather than the one the file happens to contain. If you are building on the baselines, confirm the terms from the licence file itself rather than from the metadata, and if you intend to enter the challenge, follow the challenge's licence requirement rather than assuming the repository's file already satisfies it.

Editorial conclusion

CUHK-X fits a perception or multimodal researcher who needs depth, infrared, thermal, radar, skeleton and inertial streams aligned in time and who can get institutional access to the portal. It does not fit someone who wants to start without a data use agreement, because nothing but baselines ships here, and it does not fit someone looking for a large public training set, since the published cross-subject split puts eighteen users in training against four in test. Before you commit, read the licence situation carefully, as the repository metadata reports no recognised licence while a licence file sits at the root, decide which track's constraints you can actually satisfy, and use the browser explorer first, since it exposes the label taxonomy and a sample of the data without any agreement at all.

Frequently asked questions

Official sources

1. Issues
2. openaiotlab/CUHK-X on GitHub
3. README
4. Releases