Hysen LabsHAKE-Action
As a part of the HAKE project, includes the reproduced SOTA models and the corresponding HAKE-enhanced versions (CVPR2020).
HAKE-Action: action understanding with part states
A TensorFlow project that opens SOTA action understanding studies built on a Human Activity Knowledge Engine, with reproduced models, enhanced versions, and released data.
Project scope and people
HAKE-Action is a TensorFlow project that opens state of the art action understanding studies based on a Human Activity Knowledge Engine. It includes reproduced SOTA models and their HAKE-enhanced versions. The project is authored by Yong-Lu Li, Xinpeng Liu, Liang Xu, and Cewu Lu, and currently maintained by Li, Liu, and Xu. The README notes it is freely available for non-commercial use.
Data releases
Several data releases are documented with dates. In 2022 the interactive object bounding boxes and classes within the AVA dataset were released, with a paper link, alongside a CLIP-based human body part states recognizer. In 2021 an upgraded HAKE-Activity2Vec was released covering images and videos with human box, ID, skeleton, part states, action, and representation. HAKE-Large, a larger version with 120K images and activity and part state labels, was released in 2020.
Data modes and results
The README explains the data modes. HAKE-HICO is image-level, aggregating all part states in an image, while HAKE-HICO-DET is instance-level with part states for each annotated person. HAKE-Large contains more than 120K images with action and part state labels drawn from existing datasets and crowdsourcing. GT-HAKE uses part state labels as ground truth, which the authors call the upper bound. Activity2Vec and PaSta-R are part state based modules that run action inference on part semantics. Evaluation tables cover HICO with metrics like mAP and Few@1, HICO-DET using object detections, AVA with frame based evaluation, and V-COCO, comparing baselines with and without HAKE enhancements.
Editorial conclusion
Between data releases and evaluation tables, the README reads like a lab page: what was released, when, and how well the part state approach performs. The upper bound results make the case for the direction.
Community notes