# EmotiEffLib: facial emotion and engagement recognition in Python and C++ > A small Apache-2.0 library, formerly HSEmotion, that ships ONNX and PyTorch backends for emotion and engagement recognition in photos and video, with matching notebooks for both language bindings. **sb-ai-lab/EmotiEffLib** — Efficient face emotion recognition in photos and videos - Repository: https://github.com/sb-ai-lab/EmotiEffLib - Stars: 1,065 · Forks: 156 - Language: Jupyter Notebook - License: Apache-2.0 - Published: 2026-10-07 · Updated: 2026-10-07 · Language: en - Canonical page: https://hysenlabs.com/projects/sb-ai-lab-emotiefflib ## Five runtime dependencies and two optional extras The dependency story is short enough to state in full, which is rare and welcome. `requirements.txt` contains numpy, onnx, onnxruntime, opencv-python and pillow. That is the whole runtime set for the default path, and it means an ONNX-only install does not drag a deep learning framework into your environment. PyTorch is an optional extra rather than a requirement. The `pyproject.toml` wires extras to separate files: `requirements-torch.txt` for the PyTorch backend, `requirements-engagement.txt` for engagement detection, and an `all` extra that combines the two. The package metadata records version 1.1.1 under Apache-2.0, with `license-files` set to the glob `LICEN[CS]E.*`, so a LICENSE or LICENCE variant is both accepted. The package is also described as flexible across backends, supporting both PyTorch and ONNX so analysis can run efficiently on different platforms. That is the library's central design claim, and the dependency split is the concrete form of it: pay for a framework only when you want one. Two names appear in the download badges, `hsemotion` and `hsemotion-onnx`, both separate PyPI entries from the same project. The rename history matters if you are porting existing code, since the API you remember under HSEmotion now ships as `emotiefflib`. ## One repository, two implementations, six notebooks The repository holds two implementations: a Python package in `emotiefflib/` and a C++ package in `emotieffcpplib/`. Each has its own README with build and install instructions, so the top-level README deliberately does not carry a single install command. That is a small sign of care, since the two toolchains have almost nothing in common beyond the models. The quick start is built from notebooks rather than code snippets, and there are six of them: one image emotion recognition, predict emotions on video, and predict engagement and emotions on video, each in a Python and a C++ version. The Python notebooks open in Colab through one-click badges, while the C++ notebooks open in MyBinder. That split has a practical consequence. The Python path lets you evaluate the library in a browser in about a minute without installing anything locally. The C++ path is bound differently, because MyBinder environments do not ship a native toolchain, so running it locally needs the build instructions in `emotieffcpplib/README.md`. For the C++ binding, models are not fetched at runtime. They are prepared first: ```bash python models/prepare_models_for_emotieffcpplib.py ``` There is also a `training_and_examples/` directory holding usage examples, the training process, and a mobile application example for recognising a user's emotions, which is the most concrete signal of the intended deployment target. ## The timm version constraint splits old and new models One warning in the README deserves more attention than it gets. The models were updated to work with `timm` version 0.9 and later, but for the v0.1 line the EfficientNet models for PyTorch are based on the old `timm` 0.4.5 package, and that exact version has to be installed. ```bash pip install timm==0.4.5 ``` This is the kind of constraint that produces confusing failures rather than a clear error, because a mismatched `timm` will load an EfficientNet architecture that no longer matches the checkpoint you are pointing at it. If you are starting fresh, use a current `timm`. If you are reproducing an older published result, pin 0.4.5. The three dataset preparation notebooks sit in the same area: `train_emotions.ipynb` for AffectNet, `AFEW_train.ipynb` and `VGAF_train.ipynb`. They exist because the README asks you to prepare datasets through these TensorFlow notebooks before running the code on them, which is a hint that the training path historically leaned on TensorFlow even though the inference backends are PyTorch and ONNX. The project also documents where to find full documentation, a dedicated GitHub Pages site for the library, and the per-library tutorials directory for examples of both modules. ## A competition record that is documented rather than asserted The README's news section is a list of competition results, and it is specific enough to be checkable. The HSEmotion team took first place in Fine-Grained Violence Detection and runner-up in both Expression Recognition and Action Unit Detection at the 10th Affective Behavior Analysis in-the-wild competition. At the 8th ABAW competition the team took first places in Expression Recognition and Ambivalence or Hesitancy Recognition, and third places in Action Unit Detection and Emotional Mimicry Intensity Estimation. At the 6th ABAW competition the team took second place in Compound Expression Recognition and third in Action Unit Detection. There is also a Papers With Code badge for classifying emotions and engagement, and the models repository gained a MyBinder-supported release path in v1.1.1 alongside updated tutorials and new models. The academic entry is the paper on facial expression recognition with adaptive frame rate based on multiple testing correction, accepted as an oral talk at ICML 2023. The README points at reproducible notebooks in the repository, specifically the adaptive frame rate subsections of `abaw3_train.ipynb` and a companion training notebook. Having the reproduction path checked in alongside the claim is the detail that makes the rest of the list more credible. The last commit landed on 2026-06-23, so this is a project still receiving work, and the release history is short: v0.2.1 in December 2022 added the `hsemotion_onnx` package, v1.0 in February 2025 created the Python package and the C++ implementation and added notebooks for personalised models, and v1.1.1 in September 2025 added MyBinder support and new models. ## Engagement recognition is the feature most libraries skip Most emotion recognition libraries return one label per face. EmotiEffLib's notebooks include a dedicated path for engagement and emotions on video, and engagement sits behind its own optional dependency file, which tells you it is a separate capability rather than a flag on the emotion model. Engagement is the harder half of the problem. Deciding that a face is happy is a classification task with a discrete answer. Deciding that a person is engaged with what they are looking at is a judgement about attention that a model has to infer from a sequence of frames, and it is the part most likely to break when you move from a curated video dataset to a webcam. The video notebooks exist for exactly this reason, since per-frame image prediction and sequence-level inference behave differently enough that a library which only shipped the image path would leave the interesting half undocumented. One structural note for anyone who has worked with OpenCV pipelines: the `pyproject.toml` tells pylint to treat `cv2.*` as dynamically generated members, because OpenCV attributes are added at runtime and would otherwise trigger undefined-member errors. Small detail, but it tells you the project has been linted against real OpenCV usage rather than a toy example. ## Conclusion EmotiEffLib earns its place by being small and honest about what it wraps. The runtime dependency list is five lines long, the PyTorch path is optional rather than mandatory, and the same models are reachable from Python and from C++, which is the combination you want when a desktop application needs inference but a notebook needs to iterate. The competitive record is documented rather than asserted, with placements across six, eight and ten editions of the ABAW challenges and an ICML 2023 paper on adaptive frame rate with reproducible notebooks in the tree. The two things to check before committing are the `timm` version constraint, which differs between the current models and the v0.1 line, and the fact that the C++ binding expects prepared model files rather than downloading them. Start from the one-image notebook for the Python path and `prepare_models_for_emotieffcpplib.py` for the C++ one. ## FAQ ### Can OpenCV be used for emotion recognition? Yes. opencv-python is one of five runtime dependencies alongside numpy, onnx, onnxruntime and pillow, so video decoding and face handling are handled in the same install. The pyproject even configures pylint to treat `cv2.*` as dynamically generated members. ### Can AI recognize facial expressions? Yes, and this library exists for that. EmotiEffLib is a lightweight library for emotion and engagement recognition in photos and videos, usable from Python and C++, with ONNX and PyTorch backends. ### What is the relationship between EmotiEffLib and HSEmotion? They are the same project under two names. The README calls it EmotiEffLib ex-HSEmotion, and the older `hsemotion` and `hsemotion-onnx` packages remain on PyPI alongside the current `emotiefflib` package. ### Which timm version do I need? Use timm 0.9 or later for the current models. Only if you are reproducing the v0.1 line do you need to pin `pip install timm==0.4.5`, because those EfficientNet PyTorch models are based on the older package. ### How do I run the C++ version? Prepare the models first with `python models/prepare_models_for_emotieffcpplib.py`, then follow the build instructions in the C++ package README. The C++ notebooks run on MyBinder, but a native toolchain is needed to build locally. ## Sources - [Issues](https://github.com/sb-ai-lab/EmotiEffLib/issues) - [License: Apache-2.0](https://github.com/sb-ai-lab/EmotiEffLib/blob/main/LICENSE) - [README](https://github.com/sb-ai-lab/EmotiEffLib/blob/main/README.md) - [Releases](https://github.com/sb-ai-lab/EmotiEffLib/releases) - [sb-ai-lab/EmotiEffLib on GitHub](https://github.com/sb-ai-lab/EmotiEffLib) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/sb-ai-lab-emotiefflib