Hysen Labssquirrel
An open source deep learning action recognition and segmentation framework
squirrel, a C++ framework for action recognition
An open source framework for action recognition and temporal action detection, with feed forward, CNN, and RNN support, plus the papers behind it.
The framework
Squirrel is an open source deep learning framework for action recognition and temporal action detection, with a focus on neural networks, running on Linux. So far it supports feed forward networks, CNNs, and recurrent neural networks. The scope is specific: understand what is happening in video, and when it is happening. The Linux requirement and the neural network focus are stated right up front.
Getting oriented
The README does not include a tutorial. Instead it points you to the examples, or to the manual, for how to use the framework and how to determine the correct data format. For a research oriented project that is a reasonable handoff: the code and papers carry the context, and the examples show the working entry points.
The papers behind it
The citation list is substantial. There is a paper on weakly supervised action learning with RNN based fine to coarse modeling from an IEEE computer vision conference, a bag of words equivalent recurrent neural network for action recognition, and a temporal action detection paper using a statistical language model. Earlier conference versions are cited too. The references frame Squirrel as the code behind a line of published work rather than a standalone tool.
Building it
The installation notes name the dependencies: NVidia Cuda, at least version 8.0, OpenCV in the 2.x line for reading images, and Intel MKL for parallelization. You modify definitions.make to point at the MKL and Cuda libraries, and you can comment out the Cuda, OpenCV, or OpenMP lines if you do not want them. For a different BLAS implementation, the includes in the math headers need changes too. It is a research build, not a one command install.
Credits
The acknowledgements matter here because the framework depends on external code. Special thanks go to the Human Language Technology and Pattern Recognition Group from RWTH University for permission to use their matrix and vector classes, with a citation to the RWTH neural network toolkit. That explains where the math layer came from and gives credit where credit is due, which is how a research codebase should behave.
Community notes