EyeTracker detects a pupil as a rotated ellipse in Python

The output is one rotated rectangle per frame

The core of the project is small and specific. It is an open source 3D eye tracking algorithm written in Python, and the unit of output is a `rotated_rect` that represents the pupil ellipse.

That is the whole contract. A frame goes in, and you get back a rotated rectangle describing the pupil. Everything a caller wants to build on top, the gaze direction, the eye position, the calibration, is derived by the caller from that rectangle. This is a detection primitive rather than a gaze estimation system, and the difference matters when you choose whether to use it.

The lineage is stated in the README. This is an updated version of the pupil tracker from the YutaItoh 3D-Eye-Tracker project, specifically its `pupilFitter.h`, and it has been optimized, simplified and improved upon. The original pupil tracker being the reference point is a useful signal about what the algorithm does, since the README also points to a video explaining the algorithm details and a second video showing what the results on the test video should look like.

The project is MIT licensed and the repository is not archived, with the last push on 2026-07-29. There are no GitHub releases, so the version history lives in commits rather than tags.

The audience is narrow and practical: someone who has a cheap infrared camera pointed at an eye and wants the pupil centre, and who wants to understand or modify the detection rather than call a hosted service.

The script is a demo, not a library

The usage instructions are worth reading carefully, because they show how the code is meant to be used and they also show its limits.

To run it, you execute the script from your shell.

python .\OrloskyPupilDetector.py

The script looks for a video at a path hardcoded in the `select_video()` function, and if it does not find one there it opens a browse window so you can pick a file. The `process_video()` function does the majority of the processing, and the README notes that it can be easily modified to work with a camera capture or with a single image.

That last sentence is the important one. The code is presented as a script with a hardcoded path and a file picker, not as a package with a documented API, an entry point and configuration. The function that does the work is the thing you are meant to import or adapt, and the documentation treats that as a straightforward edit rather than as a supported extension point.

The trade is that you get a complete, readable implementation rather than a maintained library. If you want to run this in a service, or on a stream of frames, or with your own camera handling, you are going to write that glue yourself, and the README is honest that this is the intended route.

There is also a lite version, `OrloskyPupilDetectorLite.py`, described as more efficient but less robust, with a warning to have an adequate light source for it. That is a real trade rather than a free choice, and the warning tells you which side of it you are on: the lite version gives up robustness in exchange for speed, and the robustness it gives up is exactly the part that depends on good illumination.

A test video named `eye_test.mp4` is included in the repository root, which is the piece that makes the project reproducible for anyone who clones it.

Five tracker directories and three detector variants

The repository layout shows that this is a research workspace rather than a single program.

At the top level there are three detector scripts, `OrloskyPupilDetector.py`, `OrloskyPupilDetectorLite.py` and `OrloskyPupilDetectorRaspberryPi.py`, plus the test video and the licence file. The Raspberry Pi variant exists as its own file, which tells you that running this on a small single board computer was an explicit goal rather than an accident of the code being portable.

Then there are five directories, each of which is a different tracker built on the detector. `3DTracker/` is the 3D eye tracker, `HeadTracker/` is head tracking, `FrontCameraTracker/` is a front camera setup, `Webcam3DTracker/` is the webcam based 3D variant, and `VREyeTracker/` is for a virtual reality headset.

The VR one is the interesting one, because the assumptions section talks about dark regions in the corners of the image, for example VR display lens borders, that should be cropped. That is a detail you only need if the camera is looking at an eye through a headset lens, and it tells you the VR configuration was actually built and tested rather than sketched.

Having the trackers separated from the detector is the right structure for this kind of project. The detection algorithm is the hard part and it is the part that has to be identical across the five applications, and the fact that it lives in its own file, or its own variant, means the head tracker is not shipping a copy of the pupil fitter with local modifications.

For a newcomer, the practical reading of this layout is that the detector is the product and the directories are demonstrations of what it can be combined with.

numpy 2.0 is the one dependency problem on record

The requirements are stated in two lines, and one of them carries a warning.

You need a Python environment, and the packages are numpy and opencv. The numpy entry has an explicit note that there is a known issue with numpy 2.0.0, and that downgrading to 1.26.0 or another version can solve the issue.

That is a small piece of information with a large practical effect, and it is the kind of detail that is easy to lose. A numpy 2.0.0 upgrade in the rest of a project will pull the failure into this code, and the error it produces is unlikely to point at the version boundary. The advice in the README, downgrade to 1.26.0 or another version, is the whole workaround, and it needs to be applied deliberately rather than discovered.

The other dependency, opencv, carries no such note, which suggests the image processing calls in use are on stable API ground. Given that the code is derived from a C++ implementation, the OpenCV calls are likely the older ones, and those have been stable in the library for a long time.

So the dependency picture is good apart from one known landmine. OpenCV for the image processing, numpy for the array maths, with numpy held below 2.0.0 until the incompatibility is dealt with. There is no dependency on a deep learning runtime, no model file to download and no GPU requirement, which is a substantial difference from most modern eye tracking code and is what makes the Raspberry Pi variant plausible.

The licence is MIT, so the code can be used and modified in a commercial product under the usual conditions of keeping the copyright and licence notice.

The assumptions are about resolution, framing and light

The README states its assumptions, and they are the part of the documentation that decides whether the algorithm will work on your footage at all.

It works best with 640x480 video. Images will be cropped to size equally horizontally and vertically if the aspect ratio is not 4:3, which is a slightly unusual phrasing but a sensible one, since the detector expects a particular image geometry and rather than failing on a wrong aspect ratio it trims the excess on both axes.

The image must be that of the entire eye. Dark regions in the corners of the image, for example VR display lens borders, should be cropped. So the framing requirement is strict. A camera pointed at a face with the eye in the middle of the frame is not what this expects, it wants the eye filling the frame, and any part of the frame that is not eye and is dark is a problem the caller has to remove before the algorithm sees it.

The lighting requirement follows from the same geometry. The algorithm needs to find a dark pupil inside a lighter sclera, and that contrast is what the detector keys on, which is why the project recommends an infrared camera and why the lite version is flagged as needing an adequate light source.

Together the three assumptions describe a specific optical setup, and the project is honest about it. The infrared route is the intended one. The README links instructions for building your own infrared camera with custom LEDs for under $100, and also mentions a small $17 infrared camera that works well with some modification, and it lists the parts it used, a GC0308 camera at around $20, LEDs for a Spinel at $8 and two USB extension cables at $8 to extend the camera cable.

The disclosure that the links are Amazon affiliate links is stated plainly, which is worth noting for a project whose documentation doubles as a shopping list.

Where a research script is the wrong answer

The limitations are of a particular kind, and they are the kind that come with the genre.

The first is that the output is a pupil ellipse and not a gaze estimate. Everything a user actually wants from eye tracking, where the person is looking, needs a calibration, a mapping from the ellipse to a screen position and a model of the eye. The README describes a detection algorithm and its outputs, and a team that needs gaze has to build the rest. For a research prototype that is a fine starting point, because the hard part is finding the pupil reliably, and for a product it is the beginning rather than the answer.

The second is the performance envelope. The lite variant exists precisely because the full algorithm is not fast enough for some uses, and the trade documented is speed against robustness. A deployment that needs a fixed frame rate on live video is therefore in the position of choosing which failure they would rather have, a dropped frame or a missed pupil.

The third is the input contract. Hardcoded paths, a file picker, and the expectation of 640x480 whole eye imagery with controlled infrared illumination is a workable setup for a developer with a bench and a camera on a stand, and an awkward one for a deployment where the camera moves or the light is whatever it is.

The alternative approach, and the difference in approach is real, is a modern learned gaze estimation model, which takes an image of a face and predicts screen coordinates directly, usually with a pretrained network and a calibration step. That approach is more accurate on varied lighting and does not need infrared illumination, and it usually needs a GPU and a model download, and it does not give you an interpretable pupil ellipse. The trade is between a transparent, editable, dependency-light classical detector and a black box that works in more situations but tells you less about what it saw.

Editorial conclusion

EyeTracker suits a developer building a low cost eye tracking setup with an infrared camera, who wants a pupil detector they can read, modify and run on a laptop, and who is willing to satisfy the resolution and lighting assumptions the algorithm makes. It is the wrong tool for a general purpose gaze estimation system, for head or eye pose estimation, and for anything that must run at video frame rate without tuning. Check first that you can supply a 640x480 image of the whole eye with adequate infrared lighting, and pin numpy below 2.0.0, since the project records a known incompatibility with numpy 2.0.0.

Frequently asked questions

Official sources

1. Issues
2. JEOresearch/EyeTracker on GitHub
3. License: MIT
4. README