AL_YOLO: a YOLOv5 screen-capture pipeline tuned for game footage

The problem AL_YOLO targets: screen frames, not files

Most YOLOv5 examples assume you already have images or a video file. AL_YOLO assumes the opposite: the input is whatever is currently on your monitor, and the loop has to keep up with it. The README frames the project as an exploration of real-time target detection, GPU inference, high-speed screen capture and low-latency visual processing, built during undergraduate study and aimed at game scenes because those scenes supply a continuous video stream with tight timing requirements.

The intended user is narrow. You need an NVIDIA GPU, a Windows machine, and an interest in the capture-to-inference path itself rather than in training a detector. The repository ships yolov5s.pt, a trained weights file, and points at a separate dataset repository for retraining. If your problem is detecting objects in stored footage, this project's screen-capture layer is dead weight.

Capture, preprocess, infer: the four-stage data flow

The README's architecture diagram is a straight line: DXShot captures the screen, OpenCV handles image processing, YOLOv5s runs under PyTorch and CUDA, target detection produces boxes, and a visualization and control interface draws the result. There is no queue, no batching and no separate inference thread described, so the frame rate is bounded by the sum of the stages plus the display refresh rate.

The interesting design decision is the capture backend. The README states the project started with the MSS capture that ships with YOLOv5, measured roughly 15 ms per screenshot, then moved to DXCam and finally to DXShot at roughly 5 ms. That is a deliberate trade: DXShot is a Windows Desktop Duplication wrapper, which is why the project is Windows-only, and the repository carries a compiled dxshot.pyd rather than source. You are trusting a binary that is not built from anything in the tree.

The README also says the YOLOv5 inference path was modified to remove framework layers and processing steps, and that per-frame preprocessing was tuned. Those edits are not described in detail, which matters if you plan to update the vendored YOLOv5 code later.

Installing AL_YOLO and running apex.py for the first time

The README gives four steps: create a Python environment, install requirements, configure CUDA, run. It recommends Conda and pins the interpreter to a single minor version, Python >= 3.10 and < 3.11.

conda create -n al-yolo python=3.10
conda activate al-yolo

With the environment active, install the pinned dependency set. Note that requirements.txt lists torch 2.0.1, torchvision 0.15.2 and ultralytics 8.1.45, so a plain pip install pulls those exact versions.

pip install -r requirements.txt

The README then says to configure CUDA according to your NVIDIA GPU, CUDA and PyTorch version requirements, without giving a specific command. CUDA 11 is the stated target. If your driver stack expects a different CUDA major version, resolving torch 2.0.1 against it is your problem, not the project's.

Once CUDA is in place, the entry point is a single script:

python apex.py

That is the whole documented startup procedure. The README says to consult the project code and configuration files for specific settings, and the repository has a configs/ directory and a GUI described as lightweight for parameter configuration. The README does not document the keys in those files, so expect to read the source before you change anything.

Where AL_YOLO breaks: refresh rate, the .pyd, and missing docs

The README itself flags the first limitation: because input comes from screen capture, the achievable FPS is capped by monitor refresh rate and hardware. The roughly 50 FPS figure is reported for an RTX 3050 Laptop GPU, and the README does not say what the display was running at. On a 60 Hz panel, a capture loop faster than 60 FPS gains you nothing visible.

The second limitation is the binary. dxshot.pyd sits at the top level of the repository with no build instructions. If it fails to load, the documented capture path is unavailable and the README offers no fallback beyond mentioning that MSS and DXCam were tried earlier.

The third is the mouse_driver/ directory and the lghub_installer executable at the repository root. The README lists mouse_control.py and a mouse_driver module in the project structure and mentions real-time control interfaces as a research area, but it does not document what the driver does or how it is configured. Anyone evaluating this for anything beyond observation has to read that code directly.

Finally, the README's own improvement list is honest about what is unfinished: PID smoothing, TensorRT acceleration, a C++ rewrite of inference, dataset cleaning, an installer package and support for other YOLO models are all unchecked.

How AL_YOLO differs from running Ultralytics YOLOv5 directly

The obvious alternative is the upstream YOLOv5 repository from Ultralytics, which AL_YOLO depends on and which the README names as the training tool. Upstream gives you a maintained codebase, documented training and inference flags, export paths to ONNX and TensorRT, and no opinion about where frames come from. You supply the source: a webcam, a video file, an RTSP stream, or your own capture code.

AL_YOLO inverts that. It supplies the capture layer and the visualization, and treats the model as a component you drop in. The cost is that you inherit a fixed Python 3.10 window, a pinned torch 2.0.1, a Windows-only capture binary, and a modified inference path whose changes are not written down. The benefit is that the capture-to-display loop already exists and is reported at roughly 5 ms per grab.

If your frames come from a file or a camera rather than a screen, upstream is the shorter path. If your frames come from a game window and capture latency is the thing you are trying to measure, AL_YOLO is the more relevant starting point, provided you accept reading the source as the documentation.

Licence, dependencies and the cost of staying current

The repository carries Apache-2.0, and the README defers to the LICENSE file rather than restating terms. Apache-2.0 is permissive and includes a patent grant, but the repository bundles a compiled dxshot.pyd and an lghub_installer executable whose own licensing is not stated in the README. If you redistribute the project, that is the part to check yourself; nothing here is legal advice.

Upgrade cost is the bigger practical issue. requirements.txt pins exact versions across roughly fifty packages, including torch 2.0.1, torchvision 0.15.2 and ultralytics 8.1.45. Those pins are what make the reported performance reproducible, and they are also what will make a future upgrade painful, because the README says the inference path was modified relative to stock YOLOv5. Moving to a newer ultralytics release means re-applying those modifications without a written record of what they were.

The last push to the repository was on 2026-08-21, so the project is not archived. Whether the unchecked roadmap items land is not something the README commits to.

Editorial conclusion

AL_YOLO suits engineers who already have an NVIDIA GPU on Windows and want a readable reference for a low-latency capture-to-inference loop, especially if they plan to swap in their own weights or a different capture backend. It does not suit anyone who needs a cross-platform tool, a packaged installer, a documented configuration surface, or a maintained dependency set, because the README ships none of those and the pinned requirements are already behind current releases. Before adopting it, verify three things in your own tree: that Python 3.10 with CUDA 11 and torch 2.0.1 resolves on your machine, that dxshot.pyd loads against your Windows build, and that apex.py runs without edits in configs/. If the .pyd fails to import, the capture path is the blocker, not the model.

Frequently asked questions

Official sources

1. EthanH3514/AL_Yolo on GitHub
2. Issues
3. License: Apache-2.0
4. README