Library / SDK
Fannovel16/comfyui_controlnet_aux avatar
Fannovel16/comfyui_controlnet_aux

comfyui_controlnet_aux: ControlNet hint images inside ComfyUI

ComfyUI's ControlNet Auxiliary Preprocessors

4,288 stars404 forksPythonApache-2.0

At a glance

What is it?
comfyui_controlnet_aux packages the ControlNet annotators as ComfyUI nodes that turn a photo into a pose, depth, lineart or normal hint image. It is a preprocessor library, not a ControlNet model, and the README is explicit about that boundary.
Who is it for?
Adopt comfyui_controlnet_aux if you already run ComfyUI and need pose, depth, lineart or normal hint images from ordinary photos; the Apache-2.0 licence and the copy-paste provenance from lllyasviel's ControlNet annotator folder make the scope easy to audit. Do not adopt it as a ControlNet model source or as a standalone preprocessing service outside ComfyUI.
Can I use it commercially?
Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
Is it still maintained?
Yes. The repository last received commits 4 days ago.
What is it written in?
Mainly Python, according to GitHub's language statistics.

Answers come from the project's GitHub data, last synced on November 19, 2026, and from our analysis. They are not legal advice.

From daily snapshots

Star history

  • Fannovel16/comfyui_controlnet_aux4,288 ★ · +93
Star history
Repository2026-09-232026-11-19
Fannovel16/comfyui_controlnet_aux4.2k4.3k

93 Stars in the last 57 days, measured from the daily GitHub snapshot. Compare with other projects

Editorial analysis

What comfyui_controlnet_aux actually supplies

ControlNet needs a hint image: a pose skeleton, a depth map, an edge map or a normal map that constrains generation. Producing that image is a separate job from running the diffusion model, and comfyui_controlnet_aux is the ComfyUI-side implementation of that job. The README describes the project as "Plug-and-play ComfyUI node sets for making ControlNet hint images", and it states plainly that the repository only supports preprocessors that make hint images. It does not ship ControlNet weights.

The audience is narrow and specific. You need a working ComfyUI installation, a ControlNet or T2I-Adapter checkpoint downloaded separately, and a reason to extract structure from an existing image rather than describe it in a prompt. The README's own node tables map each preprocessor to the sd-webui-controlnet name it corresponds to and to the ControlNet or T2I-Adapter checkpoints it pairs with. That mapping is the most useful part of the documentation, because it tells you what to download next.

The provenance is stated without hedging: the code is copy-pasted from the annotator folders in lllyasviel/ControlNet and connected to the Hugging Face Annotators repository. All credit is assigned to lllyasviel. That matters for anyone auditing what is inside the package, since the preprocessor implementations are not original work and their behaviour tracks upstream annotators rather than this repository's own design decisions.

How the preprocessor nodes are wired

The repository layout shows node_wrappers/ and src/ sitting alongside a top-level __init__.py, which is the conventional shape for a ComfyUI custom node package: __init__.py registers the node classes, the wrappers adapt the upstream annotators to ComfyUI's node interface, and src/ holds the preprocessor code itself. The repository also has a hint_image_enchance.py at the top level, and the examples directory contains files named after individual preprocessors (example_teed.png, example_metric3d.png, example_dsine.png, example_marigold.png), which is how the project documents expected output per node.

Nodes are grouped into categories that appear as sections in the ComfyUI menu. The README names three of them: Line Extractors, Normal and Depth Estimators, and Faces and Poses Estimators. Line Extractors covers Canny Edge, HED Soft-Edge Lines, Standard and Realistic and Anime and Manga Lineart, M-LSD, PiDiNet, Scribble variants, TEED and AnyLine. Normal and Depth Estimators covers MiDaS, LeReS, Zoe, BAE, MeshGraphormer Hand Refiner, Depth Anything and Depth Anything V2, Zoe Depth Anything, Normal DSINE and Metric3D. Faces and Poses Estimators covers DWPose and OpenPose with body, hand and face detection variants.

The design decision worth flagging is the AIO Aux Preprocessor node. The README states that all preprocessors except Inpaint are integrated into it, and that it lets you get a preprocessor quickly but cannot set that preprocessor's own threshold parameters. To set thresholds you must use the dedicated node. That is a real trade-off: a single node keeps the graph compact, but it silently caps how much control you have over edge sensitivity, lineart coarseness and similar settings. If your workflow depends on tuning those values, the AIO node is the wrong entry point and you should wire the individual node instead.

Installing comfyui_controlnet_aux and running a first preprocessor

The README recommends ComfyUI Manager. Install ComfyUI Manager, then follow the steps documented there to install this repository. It is the path the project points to first.

If you cannot use Manager, there is an install.bat for Windows. The README says it installs to portable if detected, and otherwise defaults to system and assumes you followed ComfyUI's manual installation steps. On Linux, or on a non-admin Windows account, the README asks you to ensure /ComfyUI/custom_nodes and comfyui_controlnet_aux have write permissions before you start.

For a manual install, the README gives a clone plus requirements install. Navigate to your custom_nodes folder and run:

bash
git clone https://github.com/Fannovel16/comfyui_controlnet_aux/

Then navigate into the cloned comfyui_controlnet_aux folder. With a portable or venv ComfyUI, the README shows a path to the embedded interpreter rather than a bare pip call:

bash
path/to/ComfUI/python_embeded/python.exe -s -m pip install -r requirements.txt

With a system Python, the command is simply:

bash
pip install -r requirements.txt

Start ComfyUI afterwards. The dependency list is not small. requirements.txt pulls torch, torchvision, opencv-python, scikit-image, scikit-learn, mediapipe>=0.8.0, trimesh, rtree, albumentationsx, einops and omegaconf, among others, plus onnxruntime-gpu on Windows and Linux x86_64. On a CPU-only machine the onnxruntime-gpu line is a mismatch worth checking before you wait through the install.

Once ComfyUI starts, the preprocessor nodes appear under the categories listed in the README. A first real use is to load a photo, pick a preprocessor such as DWPose Estimator or Canny Edge, and preview the hint image before wiring it into a ControlNet Apply node with a matching checkpoint. The README's node tables tell you which checkpoint pairs with which preprocessor, so check that column before assuming an output is broken.

Where the preprocessor-only design breaks down

The clearest limitation is stated by the project itself: this repository only supports preprocessors that make hint images. If you want a node that applies a ControlNet model, loads ControlNet weights, or manages the sampling side of the pipeline, that is outside this package. The README expects you to supply the ControlNet or T2I-Adapter checkpoints separately, and it links out to Hugging Face for several of them.

The AIO node limitation compounds this. Because all preprocessors except Inpaint are folded into one node without exposing per-preprocessor thresholds, a user who starts with AIO and then needs finer control has to rebuild that part of the graph with the dedicated node. Nothing in the README suggests the AIO node forwards threshold settings.

A second failure mode is version drift between the preprocessor and the checkpoint. The README's pairing tables exist because a hint image produced by one annotator is not necessarily the right input for an arbitrary ControlNet model. TEED, for example, is listed against controlnet-sd-xl-1.0-softedge-dexined and only "theoretically" against control_v11p_sd15_softedge. That word is the project hedging, and it should be read as a warning rather than a guarantee.

Finally, the dependency surface is heavy for a node pack. mediapipe, trimesh, rtree, albumentationsx and onnxruntime-gpu all enter the same environment as ComfyUI. On a shared or tightly pinned ComfyUI install, that is a plausible source of conflicts.

How it compares with sd-webui-controlnet and ControlNet-Union

The most direct alternative for the same preprocessor job is sd-webui-controlnet, and the README itself provides the comparison by listing a sd-webui-controlnet/other column next to every preprocessor node. The difference is packaging and host application. sd-webui-controlnet is built for the AUTOMATIC1111 web UI; comfyui_controlnet_aux is built for ComfyUI's node graph. The preprocessor names in that column (canny, hed, lineart, lineart_anime, mlsd, scribble, depth_leres, depth_zoe, depth_anything, dw_openpose_full, openpose_full) are the same underlying annotators, so the choice is about which UI you run, not about output quality. If you are on AUTOMATIC1111, this repository is not the right tool.

ControlNet-Union is a different kind of alternative. It is a model-side consolidation that folds multiple ControlNet conditions into one checkpoint, whereas comfyui_controlnet_aux stays on the preprocessing side and produces hint images for whatever checkpoint you already have. They are not substitutes. A Union-style checkpoint changes what you load; this repository changes what you feed it.

For depth and normal estimation specifically, the README lists several estimators that overlap in purpose: MiDaS, LeReS, Zoe, Depth Anything, Depth Anything V2 and Metric3D for depth, and MiDaS Normal, BAE Normal, Normal DSINE and Metric3D Normal for normals. Choosing among them is a quality question. The examples directory includes output images for some of them, such as example_depth_anything.png, example_dsine.png and example_metric3d.png.

Maintenance, licence and upgrade cost

The repository is not archived, and the last push was on 2026-10-11. The README points to a separate UPDATES.md file for change history rather than inlining it, and the repository has no published releases, so version tracking runs through the pyproject.toml version field, which reads 1.1.6. Upgrading means pulling the repository and reinstalling requirements.txt, which re-resolves the full dependency list including torch and onnxruntime-gpu.

The licence is Apache-2.0, stated in the repository metadata and present as LICENSE.txt at the top level. One point to weigh: the README says the code is copy-pasted from lllyasviel/ControlNet's annotator folders, and that all credit and copyright goes to lllyasviel. The upstream annotator code carries its own provenance. If you redistribute or ship a product built on these preprocessors, that provenance is the thing to check with your own counsel. Nothing here is legal advice.

The practical upgrade cost is environmental rather than code-level. Because requirements.txt includes GPU-specific packages and a large scientific stack, an upgrade can perturb a working ComfyUI environment. Keeping a known-good snapshot of the environment before pulling is the concrete mitigation.

Editorial conclusion

Adopt comfyui_controlnet_aux if you already run ComfyUI and need pose, depth, lineart or normal hint images from ordinary photos; the Apache-2.0 licence and the copy-paste provenance from lllyasviel's ControlNet annotator folder make the scope easy to audit. Do not adopt it as a ControlNet model source or as a standalone preprocessing service outside ComfyUI. Before relying on it, verify that the preprocessor you plan to use has its own node, because the AIO Aux Preprocessor node does not expose per-preprocessor threshold parameters.

Frequently asked questions

How to install comfyui_controlnet_aux?

The README recommends installing ComfyUI Manager and following its steps. Otherwise, on Windows you can run install.bat, which installs to portable if detected; on Linux, clone the repository into ComfyUI/custom_nodes and run pip install -r requirements.txt inside the comfyui_controlnet_aux folder, or point the embedded Python at it for a portable install.

What is the workflow for using ComfyUI Pose ControlNet with comfyui_controlnet_aux?

Load an image, run it through a pose preprocessor such as DWPose Estimator or OpenPose Estimator to produce the hint image, then feed that hint into a ControlNet apply node with a matching checkpoint. The README's node tables list which ControlNet or T2I-Adapter checkpoint pairs with each pose preprocessor.

Does comfyui_controlnet_aux include the ControlNet models themselves?

No. The README states the repository only supports preprocessors that make hint images, and it links out to Hugging Face for the ControlNet and T2I-Adapter checkpoints you pair with them. You download those separately.

Why can't I change the threshold settings in the AIO Aux Preprocessor node in comfyui_controlnet_aux?

The README says all preprocessors except Inpaint are integrated into the AIO Aux Preprocessor node, and that this node cannot set a preprocessor's own threshold parameters. To set thresholds you need to use that preprocessor's dedicated node directly.

Is comfyui_controlnet_aux still maintained?

The repository is not archived, and the last push was on 2026-10-11. Update history is kept in a separate UPDATES.md file rather than in the README.

Official sources

  1. Fannovel16/comfyui_controlnet_aux on GitHub
  2. Issues
  3. License: Apache-2.0
  4. README
Add this badge to your README

If you maintain this project, the badge below links readers to this analysis and shows its maintenance status from the daily GitHub snapshot. Paste the markdown into your README; add ?metric=license or ?metric=stars to the image URL for a different field.

Add this badge to your README

markdown
[![Hysen Labs](https://hysenlabs.com/badge/fannovel16-comfyui-controlnet-aux.svg)](https://hysenlabs.com/projects/fannovel16-comfyui-controlnet-aux)

Star history chart for your README

markdown
[![Star history](https://hysenlabs.com/chart/fannovel16-comfyui-controlnet-aux.svg)](https://hysenlabs.com/projects/fannovel16-comfyui-controlnet-aux)

Was this useful?