Krita AI Diffusion: inpainting inside a paint program

The problem: text-to-image tools ignore the canvas

Most image generation interfaces assume you start from nothing. You type a prompt, you get a picture, and if one region is wrong you regenerate the whole thing and hope. Painters work the other way around. They have a canvas, a selection, a layer stack and a deadline, and they need one area changed without disturbing the rest.

Krita AI Diffusion targets that second workflow. The README states the goals plainly: precision and control, so generation can be restricted to selections and refined with a variable degree of strength; workflow integration, so the tool stays unobtrusive inside Krita; and local, open, free, with open source models running on your own hardware. The intended user is someone who already draws or paints in Krita and wants generative fill, expansion and reference-guided generation without leaving the application. It is not aimed at people who want a hosted prompt-to-image service with a gallery of styles.

How the plugin, ComfyUI and the docker fit together

The architecture is two processes, not one. Krita runs the plugin as a Python extension inside its embedded interpreter, and a separate ComfyUI server does the diffusion work. The README describes ComfyUI as the backend and notes that the plugin can install and start that server for you, or connect to an existing installation. If a server is already running locally when Krita starts, the plugin tries to connect on its own; a remote server is possible the same way.

That split explains most of the plugin's behaviour. The docker panel is the control surface: it holds the prompt, the style presets, the job queue and the history of previous generations and prompts. The heavy state, meaning checkpoints, LoRA files, samplers and the required ComfyUI extensions, lives on the server. The README points to a list of required extensions and models for compatibility, which is the practical consequence of this design: the plugin version and the server's model set have to agree.

Generation is driven by what is on the canvas. Selections define the inpainting region, layers define regions with their own text descriptions, and control layers carry scribbles, line art, depth maps, pose or segmentation. The README lists ControlNet support for scribble, line art, canny edge, pose, depth, normals and segmentation, and IP-Adapter for reference images, style and composition transfer and face swap. Upscaling is handled by the plugin as well, described as upscaling and enriching images to 4k, 8k and beyond without running out of memory.

Installing Krita AI Diffusion and running a first inpaint

Krita 5.2.0 or newer is required. Download the plugin ZIP from the releases page, then in Krita use Tools, Scripts, Import Python Plugin from File and point it at that archive. Restart Krita, open or create a document, and enable the docker under Settings, Dockers, AI Image Generation. The README gives exactly these steps.

# No shell install step. Download the release ZIP from:
# https://github.com/Acly/krita-ai-diffusion/releases/latest
# Then in Krita: Tools > Scripts > Import Python Plugin from File...

The first real use is the Configure button in the docker. Clicking it starts local server installation or connects to one you already have. If you prefer to manage the backend yourself, install ComfyUI separately and leave it running before you start Krita; the README states the plugin will then try to connect automatically. Check the required extensions and models list first, because a server without them will not be compatible.

# Optional: bring your own ComfyUI backend instead of the automatic install.
# Start it before Krita so the plugin connects on launch.
# Required extensions and models: https://docs.interstice.cloud/comfyui-setup

With a server connected, the loop is: make a selection on the canvas, type a prompt in the docker, and generate. The result arrives as a preview in the history panel, and you apply or discard it. The README describes the job queue as letting you queue and cancel jobs while you keep working, so a long generation does not block the canvas. For a first test, a small selection on an existing image is the cheapest way to confirm the server, the model and the plugin are all talking to each other.

The 6 GB VRAM line and other hard limits

The README is unusually direct about hardware. It recommends a powerful graphics card with at least 6 GB VRAM for NVIDIA, and warns that otherwise generation will take very long or may fail due to insufficient memory. CPU generation is listed as supported but very slow. That is not a soft recommendation. Diffusion at useful resolutions is memory-bound, and the plugin's upscaling features push further.

Platform coverage has gaps worth reading closely. NVIDIA is supported via CUDA on Windows and Linux, AMD via ROCm on Windows and Linux, Intel via XPU on Windows and Linux, and Apple Silicon via MPS on macOS 14 and later. There is no Android story, and the plugin runs inside Krita's embedded Python with access to the standard library and Qt5, which constrains what the plugin itself can do.

The second limitation is operational. Because ComfyUI is a separate process, version drift between the plugin and the server is a real failure mode. The README directs users to a compatibility list rather than promising that any ComfyUI install will work. If you maintain a heavily customized ComfyUI with your own nodes, that is not a bug in the plugin, but it is friction the automatic install path avoids.

A third point: the README explicitly asks users not to seek help through official Krita channels for issues related to this extension, and points to the project's discussions, Discord and issue tracker instead. Support is community-shaped.

Krita AI Diffusion versus driving ComfyUI directly

The honest comparison is with ComfyUI itself, since that is what runs underneath. ComfyUI exposes a node graph: you wire loaders, samplers, ControlNet stacks and post-processing together and you can change any parameter at any point. Krita AI Diffusion deliberately hides that graph. The README frames the project around strong defaults and a streamlined interface, with customization limited to presets such as custom checkpoints, LoRA and samplers.

So the difference is not capability, it is where control sits. In ComfyUI you control the pipeline. In this plugin you control the image: the selection, the layer, the control layer, the region prompt. If your work involves inventing new sampling chains or chaining many models, the plugin will feel like a constraint. If your work involves painting and you want generation to behave like a brush that respects your selection, the graph is noise you do not want to see.

There is also a segmentation companion. The README points to a separate plugin, Acly/krita-ai-tools, for AI segmentation and background removal, which keeps object selection out of this codebase.

Licence, maintenance and upgrade cost

The project is licensed GPL-3.0, which matters if you plan to redistribute a modified plugin or bundle it into a commercial product; that is a question for your own legal review, not something this article can settle. The repository is not archived, and the last push was on 2026-08-28, with v1.53.0 released on 2026-08-22. Releases have been frequent through 2026, with v1.52.0 on 2026-06-28 and v1.52.1 on 2026-06-30, so upgrade churn is a real cost to plan for.

Upgrading is not just replacing the plugin. Because ComfyUI is the backend and the README maintains a list of required extensions and models, a plugin update can require matching server-side updates. The pyproject.toml sets requires-python to >=3.10 and configures ruff, black and pyright with a 100 character line length, and requirements.txt notes it is for development and tests only, since the plugin itself runs inside Krita's embedded Python with the standard library and Qt5. There is a tests directory and a pytest configuration, but nothing published describes a stable plugin API for third parties.

Editorial conclusion

Adopt Krita AI Diffusion if you already paint in Krita and want generation confined to selections, layers and control layers rather than a separate prompt box. Skip it if you have no CUDA, ROCm, XPU or Apple Silicon GPU, since the README states that local generation on CPU works but is very slow, or if your workflow lives in ComfyUI graphs you want to edit by hand. Before committing, confirm your Krita version is 5.2.0 or newer, check the required extensions and models page for the ComfyUI setup, and decide whether the automatic local server install or your own ComfyUI instance fits your machine.

Frequently asked questions

Official sources

1. Acly/krita-ai-diffusion on GitHub
2. License: GPL-3.0
3. Project website
4. README
5. Releases