comfyui_LLM_party: an LLM agent framework that lives inside ComfyUI
LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG
At a glance
- What is it?
- The project turns ComfyUI's node graph into an LLM orchestration surface, with loaders for API models, local GGUF and VLM weights, plus MCP, TTS and OCR nodes. It is broad by design, and that breadth is the main thing to weigh before installing it.
- Who is it for?
- Adopt it if you already run ComfyUI and want LLM calls, local GGUF or VLM inference, and TTS or OCR steps in the same graph as image generation, and if AGPL-3.0 licensing fits how you distribute your work. Do not adopt it if you want a small, single-purpose node pack, or if you cannot tolerate a project whose README admits a high usage threshold.
- Can I use it commercially?
- Yes, with strict conditions. AGPL-3.0 is a network copyleft licence: if people use a modified version over a network, for example as a hosted service, you must offer them its source code under the same licence.
- Is it still maintained?
- Yes. The repository last received commits 48 days ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The gap this fills between ComfyUI and an LLM stack
ComfyUI is a graph editor for diffusion pipelines. Its data model is tensors, latents and conditioning, and its execution engine has no native concept of a chat message, a tool call, or a token stream. Anyone who wants a model to write the prompt, or wants an OCR pass to feed a caption into a sampler, ends up writing a side script and copying text between windows. comfyui_LLM_party's stated aim is to close that gap: the README says it develops "a complete set of nodes for LLM workflow construction based on comfyui as the front end," so that users can build LLM workflows and "easily integrate them into their existing image workflows." The audience is therefore not LLM application developers in general. It is people who already work inside ComfyUI and want model calls, retrieval, speech and vision steps to be graph nodes rather than external processes. The README frames the range explicitly, from a single agent pipeline to "complex agent-agent radial interaction mode and ring interaction mode," and from personal social app access to a combined LLM plus TTS plus ComfyUI workflow. That is a deliberately wide scope, and it tells you the project is a toolbox rather than a framework with one prescribed path.
How nodes, loaders and model backends fit together
The architecture visible in the README is node-centric. Model access is split across loader nodes rather than a single abstraction. There is an API LLM loader node that takes a base_url and api_key, a local model loader node that takes either a filesystem path or a Hugging Face repo id, and separate local loader paths for VLM and GGUF weights. The README states that the API side adapts to providers with OpenAI-compatible interfaces, naming o1, ollama, gemini, grok, qwen, GLM, deepseek, kimi and doubao, and that aisuite is one of the supported call paths (there is a start_with_aisuite workflow). On the local side, the VLM loader is documented as supporting Llama-3.2-Vision, Qwen2.5-VL and deepseek-ai/Janus-Pro. Retrieval appears in two forms: word-vector RAG and GraphRAG, with the v0.4.0 release notes mentioning "KG graphRAG neo4j support." Speech and vision utilities sit alongside these: the release notes for v0.4.0 mention chatTTS support, and the repository topics list gpt-sovits, GOT-OCR2.0 and Omost. The v0.6.0 release is described as "The convergence of the OpenAI ecosystem and the MCP ecosystem," which is where the MCP server fits. The practical consequence of this layout is that switching providers is a node swap plus a credential change, not a code change, but each loader node carries its own parameters, so a workflow built against one backend does not transfer cleanly to another without re-wiring.
Installing it and getting the first workflow to run
The documented path assumes ComfyUI is already present. The README's quick start says to drag one of the bundled workflow JSON files into ComfyUI and then use comfyui-Manager to install the missing nodes. The workflows are named by backend: workflow/start_with_LLM_api.json, workflow/start_with_aisuite.json, workflow/ollama.json, workflow/start_with_LLM_local.json, workflow/start_with_GGUF.json, workflow/start_with_VLM_local.json, workflow/start_with_llava.json, and three prompt-generation flows (start_with_VLM_API_for_SD.json, start_with_ollama_minicpm_for_SD.json, start_with_qwen_vl_local_for_SD.json). For an API backend, the README instructs you to fill in base_url, noting it "can be a relay API, make sure it ends with /v1/" such as https://api.openai.com/v1/, plus api_key in the API LLM loader node. For ollama, you turn on the is_ollama option in that same node and leave base_url and api_key empty. For a local model, you put the path in the local model loader node, for example E:\model\Llama-3.2-1B-Instruct, or a repo id such as lllyasviel/omost-llama-3-8b-4bits. There is also a lighter branch for API-only users: git clone -b only_api https://github.com/heshengtao/comfyui_LLM_party.git into the custom_nodes folder, with the README warning that no other folder named comfyui_LLM_party may exist there. One dependency note is explicit: the Qwen2.5-VL workflow requires updating transformers with pip install -U transformers. For Windows users without ComfyUI, the README links a portable package that contains only the party and manager plugins.
Where the design gets in your way
The README is unusually direct about the learning curve: step 5 of the quick start says that "due to the high usage threshold of this project, even if you choose the quick start, I hope you can patiently read through the project homepage." That is a fair warning rather than marketing, and it points at a real cost. The node surface is large, spanning model loaders, agent topologies, RAG, TTS, OCR and social connectors, and the documentation is spread across a main page, a separate text tutorial repository, a Feishu wiki, a cloud disk link and community channels. There is no single reference page for node inputs and outputs in the supplied material, so parameter discovery happens by opening nodes in the graph. The multi-backend loader design has a second cost: a workflow is coupled to the backend it was built against, and the local VLM path has version-sensitive dependencies, as the transformers note for Qwen2.5-VL shows. If your task is a single LLM call inside a Python service, none of this is the right tool. The graph editor, the custom_nodes install step and the AGPL-3.0 licence are all overhead you would be paying for a feature you do not need. The project is also not a hosted service: the README's own framing of a portable Windows package and manual model paths makes clear that you supply the compute and the weights.
How it differs from Dify and from plain LangChain-style code
Dify appears in the repository topics, which invites the comparison. Dify is a standalone LLM application platform: you deploy it as a service, build workflows in its own web UI, and it owns the runtime, the database and the API surface. comfyui_LLM_party inverts that. It has no separate runtime; it is a custom node pack that executes inside ComfyUI's existing graph engine, which means an LLM step and a diffusion step share one execution order and one set of intermediate values. That is the whole point, and it is also the constraint: you cannot run these workflows without ComfyUI, and you inherit ComfyUI's execution model, including its caching behaviour, for LLM calls that are not deterministic in the same way image nodes are. Against hand-written code using an OpenAI-compatible client, the difference is the same in reverse. Code gives you full control over retries, streaming and error handling; the node graph gives you visual composition and immediate reuse of image nodes, at the cost of expressing control flow as graph topology. The README notes that the API node now has a streaming output mode that prints returned text to the console in real time, and a reasoning_content output that separates reasoning from response for R1-style models. Those are code-level concerns that have been surfaced as node outputs, which is the pattern to expect throughout.
Maintenance, releases and the AGPL-3.0 question
The release cadence visible in the supplied material is roughly every two to three months across the listed versions: v0.4.0 in August 2024, v0.5.0 in November 2024, v0.6.0 in January 2025, with the last push to the default branch dated 2026-07-29. Release notes are thematic rather than changelog-style, so v0.6.0 is described as the convergence of the OpenAI and MCP ecosystems, and v0.4.0 as adding chatTTS and neo4j GraphRAG support. That means upgrade planning requires reading the repository rather than a maintained changelog, and because workflows are JSON files pinned to node names and parameter keys, a node rename in a new version can break an existing graph. The licence is AGPL-3.0. This is the network-copyleft licence, and the practical implication for anyone embedding these nodes in a product is that the obligations attach to distribution and, under AGPL specifically, to providing source to users interacting with the software over a network. The README offers no licensing guidance, and the only_api branch does not change the licence of the code it contains. If your use is internal, experimental or otherwise not distributed, the question is smaller; if you plan to ship, this is a decision to take with someone qualified to advise on it, not from a README.
Editorial conclusion
Adopt it if you already run ComfyUI and want LLM calls, local GGUF or VLM inference, and TTS or OCR steps in the same graph as image generation, and if AGPL-3.0 licensing fits how you distribute your work. Do not adopt it if you want a small, single-purpose node pack, or if you cannot tolerate a project whose README admits a high usage threshold. Before installing, verify three things against the current repository: which workflow JSON matches your model type, whether the only_api branch covers your needs, and whether the node names you plan to use still exist after the latest release.
Community notes