# comfyui_LLM_party: LLM agent nodes for ComfyUI, from API calls to local GGUF models

> comfyui_LLM_party turns ComfyUI into a front end for LLM agent workflows, with loaders for OpenAI-compatible APIs, Ollama, local GGUF and VLM models, plus MCP, GraphRAG and social app connectors. The install path is a ComfyUI custom node, and the entry point is a set of example workflows.

**heshengtao/comfyui_LLM_party** — LLM Agent Framework in ComfyUI includes MCP sever, Omost,GPT-sovits, ChatTTS,GOT-OCR2.0, and FLUX prompt nodes,access to Feishu,discord,and adapts to all llms with similar openai / aisuite interfaces, such as o1,ollama, gemini, grok, qwen, GLM, deepseek, kimi,doubao. Adapted to local llms, vlm, gguf such as llama-3.3 Janus-Pro, Linkage graphRAG

- Repository: https://github.com/heshengtao/comfyui_LLM_party
- Stars: 2,369 · Forks: 205
- Language: Python
- License: AGPL-3.0
- Published: 2026-09-10 · Updated: 2026-09-10 · Language: en
- Canonical page: https://hysenlabs.com/projects/heshengtao-comfyui-llm-party

## What comfyui_LLM_party adds to a ComfyUI graph

ComfyUI is a node graph for image generation. comfyui_LLM_party extends that graph with nodes that call language models, run retrieval, and drive speech or OCR steps, so an LLM can sit upstream of a sampler instead of in a separate script. The README frames the goal as building LLM workflows that integrate into existing image workflows, and the repository is organised around that: llm.py, api.py, a model/ directory, a workflow/ directory of example graphs, and a custom_tool/ folder.

The project targets several audiences at once, and the README says so directly. It lists multi-tool calling and role setting for personal assistants, word-vector RAG and GraphRAG for local knowledge bases, agent-to-agent radial and ring interaction patterns, and social app access (QQ, Feishu, Discord) for individual users. There is also a note that the usage threshold is high and that even users taking the quick start should read the project homepage. That is an honest warning: this is not a single-purpose node pack.

## How the node architecture and model loaders fit together

The mechanism is loader nodes feeding processing nodes. For hosted models, the API LLM loader node takes a base_url and api_key; the README states the base_url can be a relay and must end with /v1/, for example https://api.openai.com/v1/. For Ollama, the same node has an is_ollama option, and the README says base_url and api_key are then unnecessary. For local weights, a local model loader takes either a filesystem path such as E:\model\Llama-3.2-1B-Instruct or a Hugging Face repo id such as lllyasviel/omost-llama-3-8b-4bits.

Model coverage is broad by design. The README describes adaptation to all LLMs with similar OpenAI or aisuite interfaces, naming o1, ollama, gemini, grok, qwen, GLM, deepseek, kimi and doubao, and it lists local LLM, VLM and GGUF paths. For vision, the VLM local loader is documented as supporting Llama-3.2-Vision, Qwen2.5-VL and deepseek-ai/Janus-Pro, with the note that Qwen2.5-VL requires updating transformers via pip install -U transformers. The pyproject.toml dependencies confirm the breadth: langchain, llama-index, sentence-transformers, faiss-cpu, neo4j, mcp, aisuite[all], transformers and bitsandbytes all appear. That list is also the main cost of adoption.

## Installing comfyui_LLM_party and running the first API workflow

There is no standalone installer. The README's quick start tells you to drag an example workflow into ComfyUI and then use comfyui-Manager to install the missing nodes, which is how the custom node gets pulled in. A Windows portable package that already contains the plugin is offered as an alternative for people who have never used ComfyUI; the README states it contains only the party and manager plugins and is Windows-only.

If you prefer to clone manually, the README documents a branch that contains only the API calling components:

```bash
git clone -b only_api https://github.com/heshengtao/comfyui_LLM_party.git
```

Run that inside ComfyUI's custom_nodes folder. The README adds a condition: there must be no other folder named comfyui_LLM_party inside custom_nodes.

After restarting ComfyUI, load the API example workflow from the repository:

```bash
workflow/start_with_LLM_api.json
```

In the API LLM loader node, fill in base_url and api_key. The README gives this example for a hosted endpoint: https://api.openai.com/v1/. Note the trailing /v1/. If you are pointing at Ollama instead, enable the is_ollama option and leave both fields empty. For a local model, use the local model loader and supply a path or a Hugging Face repo id. The README also notes that the LLM API node now has a streaming output mode that prints returned text to the console in real time, and a reasoning_content output that separates reasoning from the response for R1-style models.

## Where the project gets in the way

The dependency surface is the first real constraint. requirements.txt and pyproject.toml pull in pandas, selenium, easyocr, moviepy, librosa, faiss-cpu, neo4j, streamlit, fastapi and aisuite[all], among others. Several of these are heavy or platform-sensitive. The pinned httpx<=0.27.2 exists because some later httpx releases break dependent libraries, and aisuite[all] is an extras install that resolves a large set of provider SDKs. On a machine that already has a working ComfyUI environment, that is a plausible source of version conflicts, and the README's own advice to read the homepage before starting reflects it.

The second limitation is documentation shape. The repository ships how_to_use_nodes.md and its Chinese counterpart, plus a separate text tutorial link, but the README itself is a launch page with a long update list rather than a reference. Node parameters are not enumerated there. If you need to know exactly what a specific node accepts, the README does not tell you; you have to read the node source or the how_to_use_nodes files. There is also no documented rollback procedure for a failed install, and no compatibility matrix tying plugin versions to ComfyUI versions.

## Choosing between comfyui_LLM_party and a script-level framework

The natural alternative is calling a model directly from Python with the openai package, or building on LangChain or llama-index without ComfyUI. Those give you a normal program: version control, unit tests, a CLI, and a deployment story that does not involve a graph editor. comfyui_LLM_party is the opposite trade. You get visual composition, immediate reuse of ComfyUI's image nodes, and example graphs for API, aisuite, Ollama, local distributed, GGUF, VLM and prompt-generation cases, but your agent logic now lives in JSON workflows that are harder to diff and review.

A second comparison is with dedicated agent platforms such as Dify, which the repository topics mention. Dify is a hosted or self-hosted application with its own UI and API; comfyui_LLM_party is a plugin that inherits ComfyUI's execution model and its queue. If your output is images and you want the LLM in the same graph, the plugin is the shorter path. If your output is a service, the plugin adds a dependency on ComfyUI being up.

## Licence, maintenance and upgrade cost

The licence is AGPL-3.0, declared in pyproject.toml via license = {file = "LICENSE"}. For internal use this is usually unremarkable. If you plan to expose a modified version as a network service, AGPL-3.0 carries source-availability obligations that a permissive licence would not. That is a factual difference in the licence text, not legal advice; check it against your own distribution model.

On maintenance, the last push to the default branch was on 2026-07-29, so the repository is not archived and has recent activity. The release history is sparser: v0.6.0 was published on 2025-01-15, v0.5.0 on 2024-11-23, and v0.4.0 on 2024-08-03, while pyproject.toml declares version 1.5.0. That gap between tagged releases and the version field is worth knowing before you pin anything. Upgrades are the real cost: because the plugin sits inside ComfyUI and depends on a long list that includes transformers, bitsandbytes and httpx, a ComfyUI update or a transformers bump can change behaviour. The README's own instruction to run pip install -U transformers for Qwen2.5-VL is an example of a feature that depends on you tracking an upstream library yourself.

## Conclusion

comfyui_LLM_party fits people who already run ComfyUI and want LLM calls, RAG or TTS inside the same graph as image generation, and who accept a large Python dependency set and an AGPL-3.0 licence. It is the wrong tool if you need a standalone agent framework with a stable API, or if you cannot install the full requirements list. Before committing, verify that the workflow JSON you plan to use imports without missing nodes, that your base_url ends with /v1/, and that your ComfyUI environment can resolve the pinned httpx<=0.27.2 and aisuite[all]>=0.1.11 from pyproject.toml.

## FAQ

### How do I use comfyui_LLM_party with a local model?

Use the local model loader node and fill in your model path, for example E:\model\Llama-3.2-1B-Instruct, or a Hugging Face repo id such as lllyasviel/omost-llama-3-8b-4bits. The README also provides example workflows for distributed local models, GGUF and local VLM.

### How do I install comfyui_LLM_party?

Drag one of the example workflows into ComfyUI and use comfyui-Manager to install the missing nodes. The README also documents cloning the only_api branch into the custom_nodes folder with git clone -b only_api https://github.com/heshengtao/comfyui_LLM_party.git, provided no other comfyui_LLM_party folder exists there.

### Can comfyui_LLM_party call Ollama?

Yes. The README states that you turn on the is_ollama option in the API LLM loader node and do not need to fill in base_url or api_key. There is a dedicated example workflow at workflow/ollama.json.

### What format does the base_url need in comfyui_LLM_party?

It must end with /v1/. The README gives https://api.openai.com/v1/ as the example, and notes that the endpoint can be a relay API.

## Sources

- [heshengtao/comfyui_LLM_party on GitHub](https://github.com/heshengtao/comfyui_LLM_party)
- [Issues](https://github.com/heshengtao/comfyui_LLM_party/issues)
- [License: AGPL-3.0](https://github.com/heshengtao/comfyui_LLM_party/blob/main/LICENSE)
- [README](https://github.com/heshengtao/comfyui_LLM_party/blob/main/README.md)
- [Releases](https://github.com/heshengtao/comfyui_LLM_party/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/heshengtao-comfyui-llm-party