openmake_llm

Self hosted AI workspace

OpenMake LLM is a self hosted AI assistant you run on your own hardware. The README describes it as local first and built for open weight and bring your own key models. It serves a local model through vLLM behind a LiteLLM proxy that speaks the OpenAI compatible protocol, and it routes the same abstraction to external providers you register with your own keys, including OpenRouter, NVIDIA NIM, and Ollama, plus a built in Anthropic adapter. The stated benefit is that your data stays on your machine by default. Every request flows through a message pipeline that applies a provider gate, security and language policy, and prompt and tool assembly without an extra LLM routing round trip. Local and external models share one execution path and an always on tool loop. The README highlights role based model orchestration, where you assign a different model to each functional role such as agent, judge, research, and review, with per user and admin global mappings and server shared keys that carry token budgets. Beyond chat, it adds autonomous agents, a deep research pipeline, and an MCP tool system, all behind JWT auth and role based access control. The at a glance table lists a default local model, twenty two built in MCP tools, eighteen industry agents, a Discord gateway bot, and a four language interface. The design keeps policy separate from execution so the two layers can change independently.

Architecture and routing

The architecture section explains how a query moves through the system. A request enters the message pipeline over WebSocket or REST, which handles the provider gate, security and language policy, prompt and tool assembly, and loading an authorized custom agent. From there a single streamFromExternalProvider dispatch runs local and external models alike with an always on MCP tool loop capped at five tool turns, while special modes intercept earlier. The former per strategy layer was retired so there is one path now. A context fit safety net estimates prompt tokens on entry, including images, and if the effective 262K window is exceeded it truncates input, reduces max tokens, and in the extreme returns a ContextOverflowError as HTTP 413 with an audit record and an automatic webhook alert. User customization is described as four orthogonal axes: Model, Style, Mode, and Custom Instructions and Agents, with system prompt assembly order of memory plus custom instructions plus style. Role based orchestration resolves each subsystem's model through a single registry with a fail open fallback chain from per user mapping to admin global to environment to local default. Cross conversation memory is injected into the system prompt with a per session privacy toggle. The thinking display renders a reasoning timeline with a one line headline from a dedicated summary model, and both are persisted so reopening a conversation restores the view. This split of policy from execution is the core idea.

Agents and tools

The agent features center on a Manus style autonomous agent that works across many tool calling turns inside a persistent Docker sandbox with shell, Python, browser, file, and planning tools, plus human in the loop approval. The README says it records file attachments, injects images through a vision channel, and produces deliverables including Excel and PDF, and it honestly reports non achievement with a goal incomplete marker and a judge rather than falsely marking a task done. Tasks can be saved as reusable templates or put on a recurring schedule. Deep research fans out web search, fetches sources, verifies claims, and synthesizes with citations. The report pipeline renders model produced data through a fixed design template into an HTML artifact exportable to PDF or DOCX. The MCP tool system ships twenty two built in tools and supports external servers, each isolated in Docker with dropped capabilities, a non root user, memory limits, and a network policy. NotebookLM grounding lets you pin a Google notebook as context, with the session cookie encrypted and injected only at spawn. Artifacts render in a sandboxed iframe with optional Docker code execution. PDF and DOCX export run one shot in a Docker sandbox with no network and dropped capabilities. Security covers JWT in HttpOnly cookies, Google OAuth, role based access control, rate limiting, an SSRF guard, Helmet headers, and a unified audit and alert pipeline. The tech stack table lists Node, Express, Next.js, PostgreSQL, and Docker with PM2 for process management.

Installing it

The getting started path is a single curl piped to bash that fetches the source when run outside the repo, generates a .env with random secrets, installs dependencies, starts PostgreSQL and Redis, applies migrations, builds both apps, and launches them under PM2 before waiting for a health check. The installer checks for Node 24, Docker, and PM2, installing what is missing where it can without sudo, and it works on Linux and macOS with Intel or Apple Silicon, and on Windows through WSL2. Flags let you skip prompts, point at an OpenAI compatible endpoint, or skip Docker if you run Postgres and Redis yourself. Re running the install script is safe because it repairs rather than overwrites, and an update command pulls, builds, migrates, and restarts while refusing to touch a tree with uncommitted changes. Database migrations run automatically on boot under a PostgreSQL advisory lock, and you can opt out with an environment variable. Day to day operation goes through a wrapper script with start, status, logs, health, deploy, and stop commands. The manual setup section shows the equivalent npm and docker compose steps, and the .env reference lists the API port, database URL, JWT secret, encryption key, admin password, and LLM endpoint variables. Tests use Jest and Playwright, and the README notes that some unit tests are git ignored locally so a fresh clone reports no matches, which is expected rather than a broken install.

Editorial conclusion

The project is written in TypeScript with a Next.js frontend, is released under the MIT license, and installs with a single script on Linux or macOS.