0xSojalSec/LLMs-local: a curated index for running models on your own hardware

A README-only index, not an inference stack

The repository has one top-level entry: README.md. There is no source directory, no package manifest, no container definition and no release. Whatever you clone is a document. The README describes itself as a "list of awesome platforms, tools, and resources run for LLMs locally", and that is the whole product.

That framing matters for expectations. If you arrive looking for something to install, you will not find it here. What you get is a table of contents that spans inference platforms, inference engines, user interfaces, model families, tooling, hardware, tutorials and communities. The value is in the grouping: a reader who does not yet know that llama.cpp and vLLM occupy different niches can see them side by side under one heading and then go read each project's own documentation.

The audience is therefore narrow but real. Someone assembling a first local setup, or someone who has been running one engine and wants to know what else exists in adjacent categories, gets a useful map. Someone who already knows the space will find little new, because the list does not compare entries, benchmark them, or record which ones are maintained.

How the categories are organised

The README splits the space along functional lines rather than by vendor or licence. Inference platforms come first (LM Studio, jan, LocalAI, ChatBox, lemonade), then inference engines (ollama, llama.cpp, vllm, exo, BitNet, sglang, Nano-vLLM, koboldcpp, gpustack, mlx-lm, distributed-llama, ik_llama.cpp, FastFlowLM, vllm-gfx906, llm-scaler). User interfaces follow, with Open WebUI, Lobe Chat, Text generation web UI and SillyTavern among them.

Below that the structure widens considerably: Large Language Models with sub-sections for explorers and leaderboards, model providers, and specific models split into general purpose, coding, multimodal, image, audio and miscellaneous. Then Tools, broken into models, agent frameworks, Model Context Protocol, retrieval-augmented generation, coding agents, computer use, browser automation, memory management, testing and observability, research, training and fine-tuning, and miscellaneous. Hardware, Tutorials and Communities close the document.

The category list is more informative than any single entry. It tells you that the maintainer thinks about local LLM work as a stack with distinct layers, and that agent frameworks, MCP and RAG are treated as first-class concerns rather than add-ons. Each section ends with a link back to the table of contents, which is the only navigational affordance in the file.

Reading the entries: descriptions, badges and nothing else

Most entries follow one shape: a GitHub stars badge, a link, and a short description. For example, llama.cpp is described as "LLM inference in C/C++", vllm as "a high-throughput and memory-efficient inference and serving engine for LLMs", and mlx-lm as generating text and fine-tuning large language models on Apple silicon with MLX.

Those descriptions are accurate but shallow, and the list does not go further. There is no column for supported backends, no note on whether a project targets CUDA, ROCm, Metal or NPU, and no indication of licence. The one exception is a handful of entries that name hardware in the description itself: FastFlowLM for AMD Ryzen AI NPUs, vllm-gfx906 for AMD gfx906 GPUs such as Radeon VII, MI50 and MI60, and llm-scaler for Intel Arc Pro B60 GPUs. Those are the entries where the list actually saves you a search, because the constraint is stated up front.

The star badges are worth a word. They are rendered as shields.io images pointing at each project's repository. They are decorative, and the README does not use them to rank or recommend anything. Treat them as visual noise rather than a signal.

What the list cannot tell you

The central limitation is that a link list carries no maintenance signal. The README does not record when an entry was last updated, whether a project is archived, or whether it still builds against current drivers. If you pick an engine from this page and it turns out to be dormant, the list gave you no warning.

There is also no comparison layer. Two entries under Inference engines may look interchangeable in a one-line description while differing sharply in throughput characteristics, quantisation support or serving model. The README does not say which engine suits a single consumer GPU and which assumes a cluster, apart from the entries that name hardware explicitly. You have to open each repository to find out.

The repository itself offers no fallback either. Because only README.md exists, there is no issue template, no contributing guide and no code to inspect. If a link rots, the correction path is a pull request against a single file, and the README does not document how contributions are handled. For a reference document that is tolerable. For anything you depend on operationally, it is not.

Where it sits next to awesome-selfhosted and Awesome lists

The obvious comparison is awesome-selfhosted, the long-running catalogue of self-hostable software. The difference is scope and depth. awesome-selfhosted covers hundreds of categories and applies stated inclusion criteria, so it functions as a general directory with editorial rules. LLMs-local covers one domain in more detail, with sub-sections for MCP, RAG, coding agents and memory management that a general list would not carry.

A second comparison is the documentation of any single engine, such as ollama or llama.cpp. Those are the opposite kind of artefact: narrow, versioned, and accountable for accuracy. LLMs-local is broad and unversioned. The practical consequence is that the list is good for discovery and bad for verification. If you need to know how to serve a model on a specific GPU tonight, the engine's own README is the source that will not waste your time.

The honest position is that these are complements. Use LLMs-local to learn the vocabulary and the shape of the ecosystem, then leave it and read the project you actually intend to run.

Maintenance, licence and the cost of following a list

The last push to the repository was on 2026-06-10. There are no releases, which is consistent with a documentation-only repository. Updating it means editing README.md, so the ongoing cost is editorial rather than technical: someone has to notice that a project moved, changed its description, or stopped being relevant.

The licence is not stated in the repository's top-level entries, and no licence file appears among them. That absence matters if you plan to mirror the list, republish it, or fold it into internal documentation. A list of links and one-line descriptions may raise fewer questions than copied source code, but without a stated licence you have no explicit grant, and the safe move is to link to the repository rather than reproduce the file. This is a description of the situation, not legal advice.

One structural cost is worth naming: because entries point outward, the list inherits the licence and the support model of every project it names. Nothing in LLMs-local shields you from a linked project's own terms.

Editorial conclusion

Use LLMs-local as a starting map if you are new to local inference and want the names of the engines, interfaces and model families in one place, or if you need a shortlist before reading each project's own documentation. Do not treat it as an installation guide or a support channel: the repository contains only README.md, so there is no script, no config and no issue triage to fall back on. Before adopting anything from it, open the linked repository and verify the licence, the supported accelerators and the release cadence yourself, because the list gives a one-line description and nothing more.

Frequently asked questions

Official sources

1. 0xSojalSec/LLMs-local on GitHub
2. Issues
3. README