Ahmet-Dedeler/ai-llm-comparison: an open source AI LLM comparison table built on LiteLLM data
A website where you can compare every AI Model ✨
At a glance
- What is it?
- llmarena.ai is a Next.js 14 site that turns BerriAI's LiteLLM pricing file into a browsable AI LLM comparison chart, with a cost calculator and side-by-side versus view. The code is MIT licensed and self-hostable, but the accuracy of every number depends on a JSON file the project does not own.
- Who is it for?
- Adopt it if you want a self-hosted AI LLM comparison table whose data pipeline you can inspect, or if you are building a similar page and want a working Next.js 14 reference. Do not adopt it if you need per-request latency or quality benchmarks, since the project only compares published pricing and context limits.
- Can I use it commercially?
- Yes. MIT is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
What llmarena.ai actually solves for people choosing an LLM
Provider pricing pages are written by providers. Each one uses its own units, its own tier names and its own definition of what counts as an input token. Comparing GPT-4, Claude and Gemini by opening three browser tabs means normalising three different tables by hand, and the numbers change often enough that a saved spreadsheet goes stale. This project exists to collapse that work into one page.
The intended audience is narrow and practical. The README lists developers picking a cost-effective model, businesses comparing vendors, researchers looking at pricing trends, and students learning what the models cost. The common thread is that none of them need to run the model to make the decision in front of them. They need a price per million tokens, a context window, and a way to see two models next to each other.
That framing matters because it defines what the tool is not. It is a catalogue and a calculator, not an evaluation harness. Nothing in the repository runs inference or scores outputs. If your question is which model writes better SQL, this site will not answer it, and the README does not claim it will.
How the LiteLLM data pipeline feeds the comparison table
The architecture is a static data pipeline in front of a Next.js 14 app. The repository root contains three pieces that make this visible: model_prices_and_context_window.json, fetchLatestJson.js and transformModels.js. The README states that model data is fetched from BerriAI's LiteLLM, which is the upstream source of that JSON file. So the flow is: pull the LiteLLM file, transform it into whatever shape the UI expects, and let the app read the transformed result.
Keeping the raw LiteLLM file in the repository root is a deliberate trade-off. It means a fresh clone renders without any network call and without an API key, which is why the README can promise no registration and a free site. The cost is that the checked-in file is a snapshot. Nothing in the README describes a scheduled job that refreshes it, so a self-hosted instance is only as current as the last commit that touched that file. The README's claim of real-time data applies to the hosted site at llmarena.ai, not automatically to your fork.
The rest of the stack is conventional for this kind of page. Next.js 14 with the app directory, TypeScript, Tailwind CSS, and Radix UI primitives exposed through shadcn/ui, which explains the components.json and components/ directory. Analytics come from @vercel/analytics and posthog-js. None of that is unusual, and none of it is where the interesting decisions are. The interesting decision is treating an external JSON file as the product's source of truth.
Installing ai-llm-comparison and getting the comparison chart running locally
The README gives four steps and they are accurate as written. Prerequisites are Node.js 18 or later and npm or yarn. Clone the repository and move into it:
git clone https://github.com/Ahmet-Dedeler/ai-llm-comparison.git
cd ai-llm-comparisonInstall dependencies. The package.json declares the project name as llmarena-ai and pins Next.js at ^14.2.29, so npm will resolve a 14.x release:
npm installStart the development server:
npm run devOpen http://localhost:3000 in a browser. You should see the comparison interface with model rows populated from the checked-in JSON file. If the page loads with an empty table, the first thing to check is whether model_prices_and_context_window.json is present and parseable, because that file is the only data source in a local run.
For a production build the package.json defines npm run build and npm run start. There are also three bundle analysis scripts: npm run analyze, npm run analyze:server and npm run analyze:browser. Those set ANALYZE, BUNDLE_ANALYZE=server or BUNDLE_ANALYZE=browser before invoking next build, and they exist because @next/bundle-analyzer is a devDependency. If you plan to change the model table, run npm run analyze first so you can see what your change costs in bundle size.
Where this comparison tool stops being the right answer
The hard limitation is scope. The data source is LiteLLM's pricing and context window file. That file describes cost and capacity. It does not describe output quality, latency, rate limit behaviour under load, tool-calling reliability, or how a model handles a long system prompt. A developer choosing between two models at a similar price will get no help from this site on the dimension that usually decides the choice.
There is a second limitation that is easy to miss. The README lists providers including OpenAI, Anthropic, Google, Meta and Cohere, but the coverage for any given model depends entirely on what LiteLLM publishes. A model that a provider ships but LiteLLM has not catalogued will not appear, and there is no documented override mechanism in the README for adding one by hand.
Finally, treat the pricing calculator as arithmetic, not as a bill. It multiplies your usage pattern by published per-token rates. Caching discounts, batch pricing, committed-use tiers and regional differences are not described in the README, so a calculated figure is a starting estimate. Anyone budgeting from it should reconcile against an actual invoice before committing.
How it differs from a hosted LLM leaderboard
The closest alternative is a hosted leaderboard such as the LMArena-style evaluation sites, which rank models by human preference votes on blind side-by-side prompts. The difference in approach is total. A leaderboard answers which model people prefer; this project answers what a model costs and how much context it accepts. One is a survey, the other is a price list.
A second alternative is reading LiteLLM's own repository directly. That is the same underlying data with no UI at all. Choosing this project over that means you want the presentation layer: the versus comparison, the calculator, the responsive table. If you are comfortable reading JSON, you already have the source and the site adds convenience rather than information.
A third option is a commercial LLM observability platform that tracks your own spend and latency across providers. Those measure your traffic; this project measures published list prices. They are complementary, not competing. The mistake would be adopting one expecting the other's output.
Maintenance, upgrade cost and what the MIT licence means here
The repository is not archived and the last push was on 2026-09-14, so it is current as of writing. There are no releases retrieved, which fits a project that is deployed as a website rather than consumed as a package. You do not install a version; you clone the default branch, master, and take whatever is there.
That has a direct upgrade consequence. There is no changelog and no tagged version to pin, so pulling upstream changes means reviewing a diff. The dependency set is ordinary Next.js 14 era tooling: React 18, Tailwind 3.4, TypeScript 5. Upgrading to a later Next.js major would be a normal migration, and the presence of PERFORMANCE_OPTIMIZATIONS.md suggests the maintainers have already done at least one round of bundle trimming that a migration would need to preserve.
The project is MIT licensed, so you can fork it, modify it and deploy it commercially, provided you keep the copyright notice and licence text. That is the general shape of MIT and not legal advice; if you are redistributing it inside a product, have someone qualified read the LICENSE file. One practical note: the model data originates from LiteLLM, a separate project with its own licence, so redistributing the bundled JSON is a question about that file, not about this repository's MIT grant.
Editorial conclusion
Adopt it if you want a self-hosted AI LLM comparison table whose data pipeline you can inspect, or if you are building a similar page and want a working Next.js 14 reference. Do not adopt it if you need per-request latency or quality benchmarks, since the project only compares published pricing and context limits. Before deploying, check the age of model_prices_and_context_window.json, confirm whether fetchLatestJson.js is wired into your build, and read PERFORMANCE_OPTIMIZATIONS.md to see what the maintainers already changed for bundle size.
Frequently asked questions
What is ai-llm-comparison and who is it for?
It is an open source website, also published at llmarena.ai, that compares AI language models across providers by price and specification. The README names developers, businesses, researchers and students as the intended users.
How do I install ai-llm-comparison locally?
Clone the repository, run npm install, then npm run dev, and open http://localhost:3000. Node.js 18 or later is the stated prerequisite.
Where does the model pricing data in ai-llm-comparison come from?
The README states that model data is fetched from BerriAI's LiteLLM. The repository root contains model_prices_and_context_window.json, plus fetchLatestJson.js and transformModels.js for the pipeline.
Is ai-llm-comparison free to use and self-host?
Yes. The README describes the site as 100 percent free with no registration, and the code is released under the MIT License. You can run your own copy from the source repository.
Does ai-llm-comparison measure model quality or speed?
No. The data source covers pricing and context window information, and the README does not describe any benchmark, latency measurement or quality scoring. Use it for cost and specification comparisons only.
Community notes