Folkbench, a benchmark of AI API relays paired with a desktop config switcher

Two projects in one repository, only one of them open source

The repository is easy to misread at first, so it is worth separating the two halves. The ranking service lives on a website and is not in this repository. What is in the repository is the source of Folkbench Switch, the desktop client, together with the site and blog sources. The Switch client is built with Tauri and Rust, and the release tags confirm that the client is what ships: the three most recent releases are switch-v0.1.0-beta.3 on 2026-10-14, switch-v0.1.0-beta.2 on 2026-10-11 and switch-v0.1.0-beta.1 on 2026-10-11. The repository is not archived, with the last push on 2026-10-19. The primary language recorded for the repository is Rust, while the site itself is an Astro project written in TypeScript, and the root `package.json` is named folkbench-blog with Astro and the sitemap integration as its only dependencies. The licence is a custom one that the metadata cannot classify, so the LICENSE.md file in the root is the document to read.

What the benchmark actually measures

The hosted service continuously tests AI relay providers and publishes availability, response time and price. The board has several views, with a general ranking and separate boards keyed on the availability metric and the price metric. The framing in the project documentation is that with so many relays around, the questions worth answering are which one is stable, which one is cheap, and which one does not touch your data, and it names success rate, response speed, price and a data security grade as the axes. There is also a per model view, so you can look at how a specific model performs at each relay rather than only the aggregate. The sponsors section in the documentation makes one thing clear about the business model: the listed relay providers support the continued running of the evaluation, which means the ranking and the commercial relationships sit in the same place. A reader deciding how much weight to put on the ordering should weigh that.

The switcher rewrites connection parameters and nothing else

The core promise of Switch is that you save several services with their base URL and API key, then apply one to a target tool with a click, switching back and forth without editing files by hand. Two design choices make it safer than a text editor. First, it shows a diff of the configuration before modifying it, so you can see what is about to change. Second, when you point Codex at a third party service, it safely stashes the official ChatGPT OAuth credentials and restores them without loss when you switch back. The rest of the app is deliberately modest. Probe data from the public service is shown inside the client so you can check a line's status before switching to it, all API keys and configurations are stored only on the local device and are not uploaded, and the app runs entirely offline. There is a local usage estimate that counts token consumption per session and estimates cost from public reference prices.

The path mapping is the whole integration surface

Switch adapts itself to each tool's configuration path and field conventions, and it overwrites only the connection parameters, leaving your other custom settings alone. The documented table covers a long list. Claude Code uses `~/.claude/settings.json`, where the base URL is written automatically and the token and API key credential types are handled. Codex uses `~/.codex/config.toml`, with a custom provider written and the official credentials stashed for later restore. Gemini CLI uses `~/.gemini/.env` and `settings.json`, with environment variables and the validation method synchronised. Aider uses `~/.aider.conf.yml` for model and API endpoint parameters. OpenCode uses `~/.config/opencode/`, Kimi CLI uses `~/.kimi-code` or `~/.kimi` with compatibility for both old and new directory layouts, and DeepSeek Harness uses `~/.dsh/`. A further row names MiniMax Code, Qwen Code, Grok Build, Hermes and Pi as still being adapted. That last row is the honest signal: the list is growing and individual entries will change as those tools change.

~/.claude/settings.json
~/.codex/config.toml
~/.gemini/.env
~/.aider.conf.yml
~/.config/opencode/
~/.dsh/

Downloads, platforms and the beta line

The client is distributed through the GitHub releases page, with macOS builds as a `.dmg` for both Apple Silicon and Intel and a Windows build as an `.exe` for x64. The installation packages and their SHA256 checksum files are published together in the same place, which is the arrangement you want from a tool that handles your API keys. No Linux package is listed, which is consistent with the primary language being Rust but the desktop shell being Tauri with desktop targets only. The current line is a beta, at 0.1.0, and the three releases in the visible history are all within the same week, so the client is moving fast and its config mappings are the most likely thing to break. The site side of the repository is a separate concern: it is an Astro build with a dev server, a preview server, a blog build script and a set of check scripts, including a source hygiene check, an article locale check, a Switch check and a repository tools test suite.

Where this is the wrong tool

The scope is narrow, and the narrowness is a feature rather than a gap, but it does rule out several uses. Switch does not benchmark your own traffic. The availability and latency figures come from the hosted service's probes, so they describe what that service observed, which is a different population from your sessions, especially if you use uncommon models or unusual time zones. Switch also does not manage the relay providers themselves, and the data security grade in the ranking is a published label rather than something the client verifies. A team that needs a specific provider's status monitored continuously has to build that on top of the public board rather than expect the client to do it. And for a single provider on a single tool, a manual edit is simpler than installing a desktop app that holds all your keys, which is the case the project itself describes as the one the tool is meant to remove friction from, so it only pays for itself when you actually switch.

Editorial conclusion

Adopt Folkbench Switch if you move between AI coding assistants and API relay providers often enough that hand editing config files has become the friction, and skip it if you use one provider on one tool, since a manual edit is faster in that case. Verify three things before you trust it with your keys: that the path mapping for your tool matches the file that tool actually reads, since the list of supported tools is long and the tool directories change between versions; that the configuration diff preview behaves the way you expect before it overwrites a working setup; and where the data comes from, because the measurements are produced by the hosted service rather than by your own client, so the availability figures describe the service's own probes rather than your traffic.

Frequently asked questions

Official sources

1. Folkbench/folkbench on GitHub
2. Issues
3. Project website
4. README
5. Releases