Future AGI: one loop for tracing, evals, simulations and guardrails
Open-source, end-to-end platform for evaluating, observing, and improving LLM and AI agent applications. Tracing · Evals · Simulations · Datasets · Gateway · Guardrails. Self-hostable. Apache 2.0.
At a glance
- What is it?
- Future AGI packages LLM observability, evaluation, simulation and runtime guardrails into a single Apache 2.0 stack you can self-host with one installer command. The pitch is a closed feedback loop; the practical question is whether you want one platform owning all five stages.
- Who is it for?
- Adopt Future AGI if you are already running OpenTelemetry instrumentation and want evaluation, simulation and guardrails reading the same traces instead of four separate SaaS accounts. Do not adopt it if you only need a trace viewer, or if you cannot accept a Python and Node instrumentor layer inside your request path.
- Can I use it commercially?
- Yes. Apache-2.0 is a permissive licence: you can use, modify and sell software built on it, as long as you keep its copyright and licence notices.
- Is it still maintained?
- Yes. The repository last received commits 1 day ago.
- What is it written in?
- Mainly Python, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem Future AGI is aimed at
The README opens with a blunt framing: teams stitch together evals, observability and guardrails that never close the loop. That is a real failure pattern. A trace lands in one tool, a scored dataset lives in another, a guardrail runs as middleware in a third, and nothing carries the production failure back into the test set that would have caught it. Future AGI's answer is to own all five stages itself: simulate, evaluate, protect, monitor, optimize. The target user is a team already past the prototype stage, with agents in production and enough traffic that edge cases are arriving faster than anyone can hand-write test cases. The repository is Python-first, with a Node package published as @traceai/fi-core, which tells you the intended integration surface spans both backend and frontend runtimes.
One loop, five stages, and what actually connects them
The architecture is best read as a trace pipeline with four consumers hanging off it. Instrumentation emits OpenTelemetry-native spans; the README claims 50+ framework instrumentors and describes the trace format as OTel rather than a proprietary schema. From there, evaluation reads traces and produces scores, simulations generate edge-case inputs before launch, guardrails sit in the request path to block outputs in real time, and the gateway routes model traffic. The gateway is written in Go, which is a deliberate split from the Python core, and the README attributes weighted routing at roughly 9.9 ns, about 29k requests per second on a t3.xlarge, and P99 at or under 21 ms with guardrails enabled. Those numbers come from a committed benchmark harness according to the repository, so they are reproducible rather than asserted, but they are also the vendor's own harness on the vendor's own instance size. Treat them as a starting point for your own load test, not a capacity plan. The unifying idea is that a production failure becomes a dataset row, which becomes a simulation case, which becomes a regression check. Whether that loop is enforced by the product or merely made convenient by co-location is not something the README settles.
Installing it: one script, then a backfill you should not skip
The self-host path requires Docker Desktop or Docker Engine with Docker Compose available before you start. On macOS, Linux or WSL the sequence is to clone the repository, change into it, and run ./bin/install. On Windows PowerShell the equivalent is .\bin\install.ps1. The installer pulls published images rather than building from source, and the UI comes up on localhost:3000. For anything beyond a laptop, the README points at ./deploy/setup.sh, which generates the required secrets and pins the image version. That pinning step matters more than it looks: the README explicitly labels the current line a nightly release for early testing and warns to expect rough edges. Upgrades carry a step that is easy to miss. If your installation already contains traces, you must run ./bin/property-catalog-backfill --execute (or the PowerShell variant with -Execute) after the new stack is healthy. Ordinary restarts never trigger a historical scan. The backfill uses the image already selected by Docker Compose, skips workspaces that are already active, resumes through a durable ledger, and is bounded to active workspaces and projects admitted by the self-hosted supervisor, inside a rolling 366-day source window. That window is the constraint to internalize: traces older than 366 days are outside the scan.
Instrumenting an agent without rewriting it
The README's Python example is short and worth reading literally. You import register from fi_instrumentation and OpenAIInstrumentor from traceai_openai, call register with a project_name, then call instrument() on the instrumentor. After that, the documentation states your existing OpenAI client code is traced as-is, with no wrapper object and no changed call signature. That is the standard monkey-patch pattern used across the OpenTelemetry ecosystem, and it is the right choice for adoption friction. It is also the source of the main operational hazard: instrumentation that patches a library sits inside your process, in the same event loop as your application code, and a bug there is an application bug. The README does not describe a sampling policy, a buffer limit, or a failure mode for the exporter, so if trace volume is a concern you will need to read the docs rather than the repository front page. The npm package suggests a parallel story on the JavaScript side, but the README excerpt does not show a Node example, so the exact API there is unconfirmed from this material.
Where the all-in-one design costs you
Bundling five stages means one deployment, one upgrade cadence and one set of secrets, and it means the blast radius of a bad release covers tracing, evaluation and guardrails simultaneously. The README's own nightly warning makes that concrete. A team that only wanted trace storage is now running a Go gateway and a Python evaluation service to get it. The property-catalog backfill is the clearest example of coupling: an upgrade step that exists because traces and catalogued properties share a lifecycle and a 366-day window. There is also a scoring-transparency question the README answers only in principle. It states that every evaluator is inspectable with no black-box scoring, which is a claim about the Apache 2.0 core. The managed cloud is a separate product with its own compliance posture (SOC 2 Type II, HIPAA, regional data residency), and the README does not say whether cloud-side evaluators are the same code as the self-hosted ones. If evaluator parity matters to you, that is a question for the docs, not the repository.
How this differs from Langfuse and Braintrust
The README names its own comparison set, so it is fair to use it. Langfuse is an observability product: traces, sessions, prompt management and scoring, with evaluation layered on top of captured data. Braintrust is an evaluation product: datasets, scorers and experiment comparison, with observability attached. Future AGI's bet is that the ordering is wrong in both cases, and that simulation and runtime guardrails belong in the same system as tracing, because a guardrail block is itself a trace and a blocked output is itself a test case. The gateway is the part neither of those two offers as a first-class component, and it is where the Go/Python split lives. The trade-off is depth. A dedicated evaluation tool will have more scorer types and a more mature experiment UI; a dedicated observability tool will have more mature retention and query tooling. Future AGI is asking you to accept adequate coverage in five areas instead of excellent coverage in one. For a small team that is usually the right trade. For a platform team with an existing Langfuse deployment and a separate eval harness, the migration cost is real and the README does not address it.
Maintenance, upgrades and the licence
The release cadence visible in the repository is fast: v1.36.1, v1.37.0 and v1.37.1 all landed within roughly two days in September 2026, and the last push to main is dated the same week. Fast cadence plus a self-declared nightly label means you should pin image versions via ./deploy/setup.sh and treat upgrades as a scheduled operation rather than a pull-and-restart. Each upgrade on an installation with existing traces needs the property-catalog backfill step, which is bounded work but not zero work. The core is Apache 2.0, which permits commercial use, modification and redistribution, and requires that you preserve licence and attribution notices and state significant changes. It does not grant trademark rights, and it provides no warranty. The README does not state a separate licence for the Go gateway or the frontend, so if you plan to redistribute a modified build, confirm per-component licensing in the repository before you ship. Nothing here is legal advice; read the LICENSE file and the per-directory notices yourself.
Editorial conclusion
Adopt Future AGI if you are already running OpenTelemetry instrumentation and want evaluation, simulation and guardrails reading the same traces instead of four separate SaaS accounts. Do not adopt it if you only need a trace viewer, or if you cannot accept a Python and Node instrumentor layer inside your request path. Before committing, verify three things against your own data: that the nightly build notice in the README has been lifted for the release you pin, that ./bin/property-catalog-backfill --execute completes inside its 366-day source window on your trace volume, and that the gateway's stated P99 holds with your guardrail set enabled.
Community notes