RAG answer evaluator
Score a RAG answer the ragas way — faithfulness, answer relevance, context precision and recall — with every claim checked against the context.
Your key stays in your browser.It goes straight to the provider, never to our servers, and we neither log nor keep it. Use a dedicated key with a spending cap, and delete it in the provider's dashboard when you are done.
Input
Result
The result will appear here.A retrieval-augmented answer can be fluent, confident and wrong in a way that is invisible unless you hold it against the passages it was supposedly built from. The ragas framework defines the metrics that make the check systematic: faithfulness (is every claim supported by the context?), answer relevance (does it answer the question asked?), context precision (how much of what was retrieved was needed?) and context recall (was what was needed retrieved at all?). Paste the question, the retrieved context and the answer, and this evaluator computes all four by reading — splitting the answer into claims, labelling each supported, contradicted or absent, with the supporting words quoted — and ends with a verdict on whether the answer is safe to show and what to fix first.
How it works
- Faithfulness is computed as a ratio of supported claims to all claims, from a table you can audit line by line, not from a single opaque score.
- A true statement that is not in the context counts as unsupported, exactly as ragas treats it — the question is what the system knew, not what is true.
- The verdict points at the layer to fix: retrieval when recall is low, the prompt when faithfulness is low, the answer itself when relevance drifts.
Where your data goes
With your own key, what you type and the key go from your browser straight to the provider you choose; neither passes through Hysen Labs. In hosted mode, your text goes through our server to our provider (DeepSeek) and is billed in credits; we keep the token counts and cost of each run for billing, never the text itself or the answer. How the provider handles the text is governed by its own privacy policy.
This tool handles keys and credentials, so nothing about a run is saved, not even to your own history.
About your API key
We do not collect, store or leak your key: it lives only in this page's memory (unless you tick “remember in this tab”) and is gone when you close it. Still, treat any key you have pasted into a web page with care — create a dedicated key with a spending cap for use here, and delete or rotate it in your provider's dashboard when you are done.
What it costs
This tool is free, with no sign-in and no points.
Common questions
- Why is my faithfulness 0.6 when the answer is correct?
- Because two of its five claims were not in the retrieved passages — the model filled them from its own knowledge. That is the risk ragas measures: next time the filled-in part may be wrong. Either retrieve the missing facts or instruct the model to answer only from the context.
- Can I run this over a whole test set?
- Not here — this is for inspecting one case at a time, which is how you learn what a score means on your data. For batch evaluation install ragas or promptfoo and use the same metric definitions; the prompts here are written to match their scoring so the numbers compare.
- How reliable is a model judging a model?
- Good for faithfulness and relevance when the claims are factual and the context is given, weaker for nuance and for very long contexts. Use a stronger model as the judge than the one that produced the answer, and spot-check the claim table against the context yourself when a score drives a decision.
The open-source behind it
This tool's prompt is adapted from vibrantlabsai/ragas (Apache-2.0) and runs on the model you choose. To use the same capability from the command line or inside your own program, start with that project.
vibrantlabsai/ragasAlso known as
- rag evaluation faithfulness
- ragas metrics explained
- evaluate rag answer quality
- llm as judge rag
- hallucination check rag
- context precision recall rag