Free · no signup · browser-local

RAG Retrieval Metrics: Precision, Recall, MRR and nDCG

Evaluate ranked document IDs against your relevance labels. Set K and the nDCG gain method, then inspect per-query retrieval metrics and the aggregate report.

Browse all tools →

Browser-local calculation · no model calls · no saved input

Input and settings

Each query: {"queryId":"q1","retrieved":["a","b"],"relevance":{"a":3,"b":0,"c":2} }. IDs are literal and unique. Grades are finite numbers 0–5. Threshold 0 means grade > 0; a positive threshold means grade ≥ threshold. Up to 200 queries, 1,000 retrieved and 1,000 judged documents per query.

Precision uses the chosen denominator; an empty list scores 0. Recall divides by all relevant judgments. AP sums precision at relevant ranks and divides by the selected count. Hit is 1 for any hit; F1 is the harmonic mean of precision and recall. nDCG always uses original graded judgments, independently of the binary threshold. Unknown IDs receive grade 0. Queries with no judgments meeting the binary rule show null and are excluded from every macro mean.

Your output

rag-retrieval-evaluator-report.json

Complete the input to preview and export your result.

Scores depend on supplied relevance judgments. Incomplete judgments can penalize retrieved documents; this does not assess generated answers.

Getting started

Paste a query array with queryId, retrieved ordered IDs and a relevance map graded 0–5. Set K, the binary relevance threshold, Precision/AP denominators and linear or exponential nDCG gain, then export per-query and macro metrics.

Before you use the result

Unknown retrieved IDs receive grade 0. Duplicate IDs are rejected. Queries without positive judgments are excluded from every macro metric and shown as null. This evaluates supplied judgments, not generated answer quality.

Try an example

Retrieved [a,x,b], relevance {a:3,b:1,c:2}, K=4: Precision=.5, Recall=2/3 and AP=(1+2/3)/3.

How it works

By default Precision divides by K and AP by min(K, positive judgments). Optional denominators use actual returned count or all positive judgments. Threshold 0 means grade > 0; other thresholds include grades >= threshold. nDCG remains graded regardless of that binary threshold. nDCG uses all judged documents for its ideal ranking and gain grade or 2^grade−1. 200 queries; 1,000 retrieved and 1,000 judged documents per query; 500 KB input.

Frequently asked questions

Evaluate existing ranked document IDs against graded judgments with Precision, Recall, F1, Hit, RR, AP and nDCG at K.

How do I use RAG Retrieval Evaluator?

Paste a query array with queryId, retrieved ordered IDs and a relevance map graded 0–5. Set K, the binary relevance threshold, Precision/AP denominators and linear or exponential nDCG gain, then export per-query and macro metrics.

Does this tool upload my data?

Inputs and files are processed in your browser. Page-view analytics do not contain your tool input.

What should I check before using the result?

Unknown retrieved IDs receive grade 0. Duplicate IDs are rejected. Queries without positive judgments are excluded from every macro metric and shown as null. This evaluates supplied judgments, not generated answer quality.

Related tools

LLM Sampling PlaygroundExplore temperature, min-p, top-k, top-p and greedy selection on your own logits with probability bars and repeatable seeded sampling.RAG Text Splitter with Source OffsetsPrepare traceable text chunks with paragraph, line or fixed boundaries, a hard size cap and adjustable overlap.Token Counter & Vocabulary ComparisonCount exact o200k_base or cl100k_base text tokens locally, inspect token IDs and compare both vocabularies.