How we checked our accuracy against Lichess

Every chess app shows an "accuracy" number after a game. We wanted to know whether ours actually tracks how well you played, so we tested it against a public reference.

We took 800 grandmaster player-games, scored them with the same code that ships in the app, and compared every score against Lichess's own. Here's what came out.

The short version

Across 800 player-games (both sides of ~400 grandmaster games), BlunderZero's accuracy matches Lichess's own accuracy with a Pearson correlation of r = 0.978, a mean error of about 1.6 points out of 100, and 85% of player-games within ±3 points.

A mean absolute error of ~1.6 points means that, on a typical game, the accuracy BlunderZero shows you and the one Lichess would show you differ by little more than a single point, and they agree within ±3 points on 85% of games.

How accuracy actually works

We use the same eval-loss method Lichess uses, and the per-move accuracy curve Chess.com publishes, in three steps:

1. Eval → winning chance. A position's engine evaluation (say, +1.5 pawns) is converted to a winning probability with a standard logistic curve. Being up a pawn early matters less than being up a pawn in a sharp endgame; win% captures that.
2. Per-move accuracy comes from how much winning chance you gave up on that move. Give up nothing → 100. Hang your queen → close to 0.
3. Game accuracy combines your per-move scores. A plain average would let one blunder hide behind twenty good moves, so we use a harmonic, volatility-weighted blend (the same reason your Chess.com or Lichess accuracy drops sharply after a single disaster).

The combine is where homegrown accuracy scores usually go off, and it's what we fixed on the way to r = 0.978. An earlier build used a plain average and read about 7 points high; the per-move curve was already fine, the combine was doing the damage.

Why Lichess is our reference

We also ran the test against Chess.com, where we read about 5 points higher than their number (r ≈ 0.76). Here's why, since it's worth explaining.

Depth isn't the cause. Analyzing the same games 50% deeper moved the gap by less than a point, and our score barely changes with depth. The difference is the model: Chess.com publishes its per-move curve, which we use, but its game-level accuracy model is proprietary and runs harsher than any eval-based combine, Lichess included. We calibrated to the reference you can actually pull and reproduce. Chasing Chess.com's game number would mean reverse-engineering a model no one can see, and we'd land five points off from Lichess instead.

So we picked the reference you can actually verify yourself.

Where the numbers come from

The corpus is public. Every game, its per-move evals, and Lichess's own accuracy come straight from the public Lichess API, so you can pull the exact same games we tested. Bulk export is rate-limited, so grab a free Lichess API token and send it as a bearer header — without one the loop stops at HTTP 429:

# 8 GMs x 50 = ~400 games (800 player-games).
# TOKEN from https://lichess.org/account/oauth/token (any scope works)
for u in DrNykterstein Zhigalko_Sergei penguingm1 EricRosen nihalsarin \
         Vladimirovich9000 alireza2003 Konevlad; do
  curl -s -H "Authorization: Bearer $TOKEN" -H "Accept: application/x-ndjson" \
    "https://lichess.org/api/games/user/$u?max=50&evals=true&accuracy=true&analysed=true"
  echo
done > /tmp/lichess_corpus.ndjson

What you can check yourself, and what you're trusting us on

Public and checkable: the games are real and un-cherry-picked, each game's per-move evals are Lichess's, and Lichess's own accuracy for every side is right there in the data. The method is a published open standard — the same win-percent / eval-loss curve Lichess uses — so you can implement it exactly (below) and confirm the reference for yourself.

What the app adds: we run our accuracy code over those same public evals and correlate it against Lichess's number. The r = 0.978 is the agreement between the two across all 800 player-games, re-run on every release. Because the evals are Lichess's, this isolates the scoring method — the one piece that would otherwise be ours to massage — and the inputs and reference stay public.

The exact method (so you can reproduce it)

For each move, from the mover's-POV centipawn eval before and after:

winPercent(cp) = 50 + 50 * (2 / (1 + exp(-0.00368208 * clamp(cp, -1000, 1000))) - 1)

wpLoss        = max(0, winPercent(before) - winPercent(after))
moveAccuracy  = clamp(103.1668 * exp(-0.04354 * wpLoss) - 3.1669, 0, 100)

# game accuracy = mean of two aggregates over a side's moves:
#   - a volatility-weighted mean, each move weighted by the stdev of win%
#     over a trailing window of size clamp(round(moves/10), 2, 8), the
#     weight itself clamped to [0.5, 12] (Lichess's bounds), and
#   - a harmonic mean (so one blunder can't be averaged away).
gameAccuracy  = (weightedMean(moveAccuracies) + harmonicMean(moveAccuracies)) / 2

That is the entire algorithm — the same one Lichess documents. Apply it to the corpus above, correlate your per-side numbers against the accuracy field Lichess returns, and you get the same r ≈ 0.978. Nothing here depends on code you can't see.