evalgate

Does this pull request make the app worse?

evalgate runs a fixed set of questions with known answers on the base commit and on the pull request's merge commit, compares the two on the same cases, and fails the PR when it is measurably less accurate. This page keeps every gate run for the two apps that use it.

What it can't catch

Production traffic

About 25% of real questions to the live apps are sampled, redacted and sent to Langfuse. A weekly job queues the flagged ones in a private repo, where I label them before any becomes a new eval case. Nothing a visitor typed is published here.