How we measure Bearnier AI's answers

We grade every release against 200 device regulatory questions written and answered by regulatory professionals, and publish three scores: answer accuracy, citation validity and correct refusals.
Updated Oct 12, 2026

What the question set covers

CategoryQuestions
Predicates50
Testing40
Standards30
Software and cybersecurity30
Post-market25
Pathways25
Total200

Planned split. Final counts are published with the first scores in January 2027.

How grading works

  1. 1

    Experts write the questions

    Regulatory professionals write 200 device questions and the answers they would accept.

  2. 2

    Bearnier AI answers each one

    Every release answers the full set without seeing the expected answers.

  3. 3

    Graders score three things

    Answer accuracy, citation validity, and whether Bearnier AI correctly refused when the record does not support an answer.

  4. 4

    We publish the scores

    Scores for every release go on this page, starting January 2027.

Results

Answer accuracyCitation validityCorrect refusals
100%75%50%25%0%
First published scores: January 2027One point per quarterly release, for each of the three scores.
Jan 2027Apr 2027Jul 2027Oct 2027
Get the question set when we publish it (first results January 2027)The 200 expert-written device questions behind the scores.

Know what FDA already cleared.