If an AI Grades Your Students, Can You Defend the Grade?
When faculty evaluate an AI tool that touches grades, they usually ask whether it is accurate. That is the wrong first question, or at least an incomplete one.
The question that actually matters arrives later, in office hours, when a student sits down and says: I understood that problem. Why didn't I get credit?
At that moment, accuracy in the aggregate is no help. You need to be able to say what was decided, and why, about that student on that problem — and to change it if the decision was wrong. Most AI tools cannot help you do that, because the reasoning that produced the score was never written down.
Why grading on understanding is harder than grading answers
LabNotes.ai does not grade final answers. The instructor defines each problem as an ordered sequence of milestones — the things a student must demonstrate, not the steps to get there — and credit accrues as the student demonstrates them in conversation.
That is the right thing to measure. It is also considerably harder to get right than checking whether a number matches a key, and it happens constantly: eighty students, a dozen problems, six to ten exchanges each. Every one of those exchanges asks a model to decide whether a person understood something.
The obvious way to build this is one model that tutors and scores in the same breath. It is also the wrong way. A model that has just written an encouraging reply has every reason to conclude the student earned the credit. Fluency and understanding are not the same thing, and a single model asked to produce both will confuse them.
So we split the work. Here is what that buys you.
Four guarantees
1. The grader is separate, and it goes first.
Grading happens before any tutoring text exists. A separate model reads only the student's latest message against the instructor's own answer key, with no knowledge of what the tutor is about to say. The tutor is then handed that verdict as final and has to write a reply consistent with it.
Run it the other way and you get the failure mode every faculty member worries about: the tutor congratulates a wrong answer while the milestone stays unearned. The student reads praise, sees no credit, and loses trust in both.
2. The score is arithmetic, not a judgment.
No model produces a number. The grader answers one structured question per milestone — what did the student write, what does the key say, do they match — and the score is milestones earned times points, computed in code.
This distinction sounds pedantic and is not. Asking a model "how many milestones did they earn?" invites it to reason about a total, and a model reasoning about a total will round. Asking it to fill in a form and counting the entries ourselves leaves it nothing to round.
3. Every decision is written down, with the student's own words as the evidence.
For each graded exchange we record which milestone was being attempted, how many attempts the student had already made, whether a hint preceded it, what the grader concluded, and the sentence explaining why — quoting the student's own words, from a message the student wrote.
That last constraint took longer to enforce than it should have. Early on, the grader would sometimes credit a student with an idea the tutor had supplied moments earlier — the conversation reads as though the student understood, because the words are right there on the screen. Requiring a direct quote from the student's own message fixed it.
This is the record that answers the office-hours question. You can look at the exact exchange, see what the grader saw, and decide whether you agree. When you don't, you change the grade — final say has always been yours.
4. Grading is tested before it changes, and you can check our work.
Every change to how grading works runs against a fixed suite of cases before it ships. Each case runs twice, because the grader is configured to be deterministic: the same student work submitted on Tuesday and on Thursday must score the same. If two runs disagree, that is a failure in itself, and it stops the change.
We have published that harness as open source. If your institution wants to examine how grading decisions get made before adopting this, the mechanism is available to read: github.com/Wolsty/llm-judge-harness.
Why we built it this way
Honestly: because we got it wrong first.
For two days, a formatting error in how the grader returned its decision caused it to fail silently and award nothing. On screen, a grader that has crashed and a grader that is being harsh look identical — students demonstrate a step and receive no credit, and nothing anywhere says why. We found it by making the grader write down its reasoning where we could read it.
Every guarantee above exists because of what that episode revealed. A grading decision that leaves no trace cannot be checked, by us or by you. So now it leaves one, every time.
What this does not solve
The grader is a language model, and language models are wrong sometimes. We have not eliminated that and are not going to claim otherwise.
What we have done is make each decision inspectable and reversible: it is recorded, it cites its evidence, it is reproducible, and you can overrule it. That is a meaningfully different proposition from a score that appears with no account of itself.
It also depends on you. The grader checks student work against the answer key and worked solution you provide. A problem with no key recorded is a problem where the grader is judging reasoning without ground truth, and it is measurably more generous when that happens. The answer key is not paperwork; it is what makes the grading defensible.
The standard worth holding
Any tool that participates in assessment should be able to answer three questions about any individual grade: what was decided, on what evidence, and can it be changed.
We think that is the minimum bar for putting an AI anywhere near a gradebook. It is worth asking of us, and worth asking of anything else you are evaluating.
LabNotes.ai is an AI tutor for undergraduate STEM that guides students through problems rather than answering them. If you are considering it for a course and want to see the grading record for yourself, write to founders@labnotes.ai.