LocusBlog

Computer algebra system grading, and checking equivalent answers

Type x/(x+1) into a Locus answer box where the key says 1-1/(x+1) and it is marked right. That is computer algebra system grading: the grader asks whether your expression is equivalent to the key, not whether it looks like the key.

Everything below is measured rather than described. On 2026-09-17 we called the grader in the common crate directly, one function - grade_answer with the answer type and equivalence mode a ranked problem uses - and fed it 38 student-typed forms against 10 answer keys. Thirty-two were accepted. The six that were not are the more interesting half of the result, and they are all here.

Why equivalence is the right question

An answer box that takes typed algebra has to cope with the fact that one mathematical object has many correct spellings. Against a key of $2x$, the grader accepted 2x, x*2, x+x, 2.0*x, 4*x/2, x/(1/2) and (2)(x). None of those is a different answer. They are seven ways of writing the answer, produced by seven people who each finished the problem and then wrote down what they had.

Against a key of $(x+1)^2$ it accepted x^2+2x+1, (1+x)^2, (x+1)(x+1) and x^2+2*x+1. Expanded or factored, both are right, and a grader that insists on one is grading presentation. Against $\sqrt{2}$ it took 2^(1/2), 1.4142135 and 2/sqrt(2). Against $\ln 2$ it took log(2) and -ln(1/2). Against $e^x$ it took exp(x), E^x and e**x, the last of which is Python habit leaking into a maths field. Against $x/(x+1)$ it took 1-1/(x+1) and x*(x+1)^(-1).

The alternatives are worse in specific ways. String matching rejects six of the seven forms of $2x$, which means the student has to guess the grader’s house style on top of doing the mathematics. Multiple choice removes the typing altogether, and with it the part of the work where the interesting mistakes happen - a student who picks the right box out of five has not demonstrated that they can produce the expression, only that they can recognise it. Equivalence checking is the only option that grades the answer rather than the format.

Two stages: symbolic, then numeric

The check runs in two passes and both are needed.

The first pass is exact. Parse the student’s expression $u$ and the key $k$, subtract them, and expand:

$$\texttt{expand}(u - k) = 0$$

If the difference expands to zero the two expressions are identically equal, for every value of every variable, and no further argument is required. This is what catches (x+1)(x+1) against $(x+1)^2$ and 1-1/(x+1) against $x/(x+1)$. It is exact, and when it says yes it cannot be wrong.

It can, however, say nothing. A symbolic simplifier only reaches the forms its rewrite rules reach, and it has nothing to say at all about an answer typed as a decimal. 1.4142135 against a key of $\sqrt{2}$ does not expand to zero, because it is not zero - it is off by about $6\times10^{-8}$.

So the second pass evaluates both sides numerically at sample points and compares the results, accepting when

$$\operatorname{round}_3\big(u(x_i)\big) = \operatorname{round}_3\big(k(x_i)\big)$$

at every sample point $x_i$. The measured tolerance is three decimal places: against a key of 1.0, the grader accepted 1.0004 and rejected 1.0005, which is exactly what rounding both sides to three decimals does. Against a key of $2/3$ it accepted both 0.667 and 0.6667, and rejected 0.66.

Running them in that order matters. The exact check is strict and trustworthy, so it should get first refusal; the numerical check is the fallback that handles decimals and the forms the simplifier cannot reduce. Do it the other way round and every judgement would rest on agreement at finitely many sample points, which is evidence rather than proof - two different functions can agree at three points and part company at the fourth. Do it with the symbolic check alone and every answer typed as a decimal is wrong.

The free-symbol guard, and what it costs

Before either pass, the grader compares the set of free symbols on each side, and refuses unless

$$\mathrm{FV}(u) = \mathrm{FV}(k)$$

where $\mathrm{FV}$ is the set of variables an expression still mentions after parsing.

The reason is direct: y must not be accepted against a key of $x$. Those are different answers, and a numerical spot check that happened to sample both at the same value would not know it. The guard is what makes the numerical pass safe enough to rely on.

It has a measurable cost, and the honest way to describe it is to show the case where it produces a wrong verdict.

That second one is a false negative on a correct answer, and there is no way to read it as anything else. It is the price of the guard. The trade is defensible - typing the wrong variable is a far more common student input than an answer key stored in unsimplified trigonometric form, and the guard’s failure mode is recoverable while the alternative’s is not - but it is a price, not a free win, and the fix is on the authoring side: store that key as 1.

What the parser will not take

Three of the six rejections have nothing to do with equivalence. They are parsing.

Space-separated implicit multiplication is not read as multiplication. 2 x was refused against a key of $2x$, while 2x with no space was accepted. A space cannot be an operator in a field where x y might be two variables multiplied or might be a typo, so the parser declines to guess.

LaTeX operators do not survive in the typed field. 2\cdot x was refused against the same key. Math typed into the MathQuill editor is converted from LaTeX to a plain expression before it reaches the grader; a backslash operator pasted straight into a plain-text field has not been through that conversion and reads as nonsense.

Undefined constants become variables. tau was refused against a key of $2\pi$. It is not in the constant table, so it parses as a free symbol named tau, and an answer carrying a free symbol against a key carrying none runs straight into the guard from the previous section.

The remaining two rejections are the tolerance doing its job. 3.14 was refused against a key of 3.14159 and 6.28 was refused against $2\pi$. To three decimals those are 3.140 against 3.142 and 6.280 against 6.283, so they are genuinely different numbers, and both are what you get from truncating $\pi$ one digit too early. Accepting them would mean accepting a rounding error that propagates through any subsequent calculation.

Six refusals out of 38 inputs, then, and only one of them is the grader being wrong about mathematics. The other five are the parser drawing a boundary somewhere and saying where it is.

The reason all of this is worth building rather than sidestepping comes down to size. The Locus problem corpus is 739 generator specs holding 3165 problem templates across 14 subjects, indexed against a skill taxonomy of 760 topic-and-skill pairs. Every template produces a fresh instance with fresh numbers each time it is served, so the answer key is computed, not stored, and the set of acceptable spellings of that key cannot be enumerated in advance by anybody. Hand-maintaining an accepted-forms list per problem is not a thing that could be done at that size even once, let alone kept correct as the corpus grows.

The same grader runs on both sides of the wire, compiled into the WebAssembly frontend for practice so that feedback arrives without a round trip, and on the server for ranked play so that the verdict which moves your rating is one the browser cannot reach into. That symmetry is only affordable because the grading rule is a function of two expressions rather than a database of blessed strings. The book companions embed generated practice over the same skills as the section you have just read, which means they inherit the same grader, and the daily problem is checked by the same function again.

None of which makes the grader right in every case. The $\sin^2 x + \cos^2 x$ rejection above is sitting in the measurements as proof that it is not.