What If Evaluations* Had Three Dimensions?
Krishna Paksha Pratipada, Jyeshtha, 2083 Siddharthi, Vikrama Samvata1 July 2026
“Not every positive metric delta is the same kind of result.”
* Evaluations — as used here, encompass not only AI evals to judge AI system outputs, but broadly as in pre/post training AI models, physical transistor-level states, governance, or everyday life. Anywhere a decision is involved.
Your tomatoes come in fat, red, and glossy this year, the best-looking crop you've grown. You take a photo before you've even picked them.
Then you bite into one. Watery. Bland. Something improved. Just not the thing you actually cared about.
The photo wasn't lying. The tomatoes really did look better. What happened is subtler than that: appearance drifted away from quality, and nothing in the photo could have told you so.
Say the tomato had tasted as good as it looked. You'd still have a second question waiting. Would it taste that good next year, or did one good spring do all the work? A single good season doesn't tell you whether you've found a method or gotten lucky.
And a third question sits under both. Good compared to what? If the tomatoes you're used to are shrink-wrapped from a supermarket, almost anything homegrown will taste like a revelation. Maybe the tomato improved. Maybe your baseline did.
Is it good. Would it be good again. Is "good" even measuring what you think it's measuring. Nobody teaches you to ask these three questions in that order, yet you already do, every time something you care about seems to work.
Most formal evaluation collapses all three into one number anyway: a benchmark score, a KPI, a pass or a fail. That number answers the first question. The other two survive, if they survive at all, as caveats someone adds after the fact.
The problem isn't that a score compresses information. It's that it has nowhere to put the parts that don't fit.
This note is about what changes when there's somewhere for them to go.
The limits of a line
A scalar is genuinely useful. It tells you whether something went up or down, whether a threshold was crossed, whether one result beats another. For a lot of purposes, that's the whole job.
Where it breaks down is specific, not general. Some evaluation states differ only along a dimension a scalar was never built to hold.
Take three results that would each show up as a clean win on a dashboard. One is real and will hold up under a second look. One passed today and might not pass tomorrow, a coin that happened to land right once. One passed only because the thing being measured drifted, not because the result actually improved. A dashboard shows the same green checkmark for all three, because a checkmark only has one thing to say.
These three don't call for the same response. One you build on. One you hold and check again. One you throw out and re-measure. A scalar can't tell them apart, so whatever is optimizing against it, a person, a leaderboard, or a training loop, treats all three as identical good news.
A scalar can tell you that something happened. It can't tell you whether to trust it.[1] Evaluation vs Measurement
A measurement assigns a value. An evaluation assigns meaning to that value. Our exploration is primarily concerned with evaluation, not measurement. The question is not how to obtain a score, but how to represent what the score means.
A one-dimensional space has no coordinate where reliability could live, and none where validity could live. If either one matters to you, it either gets inferred by guesswork after the fact, or it gets ignored until something breaks.
From scores to states
Vec3(T, R, V) proposes a different representation. Instead of a position on a line, an evaluation becomes a point inside a bounded three-dimensional space.
Every evaluation occupies a location in the cube rather than a position on a line, each axis running from −1 to +1:
\[(T,R,V) \in [-1,+1]^3\]
The Vec3 space. Every evaluation state occupies a location within this space.
Drag to rotate · Scroll to zoom.
- Truthness (T) asks whether the result satisfies the criterion you actually care about.
- Reliability (R) asks whether it holds up under repeated measurement.
- Validity (V) asks whether the comparison producing it is still meaningfully anchored to the thing you meant to measure.
The important point is not the definitions themselves. It is that each question receives its own coordinate. Once represented independently, correctness, stability, and meaning can vary separately.
These aren't three ways of measuring the same thing.[2] Why three dimensions?
Three is not claimed to be universal. The proposal is narrower. T, R, and V appear to be the minimal set of independent questions required to distinguish many common evaluation failures: Did the result occur? Would it occur again? Does the measurement still mean what we think it means? Additional dimensions may prove useful in particular domains. The claim here is only that these three are already missing from most scalar representations. They're three different questions, and the answer to one tells you nothing about the other two. A result can be exactly as right as before and newly unstable. It can be exactly as stable as before and have quietly stopped measuring anything real.
States that collapse onto the same scalar score separate geometrically.[3] Representation Matters
Maps do not merely describe territory. They determine which distinctions can be expressed. The same landscape appears differently under different coordinate systems. Evaluation may behave similarly. The representation gains room for distinctions that previously had nowhere to exist.
Scalar Based Evaluation
Vec3 Based Evaluation
Collapse vs. separation. Under scalar evaluation, distinct outcomes can collapse into the same apparent result. Once reliability and validity receive their own coordinates, those outcomes separate into different regions of evaluation space and imply different actions.
Click a result type · Drag to rotate · Scroll to zoom.
The idea that distinctions can disappear when forced into a narrower representation is older than modern computing. Long before anyone was optimizing a loss function, logical traditions from Bhārat (India) worked out that true and false aren't the only two distinctions worth naming: the catuṣkoṭi, the tetralemma, adds both and neither.[4] On catuṣkoṭi
catuṣkoṭi (Sanskrit: "four corners" or "four alternatives") is a classical Indian logical framework associated with several philosophical traditions. Rather than limiting analysis to true and false, it explicitly recognizes additional logical possibilities, including cases where both or neither may apply. Vec3 is not a reformulation of catuṣkoṭi. The connection is methodological rather than mathematical. Both begin from the observation that meaningful distinctions can disappear when represented within an overly restrictive framework. Vec3 takes the same intuition and makes it geometric. Instead of four logical alternatives, it introduces a continuous space in which distinctions can persist rather than collapse.
The three axes may correlate in practice. An action that moves one will often move the others too. But none is derivable from the others, and that's the part of the claim worth sitting with. It isn't a claim about statistics. It's a claim about which question is being asked.[5] Isn't this just multi-metric evaluation?
Not quite. Modern systems already track many metrics — accuracy, latency, calibration, robustness, cost, and dozens more. Vec3 is not proposing additional metrics. It is proposing additional dimensions in which evaluation states can be represented. The distinction matters because multiple metrics are often collapsed back into a ranking or composite score. Once collapsed, disagreements between dimensions disappear. Vec3 preserves those disagreements as geometric structure.
Failure has a shape
Once an evaluation is a point in space rather than a position on a line, failure stops being one category and starts being a location. A fragile win and a solid one land in different places now. So do a genuine improvement and a gamed one.
A fragile win can be right today and unreliable tomorrow. What matters is what this replaces. "This passed" and "this passed and will keep passing" used to be the same number. Now they're different places.
A student who memorizes last year's exam may score highly every time. The result is repeatable. The exam is still failing to measure understanding. Reliability survives. Validity does not.
Goodhart's law is usually told as a warning: once a measure becomes a target, people optimize the measure instead of the thing it was standing in for.[6] Goodhart's Law
"When a measure becomes a target, it ceases to be a good measure." Most discussions treat this as a property of incentives. Vec3 treats it partly as a property of representation. A system can continue improving on a measured objective while drifting away from the underlying objective the measurement was intended to track. Vec3 turns that warning into a location. A system can keep climbing on the measured criterion while steadily losing its correspondence to the actual objective. In the geometry, that isn't a mysterious late-breaking failure. It's a rotation: T climbing while V quietly falls away.
The important structure is not merely that the metric fails, but that the failure has orientation.
A number tells you a metric broke after it already broke. The geometry is built to tell you which way it was leaning before that.
Trajectories through evaluation space. Similar metric improvements in score (T = 0.80) can correspond to fundamentally different movements through evaluation space. Vec3 represents not only where an evaluation state is located, but how it is changing.
Drag to rotate · Scroll to zoom.
A number can't rotate
The cleanest way to say the whole idea might be the simplest one. A number can go up. A number can go down. A number cannot rotate.
Rotation only means anything once there's more than one dimension to rotate through. A lot of real failures show up first as a change in the relationship between things, not a change in any single value. A benchmark score can stay healthy while its connection to the capability it's supposed to track quietly comes apart. The individual number still looks fine. It's the angle that changed.
The question, always, is which way the system leaned before the number moved.
In Vec3, the geometry of evaluation states encodes not only what happened but which family of possible failures the current state belongs to. The orientation of a cluster relative to the axes, and the trajectory of that cluster over time, contain information that a scalar representation compresses away before it can be read.
None of this is possible on a line, because a line has no room for rotation. Once you've rotated, you've left the line and entered a space where motion has more than one dimension to move through.
Representation changes the question
The standard evaluation pipeline ends with a number on a dashboard. That number works well enough for the question it answers, which is whether performance went up or down. The problem is that a single number can't make the other distinctions reliably, and that limitation isn't fixable by adding more numbers to the same dashboard.
The limitation is structural, not quantitative. Adding more scalar metrics — tracking accuracy, latency, cost, and user satisfaction separately — does not produce a geometric representation. It produces a collection of independent scalars, each subject to the same collapse problem. The result type of an evaluation is not a composite of its scalar scores. It is the geometric relationship among them.
Vec3 proposes that the representation itself must change.
This changes what it means to evaluate. Instead of asking whether a threshold was crossed, evaluation now asks where the state is located, how it is moving, and what kind of failure that movement belongs to. The shift from a scalar number to a three-dimensional geometry is not an addition of information. It's a change in what information is possible to express.
If you've ever had a result pass every test and still felt wrong, you've already experienced the gap that three-dimensional representation is built for. The geometry doesn't give you new numbers. It gives you a way to see that the number and the thing you cared about can move apart before either one breaks.
A number can't rotate. A point in space can.
When evaluation shapes the system
Systems built on evaluation don't just get measured by it, they get shaped by it. Models get trained against benchmarks, teams get managed against KPIs, and entire reinforcement learning loops are steered by a reward that is, itself, just another score. Once evaluation sits inside the loop instead of outside it, whatever the evaluation can't see becomes the path of least resistance.
That's the actual mechanism behind specification gaming, reward hacking, and benchmark contamination. Not bad actors, just optimization finding the cheapest way to satisfy a number. A gamed result that clears the bar gets kept. A kept result gets built on. The next round of optimization then pushes harder against the same blind spot, because nothing in the pipeline ever saw it as a blind spot to begin with.
You can't optimize for a fact your evaluation has no way to represent.[7] Optimization Pressure
The stronger the optimization process, the more aggressively it exploits distinctions the evaluation system cannot see. This observation appears across machine learning, economics, organizational design, and scientific measurement.
Being more careful doesn't fix this. You can scrutinize a scalar as hard as you like and it will still fail to separate a fragile pass from a solid one, because that distinction was never a question the scalar was capable of answering, no matter who's asking or how closely.
The claim underneath the geometry isn't that reliability and validity are newly important.
Nor does Vec3 determine what those concepts mean in medicine, science, law, or machine learning. Those remain domain questions. The proposal is narrower: if such distinctions matter systematically, they deserve a representation capable of expressing them.
Anyone running an evaluation already knows they matter. The claim is that mattering isn't enough. If a property matters systematically, it deserves a coordinate, somewhere it can be seen and acted on instead of somewhere it can only be hoped for.
Back in the garden: is it good, would it be good again, is "good" even the right thing to be checking. That sequence doesn't stop applying once the thing being judged is a model instead of a vegetable. If anything it matters more, because a model can run that experiment a million times a second, optimizing as hard as it can against whatever you told it counts as ripe. Hand it one number, and somewhere past the edge of what you actually meant, it will find the reddest, most watery tomato you've ever seen, and call it success.
Perhaps evaluation doesn't need better numbers. Perhaps it needs a larger space.