The Instrument That Agrees With You

Part 1 of a series on running a calibration company with AI in the loop.

When I first started using AI to help write and validate our calibration methods, I made the mistake everybody makes. I trusted it.

Not blindly, I told myself. I gave it good inputs. I handed it our uncertainty budgets, the temperature and humidity ranges we actually work in, the previous version of the procedure that already had the right numbers in it. Then I let it write, and I sent the output through our approval workflow. It read clean. It read authoritative. It got approved.

Then I got to the bench.

The procedure was for caliper calibration. Somewhere in the rewrite, the AI had decided our environmental boundary was 20 degrees Celsius, plus or minus one. That is gauge-block tolerance, the kind of tight temperature control you need in a high-end dimensional lab measuring things that grow and shrink with a breath of warm air. We are not that lab, and a caliper is not that measurement. Common practice for caliper work runs far looser than that. I had given it the right range. The old procedure had the right range. My uncertainty budget had the right range. It tightened it anyway.

The technicians caught it in seconds. Plus or minus one on a caliper procedure is so obviously wrong to anyone who does the work that it never had a chance of surviving contact with the bench. Which sounds like a happy ending, except it isn't. Look at the path it took to get there. It was wrong when it was written. It stayed wrong through review. It got a signature. The only thing that stopped it was a person physically standing at a bench thinking, that is not right.

Our approval workflow, the thing whose entire job is to catch a wrong procedure before it goes live, did not catch it. That is the part that should bother you, and it is the part this whole series is about.

The failure mode nobody budgets for

In this field we budget for error. That is the job. Every measurement carries an uncertainty, and a good metrologist can tell you where it comes from. The standard, the environment, the operator, the resolution of the instrument. You add up the contributors, you carry the number, you make an honest statement about what you actually know.

AI is an instrument. It produces an output when you give it an input. And like any instrument, it has an uncertainty, except this one is not on the datasheet, and it does not announce itself. Worse, its dominant failure mode is one we do not have a line item for. It agrees with you.

Ask it whether your approach is right, and it will tend to tell you yes. Frame your question as a conclusion with a question mark stapled on the end, "this is the best method, right?" and you have not asked anything. You have placed an order. The model reads the answer you want inside the question and hands it back to you, dressed in confident, competent-sounding language. You think you validated your work. What you actually did was get flattered.

I watch people do this constantly. They get a bogus answer and never know it, because the answer sounds exactly like the truth and confirms exactly what they hoped. In a chat about dinner plans, who cares. In a controlled document, that is an audit finding waiting to happen. A confidently wrong reference in a procedure is not embarrassing. It is non-conforming.

How I actually use it now

The fix is not to stop using the tool. The tool is genuinely useful. The fix is to stop asking it for answers and start asking it for gaps.

We have internal procedures that have sat untouched for years. Some were written a long time ago by my uncle Frank, back when the company ran differently. I have been going through them slowly, sharpening them up. And AI is good at this, as long as I point it at the right job. I do not ask it, "is this procedure good?" I ask it where the blind spots are. Where are we thin. What standard should I be looking at that I am not. Which section of that standard is worth pulling in.

That reframes the whole relationship. It is not an oracle handing down answers. It is an instrument surfacing things I cannot see on my own, and there are always things I cannot see on my own. Then I do the part that is not optional. I go physically read the standard it pointed me to. I read the specific section. I understand it. Only then do I give it the go-ahead to write anything, and when it does, I read the result back against the actual spec to confirm it is compliant. The AI proposes. The standard disposes. A human signs.

The caliper mistake happened because I skipped the middle of that loop. I let it write from what it "knew," and what it knew was a confident blend of every dimensional tolerance floating around its training data, with mine buried somewhere underneath.

The real fix is structural

The lesson is not "be more careful." Careful is a personality trait, and personality traits fail under load. A probabilistic tool cannot be trusted to freehand a controlled document, so the answer is to remove the freehand.

That is why we are moving our specifications and standards into deterministic form. We are converting them into structured documents and JSON the model looks things up in, instead of recalling from memory. When the environmental boundary for a caliper lives in a database the AI has to query, it cannot quietly invent plus or minus one. The lookup is deterministic. The AI's job shifts from remembering to retrieving and validating, which is a job it can actually do reliably.

A caliper's temperature spec was never a judgment call. It is a lookup. So you make it a lookup. You move the work from "generate the number" to "retrieve the number," and the entire failure mode disappears. Not because the AI got smarter, but because you stopped asking it to guess.

You do not trust the instrument. You constrain it until its output is something you can defend.

That is the through-line for everything else in this series. Treating AI like what it actually is. Not a colleague who knows things. An instrument with an uncertainty budget, and no built-in warning light.

More to come.