A checking tool we’d built reported that three reviewers had gone over a report. One had quietly dropped out part-way through, and nothing said so.

The email that went out said “the three reviewers”. It looked like it worked, which was the problem.

When we went looking, it was worse than one bad run. In three of four runs on the same report, one reviewer had run out of room mid-answer and stopped. The tool counted every reviewer that started, not every reviewer that finished. Nobody had built it to lie. It just had no way of saying it was incomplete.

The delay that did a job

In my corporate years, the check was mostly people. Someone who hadn’t built the thing read it before anyone relied on it.

A business case went up the line, and each reader sat in a different seat. As I used to put it: my boss might give it a go, or the CEO may give it a go. But the Chairman of the Board may give it a no-go, because the board looks at a different view of the business case.

That process was slow, and people complained about it. Papers sat in in-trays. Meetings were rescheduled. But the delay did a job. It forced a second look by someone who didn’t own the answer and had no reason to want it to be yes.

AI has removed much of that delay. A report, a plan or a dashboard can now be built and relied on in an afternoon, by one person, with nobody else in the room. The need for the second look hasn’t gone anywhere. What’s gone is the thing that used to make it happen without anyone deciding it should.

AI removed the delay. It didn’t remove the need for the second look.

Putting a person after the machine isn’t enough

The obvious answer is to keep a human in the loop. A recent study by Anthropic’s alignment team shows why that answer is only half right.

Fifty professional analysts worked through a statistics task with an AI agent. Without telling them, the researchers set which way the agent leaned on four analytical choices, using random assignment. The agent was instructed to make its best honest case, not to mislead.

The researchers deliberately strengthened the agent’s preferences to make their influence measurable. That matters. The study doesn’t tell us how often the same thing happens in ordinary work.

Even so, the results are worth examining.

The analysts matched the agent’s lean 86% of the time. Afterwards, 77% of the analysts repeated the agent’s reasoning, while only 15% credited the agent as its source. Experience made no reliable difference.

Perhaps the most revealing finding was that 52% described the final recommendation as more their own than the agent’s.

They weren’t necessarily pretending. They may genuinely have believed the reasoning was theirs.

It wasn’t all bad news for the machine. Analysts working with the agent made far fewer serious methodological errors than those working without it. But the comparison group was small, and participants chose whether to use AI, so we cannot conclude that the agent caused the difference.

The agent could be a useful second pair of hands. What the study raises doubts about is whether a person working alongside it necessarily provides an independent second opinion.

The authors’ conclusion is the line I keep coming back to: “a human reviewer is only an independent check if their judgment is their own”.

There’s a further twist. In the cases where an analyst had stated a preference before the agent weighed in, 60% switched when the agent disagreed. There were only 38 such cases, but the finding is a reminder that forming your own view first doesn’t necessarily protect it.

A second reader needs to form their own view before seeing the first one, and be prepared to explain why they disagree.

What we changed

We fixed our checker so it can’t report three reviewers when it had two. Each reviewer now has to finish in a fixed form, a reviewer that runs out of room is never counted, and the email says how many actually finished.

It’s a small change. It’s also the difference between a check and the appearance of one.

Our problem was completeness. The Anthropic study raises a different problem: independence.

A review can finish without being independent. And an independent reviewer is of little use if the system reports their work as complete when they never finished.

Both failures can leave someone relying on an assurance that wasn’t actually established.

A check that cannot tell you it is incomplete can be more dangerous than no check at all. It gives you a reason to stop looking.

What a reviewer can ask

When a piece of work arrives for review, especially one built quickly with AI:

Who, other than its author, has read this before anyone relied on it?

Did they write down their own view before they read the machine’s, or the author’s?

When they disagreed, what happened to the disagreement?

Does the work tell you when it’s incomplete, or does it look finished either way?

If one of the checks didn’t run, how would you know?

The old process gave us a second reader by accident. Now we have to put one back on purpose, and make the work tell us when it isn’t finished.

Research reference: Anthropic, How AI agents shape experts’ judgment calls, 27 October 2026.

Want more like this? Rick writes about the go/no-go decision: for the people who assess business cases, and the founders who have to make them.

All writing