The Cost of Being Checked

AI · Trust · Attention

The Cost of Being Checked

When making is free, checking is the whole price.

Easy to check, hard to make

There is an old asymmetry at the bottom of computing. Some problems are hard to solve and easy to check. Hand someone a completed jigsaw puzzle and they see it is finished in a glance. Handing them the box of pieces is a different matter. Finding a solution and recognizing one are different jobs with different prices. Mathematicians gave this a name, the gap between P and NP, and nobody has proved the gap is real. Everyone who has ever proofread anything has lived inside it.

For all of human history that gap protected us, quietly. Making was expensive, so the person who made a thing had already spent most of the cost, and checking was the cheap last step. A contractor built, an inspector looked. A writer wrote, an editor read. The price of making was a filter. Anything that arrived finished had passed through someone’s effort, and effort was evidence.

Now the first price is collapsing. A machine can make the report, the contract, the code, the diagnosis, the proof of a theorem, in the time it takes to ask. The second price has not moved. Reading is still reading. Understanding is still slow, and it still happens at the speed of one person’s attention. Checking a thing is cheaper than making it, but cheaper than something that costs nothing is still not nothing.

A finished thing used to be a signal. Now it is only a claim.

The invoice with your name on it

Here is the consequence most people miss. When making cost a lot and checking cost a little, checking was a footnote. When making costs nothing, checking is the entire bill. Every answer you receive now carries a hidden invoice with your name on it. You can pay it by reading, or you can decline to pay and accept the risk. There is no third option in which the invoice disappears.

It also changes what a finished thing means. A finished thing used to be a signal: someone cared enough to get this far. Now it is merely a possibility. The report is no longer evidence that the work was done. It is a claim that the work was done, and a claim needs somebody standing behind it.

Five things I expect

1. Checking becomes the product.

   The valuable service stops being the answer and becomes the verified answer. People will pay, in money or in attention, to be spared the invoice. The first thing to be sold for real will be the receipt that says someone read it.

2. Trust moves to whoever stakes something.

   A claim is only as good as what the claimant stands to lose. A source that loses nothing when it is wrong will be treated like a stranger shouting in a street. A source that posts a bond, carries liability, or puts a name and a history on the line will be treated like a colleague.

3. The cheap check becomes a design principle.

   The best work will be built so that it is easy to verify, the way a jigsaw is easy to verify. Answers will arrive with their own proof, their own sources in line, their own way to be spot-checked in a minute. Work that cannot be checked cheaply will quietly lose its value, however good it is.

4. Spot-checking becomes a skill, and then a profession.

   Nobody will read everything. The people who matter will be the ones who know where a thing is most likely to be wrong and look there first. It is the oldest skill of the good editor and the good inspector, and it is about to be needed everywhere.

5. The people who skip the check will not know they skipped it.

   This is the dangerous one. Unchecked work looks exactly like checked work. The cost of skipping is silent, deferred, and paid by someone else later. Institutions will drift into trusting what no one has read, and the first they will hear of it will be the failure.

Where I am living this

This is the same pattern I keep coming back to. The faster a machine hands you something, the easier it is to forget that what you were handed is a claim. The more it hands you, the more the scarce part is the thing only you can do, and here the scarce part is looking. I am living inside that arithmetic every day, and so are you.

The claim ledger

Already true

Anyone who has pasted in an AI answer without reading it has paid the price in risk instead of in time. The gap between making and checking is already the largest line in most people’s day.

What has to happen

Making has to stay nearly free while checking stays tied to human attention. Verified answers have to become cheap enough to buy, and liability has to attach to claims in a way people can see.

Where I am probably wrong

If machines learn to check each other so reliably that a person never needs to look, the invoice goes away and this essay is wrong. I do not think that happens, because someone still has to be the one who loses when the check fails.

Back to the desk

Back to 4:40 on that Friday. The summary is sitting there, clean and quiet, asking for nothing. It asks for nothing because it has already taken what it needed, which is your trust. Somebody has to pay for the reading. It will be you, or it will be someone you can name.

The invoice is on your desk.

Background

• Stephen Cook, “The Complexity of Theorem-Proving Procedures” (1971), where the gap between finding and checking is first made precise
• The P versus NP problem, Clay Mathematics Institute

Filed under the same question as the rest of this body of work: what stays scarce when the machine does the making.