The Fence Remembers

It is 9:14 on a Tuesday morning. A new manager is reading through how refunds work. She finds a step: any refund above a certain size goes to a second person before it is paid. The step adds a day. Nobody on the team knows why it is there. The person who set it up left years ago.

She deletes it. Refunds get faster. Everyone is pleased. Three months later a pattern of large, fraudulent refunds starts to run through the gap, and the step that would have caught it is gone.

The oldest rule about rules

In 1929 G. K. Chesterton described a fence across a road. The reformer says he sees no use for it and wants it cleared away. The wiser man says: if you do not see the use of it, I certainly will not let you clear it. Go away and think. Then, when you can come back and tell me you do see the use of it, I may allow you to destroy it.

That is the whole principle. A rule is a fossil of a decision. Someone once had a reason, and the reason has usually left while the rule stayed. A rule outlives its reason because removing it costs attention, and keeping it costs nothing anyone notices.

Notice what the principle quietly assumes. It assumes that finding the reason is hard. You would have to find the people, dig through the files, read the old arguments. That cost is real, and it has always been the thing that decided what happened next. When the reason is expensive to recover, institutions drift to one of two failures. Some keep every fence forever, and rot under rules nobody can justify. Others hire a reformer, who clears the road and finds out about the cattle the hard way.

So Chesterton’s fence was never really a principle about wisdom. It was a principle about the price of memory.

What AI changes

AI changes the price. The reason behind a rule lives somewhere: in old messages, tickets, meeting notes, version histories, the contract that was signed the week the rule appeared. For a person that archive was a library with no catalogue. For a machine that can read all of it in an afternoon, it is a question you can ask.

The easy conclusion is that this makes fences safer to remove, because now you can check. That is true, and it is the smaller half. The larger half is that it breaks the old stalemate in both directions at once.

Most fences turn out to have no reason left. The cause is gone, the threat moved, the person retired, the system it guarded was replaced. For the first time, you can know that and say so with evidence, and remove the fence with a clear conscience. Some fences turn out to have a reason that is still very much alive. Those stop being defended by folklore and start being defended by record, which is a far stronger position than “we have always done it this way.”

And one more thing most people will miss. A fence whose reason was never written down, because it lived only in someone’s head or in a lesson learned the hard way, will now look exactly like a fence with no reason. The archaeologist will search, find nothing, and report that nothing is there. The absence of a record becomes a verdict. That is the new danger, and it is a danger created by the new power.

The same Tuesday, a few years on

It is 9:14 on a Tuesday morning. A new manager asks her assistant why refunds above a certain size go to a second person. It answers in seconds. The step was added after a run of duplicate payments in a year when two systems were merged. The merge ended. The duplicates stopped. But the assistant adds that the step also catches a second kind of problem, and it shows her the three cases from the last year where it did.

She keeps the step, and changes who it goes to. Then she does something nobody did the first time. She writes the reason next to the rule, in a sentence, with today’s date.

Five things I expect

1. Every rule will grow a “because.”

   Once an agent can ask what a rule is for, rules without an answer will be the first ones questioned. The sensible response is to attach the reason to the rule at the moment it is made. A policy will come to be understood as a sentence of purpose plus an instruction.

2. A wave of quiet deletions.

   The first thing organisations will do with cheap archaeology is clear out fences whose causes died long ago. Approval steps, duplicate forms, review stages. Much of it will be right, and the places that do it first will feel lighter and faster than the places that do not.

3. A new kind of failure: the unwritten fence.

   The expensive mistakes will come from rules whose reason lived only in people. An agent that finds no record will conclude there is no reason. Organisations that care will pay people to say, out loud and on the record, why they do what they do.

4. “We have always done it this way” will stop working in both directions.

   It will no longer defend a rule, because the history can be read. It will also no longer be dismissed as mere habit, because the history can be read. Tradition will be judged on what it actually did.

5. Decisions will be written for a reader who is coming.

   People will start recording why at the moment they decide, because they know a patient reader will arrive years later and find out. The record stops being paperwork and becomes a letter to whoever inherits the road.

Where I am in this

I keep coming back to a plain idea: a business is a set of decisions that remembered themselves. I wrote about the same shape from another side in Only Variety Absorbs Variety, where the menu of responses has to grow to fit the world, and in The Cost Is in the Pairs, where one shared record between agents is what keeps the cost of a team from exploding. This is the same record, seen from the other end. It is not only what a team needs to work together now. It is what a team leaves for whoever comes after.

The habit I want from my own AI is the one I would want from a good colleague: before you change anything, tell me what it was for.

The claim ledger

Already true

Models can read an organisation’s whole written history and answer questions about why something exists. Reasons in writing are already recoverable in a way they never were for one person.

What has to happen

Decisions have to be written down close to when they are made, and the archives have to stay readable by the tools that will ask. If the reasons stay in chat that vanishes or in heads that leave, the fence keeps its secret.

Where I am probably wrong

If most real reasons are tacit, and no amount of reading recovers them, then cheap archaeology will mostly produce confident wrong answers, and organisations will delete more good fences than they save. I would then be describing a better flashlight in a room where the reasons were never on the walls.

Back on that Tuesday: the step is still there. It has a sentence beside it now. It is waiting for whoever comes next.