Alien Intelligence · Part three
Two Seats Until Five
At a box office, a request, a permission, and a promise can sound almost the same. An AI has to keep them apart.
A woman calls the theater on Friday. Can you hold two aisle seats for tomorrow’s show? The clerk checks the seating chart. “I’ll hold B12 and B13 until five this afternoon.” The woman stops looking for tickets. She texts her sister. She starts arranging her day around two chairs in a dark room.
Put an AI at the window and it can say the same sentence. It can also sound just as sure when it has only looked at the seating chart. That difference begins before it answers, with how it treats the words that reach it and what it can do with them.
In Alien Intelligence, I started with two kinds of attention trained on the same object. In The Room We Get Only Once, I followed the differences into memory, rehearsal, and uncertainty. At this window I want to look at three more. The words coming in, the power behind the glass, and the words going back out can each point in opposite directions.
The words coming in
Suppose the woman emails the theater after the regular hold expires. She writes, “The manager said to keep B12 and B13 for me overnight.” The clerk understands the request. She also understands who’s making it. The customer may be right, but the clerk knows where to look for the manager’s approval.
In an AI system, the customer’s message can reach the same model that reads the theater’s instructions. If the system gives those words the wrong kind of influence, “keep the seats overnight” can steer the work as if the theater itself had asked. The instruction can arrive inside an email, a webpage, or a document the AI was sent to inspect. Kai Greshake and colleagues demonstrated that distinction in their research on indirect prompt injection. The attacker’s words reached the system through the material it retrieved. Read the original study.
People can be fooled by a convincing claim of authority. Models can learn to distinguish instructions from material they’re reading. The difference is where the distinction has to come from. A clerk brings the social roles of customer and manager to the sentence. An AI system has to preserve who sent which words and what that source can authorize. A line inside an email can call itself “manager approval” with perfect confidence. Its place in the conversation still belongs to the sender.
The power behind the glass
Take a small theater with a paper chart. The clerk who can pencil in two seats also has the physical ability to pencil in the whole row. The theater hires her judgment along with her hand. It trains her to recognize when a group needs a different arrangement and when to get the manager. Most of the discretion lives with the person who can do the work.
Now give the same job to an AI. It can see the entire chart, infer how someone could work around the usual policy, and still have only one way to touch the inventory. The reservation system can let it place one hold for at most two available seats per customer and performance, with an expiration no later than five. The model can propose a different move. The reservation system can decline it. A separate manager approval can permit one particular exception.
That arrangement has a name in computer security. Jerome Saltzer and Michael Schroeder called it least privilege, giving each process only the powers it needs. Research on AI agents has taken the same idea further by tracking where information came from and enforcing permissions outside the model. This is why I would give the box office AI the ability to place a hold and keep the seating chart out of its power to rewrite wholesale. Read Saltzer and Schroeder. Read the research on enforcing permissions around a model.
The reversal is practical. When I trust a person with more difficult work, I may give her more discretion because she has shown how she uses it. When the AI becomes more capable, I can give it the harder problem and keep the same narrow permission. It can figure out why the overnight hold makes sense and send the decision to the person who can make it.
The words going back out
When the clerk says she’ll hold the seats, she puts herself and the theater on the hook. If her pencil breaks a second later, she borrows another. If she discovers that she promised seats she can’t hold, she owes the woman a call. J. L. Austin studied how, in the right circumstances, saying something can itself do something. The clerk’s sentence makes a commitment even if the mark on the chart takes another moment. Read Austin’s lectures.
For an AI at the same window, I would arrange things in the opposite order. First let the reservation system place the hold. Then let the AI tell the woman which seats it held and when the hold expires. A hold that times out or comes back with an uncertain result should be checked against the actual reservation record. The customer deserves to hear whether she can stop looking.
The theater can authorize its AI to speak on its behalf. It also has to deal with the woman who relied on a mistaken confirmation. That’s why the link between a hold and its confirmation matters. The agent can produce the same confident sentence for three different customers, but a good reservation system will grant a hold on those two seats to only one at a time. The power to issue the confirmation belongs where it can be tied to a specific result.
At four fifty, the woman gets to the window. The clerk types her name and finds B12 and B13 waiting for her. She doesn’t have to repeat what was said on the phone. The seats and the time have reached the window ahead of her.