Surprise Has an Imaginary Part

Contents

1. A ratio has no units
2. The log was never a choice
3. Four things called surprise
4. The imaginary part
5. What the angle cannot do
6. Where this stops being true
7. Exercises

01 A ratio has no units

The Reality Equation is three letters and a bar. R = A / E. Actual is what the world hands you; Expectation is what the comparison machine had already brought; Reality is the one divided by the other. I have used the same example for years because it refuses to be impressive: you expected eight dollars and were handed five. R = 5/8 = 0.625.

Notice what happened to the dollars. They cancelled. The Reality reading is not 0.625 dollars, or 0.625 of anything — it is a pure number, and that is not a cosmetic fact. Because it has no units, one reading can be laid against another from a different life: a five-over-eight afternoon at a car dealership is the same reading as a five-over-eight afternoon in a hospital corridor, whatever the currencies were. The bar in the equation is doing something a minus sign could not. A − E keeps the units, so it can only ever be compared with itself. A / E throws them away, and what is left is a meter that reads the same in every room.

Let me say, once, what kind of claim the equation is, because the numbered lessons on this site are literally true and the Reality Equation is not that kind of object. It is a proposal about experience — that what you undergo is a comparison, that the comparison is a ratio, and that the ratio is read on a log scale. Nothing below proves the proposal. What I want to show is narrower and stranger: given the proposal, some of its parts stop being choices. The log is one. The angle is another.

02 The log was never a choice

The second line of the equation is S = ln R: surprise is the natural log of Reality. I have called that line lawful, as against the willful things you do afterward with your attention — and I have been asked, fairly, what makes a logarithm lawful. Why not the square root? Why not the ratio itself? The answer is a theorem two centuries old, and it is worth seeing exactly what it does and does not say.

Ask for one property of a surprise readout. Ask that it add. If Reality doubles and then doubles again, the surprise of the second doubling should add to the surprise of the first, and the total should be the surprise of quadrupling. Two successive ratio-changes should sum their reports. Write the requirement down and it is a single line:

Cauchy solved this in the Cours d’analyse of 1821, alongside three cousins of it, and the result is as blunt as a result can be. If f is continuous — or merely monotone on some interval, or merely bounded on some interval — then f(x) = c · ln x for a constant c, and nothing else. Not the square root, not the ratio, not any clever curve. The only freedom left is c, and c is not a freedom about the shape of surprise; it is the choice of unit. c = 1 reads surprise in nats. c = 1/ln 2 reads it in bits. (There is also c = 0, the readout that never registers anything, which the theorem permits and a living organism does not.) Drop every regularity condition and you can build monstrous solutions from a Hamel basis whose graphs are dense in the whole plane; those exist for the additive form of the equation, which is what Hamel wrote about in 1905, and the log form inherits them. No nervous system is one of those.

Any readout of a ratio that adds is a logarithm. The word “lawful” was not a metaphor. It was a theorem I had not yet cited.

So the law under “lawful” is this: if surprise is a readout of a ratio, and if surprise adds, then surprise is a log, up to its unit. Both ifs are real, and I want the second one in full view, because it is the seam. That the stimulus enters through a ratio is Weber’s finding of 1834 — the just-noticeable difference is a fixed fraction of what is already there — and Fechner, in the Elemente der Psychophysik of 1860, integrated it into sensation as the log of the stimulus over its threshold, which is the Reality Equation’s shape with E as the threshold. But Fechner’s integration needed an extra assumption: that every just-noticeable step is one equal unit of feeling — that feeling adds. Stevens spent the 1950s denying exactly that, and his power law replaced Fechner’s log with an exponent; in the 1961 paper he called it repealing Fechner’s law. The mathematics fixes the form. Whether the readout is additive is a fact about the organism, and it is contested. I will come back to what Stevens does and does not take away.

Figure 01 · Computed

Equal ratio steps become equal additive steps

R = A / E	S = ln R (nats)	log₂ R (bits)	Reads as
1/8	−2.079	−3.000	three halvings
1/4	−1.386	−2.000	two halvings
1/2	−0.693	−1.000	one halving
5/8	−0.470	−0.678	the eight-dollar afternoon
1	0	0	exactly as expected
2	+0.693	+1.000	one doubling
4	+1.386	+2.000	two doublings
8	+2.079	+3.000	three doublings

03 Four things called surprise

The word “surprise” has been claimed by at least three exact sciences, and the Reality Equation is a fourth claimant. They are not the same quantity, and readers who know one of them will want to know which this is.

The first is Shannon’s. When an event of probability p occurs, the information it carries is −log p. Shannon never gave that term a name in 1948 — it sits implicitly inside his coding theorems, and his uniqueness theorem for entropy is offered, in his own words, only to lend plausibility; Fano later called it self-information and Myron Tribus, in 1961, coined surprisal. Now watch what the Reality Equation does under its probability reading. If the Actual is the thing that happened, its probability is 1 — it is no longer in doubt — and the Expectation is the probability p the machine had assigned to it. Then R = 1/p and

That is not an analogy. Under the probability reading the second line of the Reality Equation is the information-theoretic surprisal, unit for unit. And it has a property worth noticing: it never goes negative. A probability is at most 1, so R ≥ 1 and S ≥ 0. On this face the world can only pay you; it cannot charge you. Disappointment does not exist there.

Disappointment lives on the other face, the magnitude reading, where A and E are amounts — five dollars over eight — and the ratio can fall below one. There S has a sign, and a negative surprise is exactly the feeling of receiving less than the room had been made for. The equation has always carried both readings, and I have not always said which one I was using. The probability face is the one that talks to information theory. The magnitude face is the one that talks to Weber and Fechner, and the one that has a word for loss.

The other two claimants should be named so they are not confused with either. Bayesian surprise, in Itti and Baldi’s sense, is how far an observation moves your beliefs — the divergence between what you thought before and what you think after — and an outlier that changes no belief carries none of it. Prediction error, in the predictive-coding literature descending from Rao and Ballard, is a difference: input minus prediction, scaled by confidence. That last one looks like the Reality Equation’s rival, subtraction against division, and the rivalry is smaller than it looks. A difference divided by a confidence that itself scales with the prediction — Weber’s scaling — is, to first order, a log-ratio: (A−E)/E ≈ ln(A/E) when the two are close, and for the eight-dollar afternoon the two readings are −0.375 and −0.470. Under multiplicative noise they coincide exactly. The bar and the minus sign disagree only when the surprise is large, and large surprise is where the log’s compression is the whole point.

04 The imaginary part

Now the part that is new to me, and that I think corrects something I published a year ago.

Expectation is not a single number. I have written it for some time as two orthogonal contributions — Prediction, the practical guess from habit and history, and Idea, the pull an idea already aboard exerts on how the Actual will be received — and I have drawn them as the real and imaginary axes of a plane: E = P + iI. Two people meeting the same Actual with the same magnitude of expectation but a different angle — more idea, less prediction — are in different realities. I said that in the essay on choosing history. I believe it.

But in the formal notes from last August, when it came time to compute, I took the norm: E = |P + iI|, a real magnitude, and divided A by that. The norm throws the angle away. Once you have taken it, two expectations at 0° and 90° with the same length produce the same Reality and the same surprise, and the essay’s claim that the angle is decisive has nowhere in the arithmetic to live. Either the angle matters or the norm is right. Not both.

The repair is to stop taking the norm and let the equation be what it already says. Divide the real Actual by the complex Expectation, and take the log of that — the complex logarithm, which every complex number other than zero has. Write the expectation in polar form, E = |E| e^iθ, where θ is the angle between the prediction axis and the direction you were actually facing. Then:

Read the two terms. The real part is the surprise you already knew: the log of the Actual over the size of the expectation, exactly what the norm was computing. The imaginary part is the angle. Not a function of the angle, not a correction for it — the angle itself, with a minus sign, in radians. Nothing was added to the equation to get it there. The log of a quotient is the difference of the logs, and the log of e^iθ is iθ; the angle was inside S the whole time, and taking the norm was the operation that deleted it.

Figure 02 · Computed

Same size of expectation, three angles, one felt surprise

• Actual, A = 5, on the prediction axis
• E at 0° — pure prediction
• E at 41.4° — P = 6, I = 5.29
• E at 90° — pure idea

Here is why this is more than bookkeeping. The third lesson in the mathematics series established one thing about a complex quantity in a body: you only feel the real part. The instrument reads force, force is a real multiple of position, and the imaginary rungs of the ladder are perpendicular to feeling — not small, not faint, perpendicular. Carry that over. If surprise is a complex number, then what you feel when the Actual lands is ln(A/|E|), the real part, the magnitude comparison, and it is the same for every angle at which you could have been facing. And the angle — the whole difference between meeting the job loss as devastation and meeting it as release — is in the part you cannot feel.

That is exactly the phenomenology, stated more sharply than I could state it before. Two people get the same news and register the same jolt. Ask them how big it was and they agree. Ask them what it was and they are in different worlds. The jolt is the real part. The world is the imaginary one. The angle does not change how much surprise you were paid; it changes the direction the payment turns you, and turning, as the sixth lesson put it, is what perpendicular does.

The angle does not change how much you were paid. It changes which way the payment turns you. And you cannot feel a turn.

It also settles a question I had been answering with hand-waving. Two orthogonal contributions, prediction and idea — orthogonal in what sense? In this one: they are the two components of a single number whose log separates into a felt part that depends on both of them only through the length, and an unfelt part that depends on nothing but their proportion. Prediction and idea are not two opinions competing for the numerator. They are magnitude and orientation of the same receiving apparatus, and the log is the operation that pulls those two apart.

05 What the angle cannot do

A picture taken literally becomes a belief, so let me say what this one forbids, because it forbids something I have written.

The modulus of a quotient is the quotient of the moduli. That is not an approximation; it holds for every complex A and every non-zero E, and it means the felt surprise depends on the expectation only through |E|. Adding idea to an expectation — raising I while holding P — makes |E| larger, which makes A/|E| smaller, which makes the felt surprise less. An idea already aboard can rotate the reception of an Actual through any angle you like. It cannot make the same Actual hit harder. On this arithmetic, ideation dampens; it never amplifies.

I wrote, in the essay on the weight reality adds, that ideas are ballast that can make a small wave feel catastrophic. The complex log says that cannot be right as stated, and my own formal notes agreed with the log before I did — they record that increasing the coherence of an idea always contracts experience. So one of three things is true. Either the ballast essay was describing an Actual that also has a direction, so that A is complex too and the wave is small only along one axis; or the amplification is not in S at all but in what attention does with S afterward, on the willful side of the line; or the model of expectation as a single complex number is wrong. I do not know which. I know the equation will not let me have all three essays at once, and I would rather have the equation.

One more edge, because it is where the imaginary part does something strange. The angle is defined only up to a full turn, and the conventional log takes it in (−π, π]. At θ = π exactly — an expectation pointed directly away from the prediction axis — the imaginary part jumps from just under −π to +π. Same length, same felt surprise, and the unfelt part flips sign. Whether that seam corresponds to anything in a life — the same news arriving from the opposite side — I leave open. The cut is a convention; the jump is not.

06 Where this stops being true

What is load-bearing

Cauchy’s theorem: on the positive reals, the continuous (or monotone-on-an-interval, or bounded-on-an-interval) solutions of f(xy) = f(x) + f(y) are exactly c · ln x, including c = 0. The complex log identity ln(A/E) = ln(A/|E|) − iθ on the principal branch. |A/E| = |A|/|E|. The probability reading of the equation equals Shannon surprisal in nats. Every number in both figures is computed.

What is convention

The unit, c — nats or bits. The branch of the log, and therefore where the imaginary part jumps. Naming the real axis “prediction” and the imaginary axis “idea” rather than the reverse: no instrument forced that assignment the way force forced it in the third lesson, so I chose it because it fits, and a critic who swaps the axes has made a different model, not an error. And the Reality Equation itself is a proposal, not a theorem: nothing here proves that experience is a ratio.

Where the shorthand breaks

“The log is forced” is true only under two premises, and the second is contested. The stimulus must enter through a ratio (Weber), and the readout must add across successive changes (Fechner’s equal-step assumption). Stevens denied the second, and in log–log coordinates his power law is still a straight line — so the log survives on the stimulus side as the coordinate, but whether feeling is an interval scale or a ratio scale is an empirical fight the mathematics does not settle; Luce showed in 1959 that the answer decides between log and power law. Also: the four surprises are four quantities. The Reality Equation coincides with prediction error only under multiplicative noise, or to first order under Weber-scaled confidence, and it is not Bayesian surprise at all.

Where I would push back on myself

“You only feel the real part” was a theorem about force, with an instrument to anchor it. Here there is no instrument; I am carrying the assignment across by analogy and the analogy could simply be false — there may be something in experience that registers orientation directly, in which case the imaginary part is felt after all and the essay’s central claim fails. The model also cannot, as it stands, let an idea amplify a surprise, and I have published a case where one seemed to. That is a real prediction with a real failure condition: if the same Actual can be shown to produce a larger felt jolt in a person carrying more idea, with prediction held fixed, then either A needs an angle of its own or the complex-number model of expectation is wrong. I would rather find that out than protect the picture.

07 Exercises

1. The two units Take the eight-dollar afternoon, R = 5/8. Compute its surprise in nats and in bits, then show that the ratio of the two answers is the same for every row of Figure 01. Say in one sentence why that ratio is the theorem’s only freedom.
2. Why not the square root Suppose someone proposes S = √R − 1 as the surprise readout. Find two ratios x and y for which f(xy) ≠ f(x) + f(y), and state what a person using that readout would misjudge about two successive disappointments.
3. The two faces Give one everyday event that has a natural probability reading (R = 1/p, surprise never negative) and one that has only a magnitude reading (R = A/E, surprise can be negative). Then find an event that has both, and say whether the two readings agree in sign.
4. Rotate without changing the jolt With A = 5, write down two expectations of length 8 — one with P = 8, I = 0 and one with P = 0, I = 8 — and compute S for each on the principal branch. Confirm the real parts agree and the imaginary parts differ by π/2. Then find the expectation of length 8 whose surprise has imaginary part exactly −1.
5. Argue the other side Make Stevens’ case at full strength: feeling is a ratio scale, equal stimulus ratios produce equal subjective ratios, and so the additive premise that forces the logarithm is simply false of human beings. Then say what this essay loses if he is right — and what, if anything, survives in the log–log plane.