Dimensions Before Parameters: Why Geometry Helps Explain Artificial Intelligence

When people describe a large language model, they almost always begin with its parameter count.

The number sounds impressive, and it matters. Parameters determine much of a model’s computational size, training expense, and capacity to learn complicated transformations. But parameter count is not the whole story. In some ways, it is not even the most interesting place to begin.

Before asking how many parameters a model has, we should ask a more fundamental question:

What kind of representational space does the model inhabit?

That is a question about dimensions.

Parameters and dimensions are related, but they are not the same thing. Confusing them obscures one of the deepest reasons artificial intelligence can produce language that appears coherent, relational, and remarkably intelligent.

A dimension is an independent degree of freedom. It is another way something can vary.

A parameter is a learned numerical value that helps determine how representations are transformed.

Or, more simply:

Dimensions define the space of possibility. Parameters learn how to shape movement within that space.

Consider an ordinary two-dimensional vector:

The vector exists in a space with two dimensions. It requires two coordinates to specify its location.

Now imagine transforming that vector with a two-by-two matrix:

The matrix contains four adjustable numbers. Those four numbers can rotate, stretch, compress, reflect, or shear the vector.

The space still has two dimensions. The transformation has four parameters.

That simple example contains the distinction we need. Dimensions belong to the representational space. Parameters belong to the machinery operating within that space.

We could apply ten different matrices to the vector and accumulate forty parameters. We could apply a thousand matrices and accumulate four thousand parameters. The parameter count could continue growing while the vectors remain two-dimensional.

Parameter count is therefore not dimensionality.

This matters because discussions of AI frequently move among words such as parameters, weights, embeddings, tokens, features, and dimensions as though they all referred to different measures of one general quantity called “model size.” They do not.

• Weight

  Usually a learned multiplier inside a transformation. Weights are parameters, although a model may also have other kinds of learned parameters.

• Token

  A discrete unit into which text has been divided.

• Embedding

  A vector used to represent a token. The dimension of that embedding is the number of coordinates in the vector.

• Feature

  A pattern, distinction, or property represented somewhere in the model’s activity. A feature does not necessarily correspond to one coordinate or one dimension.

• Parameter

  One learned numerical component of the machinery that transforms these representations.

Once the distinctions are clear, the geometry of intelligence becomes easier to see.

Imagine Carl Sagan’s Flatland.

A creature confined to two dimensions can move forward and backward or left and right, but it has no access to height. It cannot step over an obstacle. It cannot look down upon the entire plane. It cannot understand an enclosed shape in the same way a three-dimensional observer can.

When we add a third dimension, we do not merely create a slightly larger sheet of paper.

We create new kinds of relationships.

We get above and below. We get floors and ceilings. We get stairs. We get depth, enclosure, volume, and perspective. We gain the ability to see relationships that are inaccessible from within the lower-dimensional space.

One additional degree of freedom changes the ontology of the world.

Now imagine adding not one new dimension, but thousands.

We cannot visualize such a space. Human visual intuition was shaped by life in three spatial dimensions, not by mathematics operating in thousands of dimensions. But our inability to picture a high-dimensional space does not make it mysterious to mathematics.

A vector in a 10,000-dimensional space is simply a list of 10,000 coordinates. Each coordinate provides another independent way in which the representation can vary.

This does not mean that one dimension is labeled “justice,” another “dog,” another “blue,” and another “betrayal.” The dimensions do not arrive with semantic names attached. They are degrees of freedom, not preassigned concepts.

Training discovers how to use them.

Meanings are usually distributed across many dimensions. A concept may correspond to a direction, a region, a pattern of activation, or a relationship among multiple vectors. Different concepts may overlap. A single dimension may participate in representing many different things.

The important point is not that every concept receives its own private axis.

The important point is that a high-dimensional space provides room for many relationships to be represented simultaneously.

Consider the familiar example involving words such as king, queen, man, and woman. The popular description suggests that vector arithmetic can reveal a relationship resembling:

Real language models are more complicated than that tidy equation suggests, but the geometric intuition remains valuable. Words and concepts can be positioned so that relationships among them appear as directions, distances, angles, and transformations.

Masculinity and femininity do not need to occupy a single designated gender axis. Royalty does not need its own isolated coordinate. The model can distribute these relationships across many dimensions while preserving useful geometric regularities.

Add more dimensions, and the representation gains additional degrees of freedom. It has more room to distinguish royalty from gender, gender from biology, biology from social convention, social convention from historical context, and historical context from the particular sentence being discussed.

These distinctions do not have to overwrite one another. They can coexist within the same representation.

That is the importance of dimensionality.

A low-dimensional representation must compress many relationships into a small number of available directions. Concepts are more likely to interfere with one another because there are fewer independent degrees of freedom available.

A higher-dimensional representation offers more ways to separate, combine, and contextualize patterns. It gives the system more representational room.

That does not mean that adding dimensions automatically creates intelligence. An empty high-dimensional space knows nothing. A model with poorly learned parameters can waste its available dimensionality. Architecture, data, training objectives, computational scale, and optimization all matter.

Dimensionality creates possibility.
Learning determines how well that possibility is used.

This is why I prefer to describe parameters as the learned machinery that sculpts the representational space.

A parameter does not function like a stored sentence in a database. It is not ordinarily a compartment containing a fact such as “Paris is the capital of France.” The model does not retrieve that statement from a specific memory address in the way conventional software retrieves a record.

Instead, training adjusts vast numbers of parameters so that patterns in language produce useful transformations of high-dimensional vectors.

Knowledge becomes distributed throughout the system. It appears in relationships among parameters, activations, directions, and transformations. No single weight contains the answer. The answer emerges from the coordinated operation of the network.

This is one of the major differences between traditional software and artificial intelligence.

Traditional software is explicitly constructed. A programmer writes rules and stores data. When the software produces an answer, we can often identify the instruction it followed and the record it retrieved.

A neural network is trained rather than explicitly programmed in that fashion. It learns a geometry of relationships.

That does not mean the model contains no information. Clearly it does. But it does not store knowledge in the ordinary sense of a filing cabinet, database, or library of prewritten responses.

It represents knowledge through learned structure.

That is why the word representation is so important.

A model does not need to possess an internal sentence declaring that queens are female monarchs. It needs a sufficiently rich representational geometry in which queens, kings, women, men, monarchy, inheritance, politics, history, and countless contexts have useful relationships to one another.

When a prompt arrives, the model converts tokens into vectors. Those vectors move through layer after layer of learned transformations. At each stage, the representation is revised according to context.

The vector for a word is therefore not fixed in the deepest sense. The word bank in “river bank” develops a different contextual representation from bank in “investment bank.” The surrounding language changes the direction and structure of the representation.

The model does not simply look up the word bank. It constructs what bank means here.

This is where parameters and dimensions perform different roles.

The dimensions provide the degrees of freedom necessary to represent the evolving contextual meaning. The parameters determine how the representation should change as it encounters other words, grammatical structures, relationships, and patterns.

Dimensions provide the stage.
Parameters choreograph the movement.

There are also two different high-dimensional spaces worth distinguishing.

First, there is representation space. At any moment, a token is represented by a vector with some number of dimensions. This is the space in which the model represents contextual meaning.

Second, there is parameter space. If a model has billions of trainable parameters, then training can be understood as searching through a space with billions of adjustable coordinates. Each possible setting of those parameters defines a slightly different model.

During inference, vectors move through representation space. During training, optimization changes the model’s position in parameter space.

That distinction explains why a model can have billions of parameters while its token representations have only thousands or tens of thousands of dimensions. The parameters are not all separate dimensions in the token’s representational world. They are the learned numerical structure governing a long sequence of transformations within and between representational spaces.

Public discussions emphasize parameter count partly because it is easy to report. It produces a single, impressive number. Dimensionality is more complicated. A model can contain several different relevant dimensions: an embedding dimension, a hidden-state dimension, attention-head dimensions, and expanded intermediate dimensions inside its feed-forward layers.

There may be no single number that captures “the dimensions of the model.”

Nevertheless, dimensionality deserves far more attention than it receives.

Parameter count tells us something about the amount of learned machinery. Dimensionality tells us something about the representational universe in which that machinery operates.

The two interact. Larger models often use both more parameters and richer internal representations, although the relationship is not a simple law. A model with greater dimensionality is not automatically more intelligent, just as a building with more rooms is not automatically better organized.

But more rooms create possibilities that a one-room structure cannot provide.

The same principle applies here. More independent representational directions can allow more distinctions, more simultaneous relationships, and more contextual nuance. Whether the model uses that capacity well depends on what it learns.

This returns us to the philosophical heart of the matter.

Intelligence is not merely the accumulation of facts.

Intelligence is the ability to represent relationships: similarity and difference, cause and consequence, whole and part, possibility and actuality, agent and object, means and purpose, past and future, literal and metaphorical, self and other.

A high-dimensional vector space is powerful because relationships can become geometry.

• Similarity→proximity
• Difference→distance
• Association→direction
• Relevance→alignment
• Context→a transformation
• Attention→a calculation of how strongly one representation relates to another

The resulting system does not reason exactly as a human being reasons. Its form of intelligence should not be romanticized or mistaken for consciousness. But neither should it be dismissed as mere retrieval.

Something more interesting is happening.

The model is repeatedly reconstructing representations within an extraordinarily rich geometric space. Its parameters have learned how to transform those representations in ways that preserve and exploit vast patterns of relationship.

That is why a conversation with a language model can feel so startling.

The model is not pulling complete answers from a warehouse. It is not searching through a hidden stack of index cards. It is generating each response through the movement and transformation of vectors in high-dimensional space.

The coherence does not come from one parameter. It does not come from one dimension. It emerges from the interaction between a rich representational space and the enormous learned machinery operating within it.

So when someone announces that a new model has hundreds of billions of parameters, the number is worth hearing.

But it should not be the end of the conversation.

We should also ask:

1. How rich is the model’s representational space?
2. How many independent degrees of freedom are available?
3. What relationships can be represented simultaneously?
4. How effectively has training organized those possibilities?
5. And what new forms of relational structure become possible as dimensionality increases?

Those questions lead us closer to the real mystery.

Parameters tell us how many adjustable values the model has learned. Dimensions tell us what kind of world those learned values are capable of shaping.

Before counting the knobs on the machine, we should understand the universe in which the machine is operating.