The robot brain is being rewritten.

01 / The robot brain we grew up with

The classical robotics stack worked because it turned an impossible problem into a collection of tractable ones. If a planner needed to know whether a lane existed, perception gave it a lane. If it needed to know where the vehicle was, localization gave it a pose. If it needed to know whether an object was traversable, the world representation encoded that property in a form the planner could consume.

Each capability came with its own meaning. Engineers decided what each module should know, what it should forget, and what it should pass to the next part of the system.

I inherited a system like this once. An outdoor machine in an unstructured environment, heavy on LiDAR, almost no learning anywhere, and a behavior tree grown by hand, one condition at a time. It worked. Then we took the same code base into a slightly different version of that environment, and logic that was right in one place was quietly wrong in the next. Every fix added a branch, and every branch touched others. From the outside the machine looked autonomous. From the inside it was a board full of wires, and we were the ones holding them. The robot never decided anything. It selected among contingencies we had already authored.

Robotics has always needed an internal representation of reality. What’s changing now is where that representation comes from, what it contains, and how tightly it’s coupled to reasoning and action.

02 / Then AI started eating the boundaries

Deep learning first replaced individual boxes. Object detection and semantic segmentation improved. Learned prediction replaced pieces of hand-designed prediction systems, and policies learned behaviors that once required handcrafted controllers.

But for a while, the architecture mostly survived. A learned perception module was still a perception module.

Later I saw the other side, on a machine that had to move its whole body through the world. You want it to climb stairs, so you let it climb thousands of them in simulation, every height, every depth. When it meets a real staircase, it just climbs. No branch for this stair or that one. That robustness is real, and the old stack never had it. But what you get is one action, done well. The robot can climb. It has no idea when it should, why, or what comes next.

Two systems, two kinds of rigidity. One could attempt anything and break anywhere. The other could do one thing and never break. In both cases, the higher-level decision still came from us: what should the robot do now? That’s what the current shift is really about.

A few research projects became landmarks for me in seeing where the boundaries of the robot brain might move next.

That changes the architecture itself. The question becomes: where does intelligence live inside the machine?

03 / The world model moves toward the center

A physical agent needs to know how the world behaves: what persists when it’s out of sight, what can move, what can be manipulated, and what an action is likely to change. Recognizing an object doesn’t tell it what happens next.

The model learns how the physical world changes across a broad range of human activity, then transfers that representation across embodiments. Its training experience no longer has to come entirely from the robot that will use it. Once perception, physical prediction, language grounding and action share representations, the borders of the old architecture become harder to draw.

04 / A robot brain can’t live in one frame

Time is another boundary that’s changing. A reactive policy can map the current observation to an action, but it can’t solve tasks that depend on something it saw earlier. The machine has to remember what happened, what disappeared behind an occlusion, which steps it’s already completed, and what changed because of its own actions.

05 / The field is producing two different architectural answers

These systems haven’t converged on one architecture. One direction is to let a multimodal model absorb perception, physical priors, language, memory and action into a shared learned representation. Increasingly general VLAs, world-action models and long-context policies all push in that direction.

The other direction keeps important parts of the world explicit and queryable. The model can consult that structure when it chooses an action.

“End-to-end versus modular” doesn’t describe this split very well. Both approaches use learned models; they differ in what they keep outside them.

06 / Maybe “one giant model” is the wrong mental picture

A model can sit inside a system of memories, structured context, tools, execution interfaces and feedback. Its behavior depends on how that system stores observations, exposes capabilities and checks what happened.

The analogy carries over to robotics: the model chooses what to do, while explicit infrastructure carries out the operation. After living inside both kinds of rigidity, I stopped thinking of the model as the brain.

What the machine can do, what it currently knows about the world, and what it’s trying to achieve have to remain explicit enough that the model can query them and the system can verify them. I now think of the emerging architecture as a learned core surrounded by structure it didn’t learn. The reason is practical: the parts that must be checked, guaranteed, and fixed at three in the morning have to live where a person can reach them.

07 / The stack is being reorganized around the model

The classical stack placed intelligence along a pipeline. Information moved from sensors through increasingly abstract representations until a planner produced a decision and a controller executed it. The systems described here are starting to break that sequence.

Perception, language, memory, prediction and action may increasingly meet inside a shared learned representation. Around that learned core, explicit interfaces may remain for physical capabilities, deterministic control, safety and verification.

Some current systems push more of those outer functions into the model; others pull them back out. The boundary is still moving.

Three changes keep repeating across these systems. Capabilities that used to live in separate modules are sharing models. History is becoming part of the policy. As models take on more of the work, the structure around them is taking the form of memory, tools, world representations, safety boundaries and execution interfaces.

For years, if someone had asked me to draw a robot brain, I’d have drawn something close to the pipeline on the left of Fig. 07.

I wouldn’t draw it that way anymore.

I don’t know exactly which boxes disappear, which ones return, or which become contracts around a learned model. Nobody does yet. But that uncertainty is itself the change.

The architecture of the robot brain has become a research question again.