Physical AI: How Agents Are Changing Robot Learning

TL;DR: Physical AI helps robots learn from their own mistakes and pass successful motions to other machines. An AI agent can investigate a failure, propose a correction, and save the verified result. Most of these systems still live in labs. Their goal is clear: one attempt should create experience for an entire robot fleet.

An AI agent connects the goal, the robot’s motions, and the outcome of each attempt.

A robot reaches for a mug, clips it with the edge of its gripper, and drops it. An engineer would usually review the recording, find the cause, change the program, and start another attempt.

An AI agent can now take on part of that investigation. It sees the camera image, reads the sensors, finds the moment of failure, and proposes a different motion. A successful correction can be saved and passed to other robots.

Here, Physical AI means a system that perceives the world, plans an action, moves through a robot body, and uses the physical outcome for learning. A polished robot video proves only that someone captured one take. A real skill has to survive a new camera angle, a worn actuator, an unfamiliar object, and a full production shift.

Three technologies came together in 2026: models connected images and instructions with motion, AI agents learned to investigate failures, and robot fleets began returning their attempts to shared training.

Physical AI runs at three speeds

“Place the part in the bin” passes through three layers. At the top, an agent understands the goal and selects a skill. Next, a vision-language-action model, or VLA, turns an image and instruction into a short motion. Fast controllers at the motors maintain balance, contact, and grip force. Gemini Robotics 2 splits those roles across planning, execution, and local models.

The body must react in milliseconds. In Helix 02, the model tells the joints where to move 200 times per second, while the controller corrects motion 1,000 times per second.

Anthropic’s robotics experiments showed the gap. Direct control of a Unitree Go2 required about 83 commands per second, while one language-model response took 2.5 to 5 seconds. The researchers paused the simulation while they waited. In a continuous manipulation experiment, the models fully completed only 0% to 5.5% of the tasks.

Physics keeps moving while the model thinks. The agent owns the goal and plan, controllers handle the motors, and a separate safety system owns the emergency stop.

A failure starts the next attempt

A manipulator inserts a part into a slot, the edge catches on the housing, and a force limit stops the motion. The camera preserves the part’s position, sensors record the contact, and the controller reports which limit fired. An agent can connect those signals and try another angle.

Code as Policies proposed building behavior from allowed commands: find the object, approach it, lower until contact. Engineers can read the program and test it in simulation. The authors of CaP-X removed convenient high-level commands and left models with basic ones. Performance fell across all 12 models. Some of the robot’s “intelligence” came from the toolbox built by engineers.

ASPIRE closes the loop: an agent writes a program, runs it, reads the trace, corrects the failure, and saves the successful version. On new long tasks, the system completed 31% of assignments, compared with 4% for the other methods. Most of the evidence comes from simulation, with preliminary real-robot results.

RHO moves the search into simulation and sends a fixed program to the robot. Its authors report a 70% success rate with no language-model calls during execution. This result is also limited to simulation. Free-form learning on a factory floor is still far away: the agent needs room to experiment and a boundary beyond which the machine runs only a tested release.

Motion needs a memory

One successful attempt is weak evidence. The object may have been placed more conveniently, an operator may have helped, or a fresh calibration may have hidden a problem. A useful record stores the task, model and robot versions, commands, sensor readings, human help, safety events, and outcome.

The video below is one illustration. It tells us nothing about the platform’s autonomy, but it shows the density of physical experience.

This history turns motion into memory another machine can trust. I call it a registry of physical experience: the agent connects a failure with its correction and passes verified constraints forward.

One robot teaches the rest

A shared record format turns one robot into part of a larger experiment. Over seven months, AutoRT collected 77,000 real-world attempts with a fleet of up to 52 robots and proposed 6,650 tasks. The results returned to a shared dataset.

π0.7 adds successful and failed robot attempts to human demonstrations. Failures near a skill’s edge are especially useful: a new camera angle, different weight, reflective surface, or delayed camera. The agent groups failures and selects the next test that should reveal the most information.

Without reliable checks, a fleet produces garbage at industrial speed. Hundreds of machines can repeat one mistake and treat an accident as success. Weak candidates should be removed before they reach a real robot.

The robot rehearses before it acts

A world model sees the current scene and produces several continuations, almost like rough videos of the future. The agent compares them and sends the strongest candidates to physical testing.

A world model generates several continuations after a failure. The real machine tests the strongest candidate.

Hi-WM starts from a real image, returns to the moment of failure, and generates corrected continuations. Across three real-world tasks and one dual-arm setup, the original model’s result improved by an average of 37.9 percentage points. The technology still needs to prove its range.

A world model can draw a convincing, physically impossible future. Virtual rehearsal saves attempts; only a test on the real machine turns a candidate into a skill.

The factory separates a skill from a good demo

A factory adds wear, dirty cameras, network delays, people, and recovery after a stop to the laboratory scene. An independent audit of five robot-manipulation tests showed how a model can use clues from the exam itself. In LIBERO, a compact model with no language component approached published leaders through the appearance of scenes and the order of tasks. Only about one fifth of reported improvements in LIBERO and SimplerEnv could be separated confidently from random variation.

For production, a more useful metric is the cost of one reliably completed action, along with human interventions, protective stops, recovery time, and transfer across robots.

According to BMW Group, Figure 02 logged about 1,250 operating hours during a ten-month pilot, moved more than 90,000 components, and supported production of more than 30,000 BMW X3 vehicles. The robot performed one narrow sheet-metal handling operation during ten-hour shifts, five days a week.

The BMW numbers required infrastructure: IT, safety, and logistics teams, better 5G coverage, a redesigned work area, and physical barriers. ASIMOV-Agentic separately checks refusal of prohibited tasks, safe stops, and requests for help. Emergency interlocks remain independent of the agent. ISO 10218-1:2025 covers industrial robots, not every kind of machine.

Testing follows a skill from simulation through a production shift. Only then can the motion return to the factory and enter the training data for other robots.

Faster verification means faster learning

An agent sets a task, a robot makes an attempt, and the system connects the outcome to a specific model and machine. A successful correction passes the test stand and enters the next skill release.

Robots produce attempts in parallel. Verification determines how many become useful experience.

One hundred robots can produce 100 times more recordings. At the old pace of failure analysis, the volume of reliable experience barely grows. The agent creates leverage here by grouping failures, proposing a cause, and preparing the next experiment.

This lesson has a technical name: a verified physical episode. It records the data source, model and robot versions, success criterion, human help, and approved conditions for use.

The meaning of Physical AI is straightforward. One machine fails, the system tests a correction, and the next robot does not have to repeat the entire journey from scratch.