When AI Gets a Body: How Agents Are Changing the Way Robots Learn

A planning agent, fast control loops, and a registry of physical experience operate as one machine.

A manipulator receives an instruction to insert a part into a slot. The camera finds it, the model plans the motion, and the gripper approaches the assembly. The last eight millimeters break the plan: an edge catches on the housing, force climbs, and the controller stops the arm.

An error like this has coordinates, mass, a telemetry trail, and a cost. Sometimes it wastes a run. Sometimes the gripper damages the fixture.

In 2026, an AI agent can already analyze the episode, revise the robot’s behavior, and launch the next attempt.

Three technologies have converged to make this possible. Vision-language-action (VLA) models connect images, instructions, and movement. Reasoning agents plan long-horizon tasks. Robot fleets feed their own attempts back into training.

That convergence is creating agentic robotics. The agent has access to cameras, models, a skill library, and permission to start an experiment. Its most valuable work begins after the first failure: turning a physical event into verifiable experience.

A command travels from words to motors

“Place the part in the bin” takes a second to say. During execution, the robot operates across three time scales at once.

At the top, an agent interprets the goal, breaks it into steps, and selects a skill. Seconds of reasoning can be acceptable here. Below it, a VLA model receives a fresh image and produces a short sequence of movements. Gemini Robotics 2 makes this division explicit: ER 2 manages the multistep task, the VLA executes physical skills, and On-Device 2 runs locally for latency-sensitive actions.

At the body, the clock advances in milliseconds. In Helix 02, the visuomotor policy produces joint targets at 200 Hz, while the whole-body controller updates commands at 1 kHz.

The gap is especially clear in Anthropic’s robotics experiments. Real-time control of the Unitree Go2 would have required roughly 83 updates per second, while a single model call without extended reasoning took about 2.5 to 5 seconds. The researchers paused the simulator between responses. The reported task performance is therefore an upper bound measured outside real-time control.

They ran a separate experiment with a fixed-base arm and no pause. The arm did not have to maintain its balance continuously. Under direct model control, the models fully completed between 0% and 5.5% of the LIBERO tasks.

The agent owns the goal and the order of operations. Controllers maintain balance, contact, and force. An independent safety loop enforces the work envelope and owns the emergency stop. Authority narrows and the feedback loop shortens as a command moves closer to the motor.

Physics does not wait in line for a language model.

The failure travels back up

A failure signal travels from sensors and controllers to the layer that can change the next action. That adaptive capability can live in three places.

Inside the control model. A VLA learns the motor skill end to end and produces motion directly from an image and instruction. The correction enters the model weights, which makes its cause harder to isolate and test independently.

Inside the program. In Code as Policies, a language model composes behavior from a set of allowed functions. The change is readable, testable, and reusable. Its range is limited by the functions the system exposes.

Inside the interface. VIA gives a general-purpose model a virtual gripper and a small toolset. The model observes the result of each step and revises the action through the same interface.

Interface design has a measurable effect. In CaP-X, performance across 12 models generally declined as convenient high-level functions were removed. VIA achieved 96.7% across three standard tasks and 40% on precision insertion. VIA’s reported evaluations ran in simulation, and a single attempt could take up to an hour.

ASPIRE implements the full debugging loop: an agent writes a program, runs it, reads the trace, diagnoses the failure, and verifies the fix. A successful version enters the skill library. On new long-horizon tasks, the authors report a 31% success rate compared with 4% for prior methods. Evidence from real robots is still preliminary.

In the opening example, a verification module records an incomplete insertion. The force sensor shows early contact, while the camera confirms the correct part position. The trace localizes the error to the approach angle.

The agent adds lateral alignment, slows the final few centimeters, and checks force before the last motion. After a successful attempt, the library gains a skill with the illustrative name insert_with_edge_alignment, along with its pose tolerance, force limit, and completion criterion. One line now carries a verified physical history.

Online correction can be expensive: in one experiment, the agent made as many as 66 model calls per attempt. RHO searches for a program in simulation and deploys frozen code to the robot. The authors report a 70% success rate without model calls during execution; the results remain limited to simulation.

Self-learning easily turns into marketing fog. A deployable architecture handles the current failure, the growing skill library, and training of the next model version as separate processes. Every loop needs a complete record of the physical episode.

Every attempt needs a paper trail

A robot can complete a task by accident. The gripper slips into a lucky position, an operator adjusts the object, or the table sits a few centimeters closer than usual. A success=true flag erases those circumstances.

52 robots turn motion into data

Over seven months, AutoRT collected 77,000 real-world episodes with a fleet of up to 52 robots. The system proposed 6,650 unique tasks.

This is a data factory: one model describes the scene, another proposes a task, safety rules filter the instruction, the robot acts, and the result returns to the shared dataset. Training for π0.7 already combines human demonstrations, failed autonomous attempts, and data from outside robotics.

Repeatedly dropping the same cup provides poor coverage. The agent has to search the boundaries of a skill: a new camera angle, a different weight, a reflective surface, sensor delay, or a partially occluded object.

Collection scales faster than verification. A weak criterion admits coincidences, repeated failures, and unsafe shortcut strategies.

A fleet with weak verification scales garbage.

A world model rewinds the failure

Physical experience is expensive: a robot moves in real time, occupies a test cell, and wears down its hardware. A world model can play out a possible continuation from the current state and a chosen action before the robot moves.

A world model rewinds the failure and branches possible corrections. Physical verification assigns the final outcome.

Hi-WM begins a virtual attempt from a real observation. A person intervenes near the failure, and the system rewinds the scene to produce several continuations. Across three tasks and one dual-arm setup, the authors report an average improvement of 37.9 percentage points over the original control model.

Prediction error compounds with the length of an attempt. Models simplify contact physics. An unusual material can look plausible on video and behave strangely on the table. A world model can invent fake physics with complete confidence.

The world model therefore acts as a candidate generator. Promising trajectories pass through a simulation with physical constraints, followed by a bounded run on the test stand. Physical verification assigns the final status and measures the gap between simulation and reality.

A benchmark can mistake a shortcut for a skill

Reported robot model scores often depend on the structure of the test itself. An audit of five popular manipulation benchmarks found hidden shortcuts, statistically weak improvements, and overfitting to evaluation procedures.

In LIBERO, a compact 90-million-parameter model with no language component approached reported leaders by exploiting visual regularities and task order. In CALVIN, randomly moving blocks within a familiar range caused a meaningful performance drop. An experiment can easily confuse regularity in the scene with a general skill.

Google’s published Gemini Robotics 2 results show a wide spread even within one model version. Franka Duo with a two-finger gripper scored 74.2%, 78.9%, and 89.6% across three task categories.

The Apptronik Apollo 2 configuration with SharpaWave hands ranged from 32% when using a dustpan to 92% when unscrewing a light bulb; its other tasks scored 36%, 40%, and 44%. Another Apollo 2 configuration with Inspire hands scored 45.7%, 68.4%, and 76.3% across three whole-body tasks.

These scores are difficult to compare directly. For grasping and whole-body motion, Google reports averages across several tasks within a category. For SharpaWave, it reports individual tasks. The charts include error bars, while the page does not explain how they were calculated, how many trials were run, or the full task composition of each aggregate category.

A single mean can hide the real shape of a skill.

For Physical AI, I would put cost per verified successful episode at the top of the scorecard. It should sit alongside confidence intervals, intervention rate, safety stops, recovery time, transfer across robots, and degradation after the environment changes. These metrics connect model quality to operating cost. I cover the measurement framework in more detail in Evaluating LLM Agents in Production.

The factory tests a skill one shift at a time

A polished demo lasts seconds. Reliability is measured in shifts, stops, and recovery after failure.

BMW reported that Figure 02 logged about 1,250 operating hours, moved more than 90,000 parts, and supported production of more than 30,000 BMW X3 vehicles. The robot worked ten-hour shifts, five days a week.

Those hours required infrastructure. The deployment involved IT, occupational safety, process management, and logistics. BMW improved 5G coverage, reconfigured the work area, and developed a safety concept with physical barriers and partitions.

When a robot stops halfway through a shift, an engineer must quickly distinguish a displaced part from a sensor, model, or mechanical failure. The log connects the stop to the model version and machine state. A new release reaches a small group of robots first, while the previous version remains ready for rollback.

ASIMOV-Agentic separately evaluates refusal of prohibited tasks, protective stops, requests for help, and responses to infeasible instructions. Machine safety depends on hard constraints, interlocks, and risk assessment under standards such as ISO 10218-1:2025. A generative model operates inside those boundaries.

Verification sets the pace of learning

After each shift, a mature system retains a skill library, an episode dataset, verification rules, and safety boundaries. Together, they form the fleet’s memory. Once validated, a correction for one physical failure can reach dozens of machines.

Verification speed constrains the entire loop. One hundred robots can increase the flow of raw episodes by a factor of 100. If failure analysis and test-stand capacity stay fixed, the volume of reliable experience barely moves.

The fleet produces raw episodes in parallel. A narrow verification pipeline controls how quickly motion becomes knowledge.

This is where an agent creates the most leverage. It clusters failures, replays traces, proposes a diagnosis, designs the next experiment, and routes ambiguous cases to a person.

The defining shift in 2026 is happening here. Robotics is learning to store physical experience as an engineering artifact, complete with provenance, version, success criteria, and a path to specific hardware.

At the beginning of this article, the part jammed with eight millimeters left to travel. After analysis, the failure becomes a skill, the skill passes test-stand verification, and the result enters a new release. Every compatible machine receives the force limit, approach angle, and completion check before making its own first attempt.

This is how AI gets a body, with accountability attached to every movement. Each new machine starts with experience inherited from the fleet.