Skip to main content

Diffusion Policies in Robotics: Why Robot Actions Are Becoming Generative

8 min read
technology robotics
robotics robot-learning diffusion-policy imitation-learning

Updated

Real-world Push-T task with diffusion policy

Robotics has many new names now. Vision-language-action models, world models, foundation policies, generalist robots. Diffusion policy is one of the ideas that started to sit inside this bigger wave, and I think it is useful to understand even if your background is more control than pure AI.

At first I thought diffusion policy is maybe just another fashionable name. But after reading more, it seem more serious than that. The main idea is to generate robot actions by a denoising process, not by one direct mapping only.

In simple words, the robot starts from a noisy action sample and the model keep cleaning it step by step until it become a useful action sequence.

This is why the word diffusion appear here. It is not only for image generation anymore. It can also be used for action generation.

The simple idea

In a normal learned policy, we often think like this:

a=π(o)a = \pi(o)

The policy sees observation (o), for example camera image and robot state, then gives one action (a). This is simple to imagine.

Diffusion policy is different. It tries to model an action distribution and then sample from it by denoising. A very simplified version can be written like this:

ak−1=fθ(ak,o,k)+σkza_{k-1} = f_\theta(a_k, o, k) + \sigma_k z

Here (a_k) is a noisy action at denoising step (k), and (o) is the observation. This equation is not the full theory, but it show the main engineering picture.

The action is refined many times, not guessed only once.

Simplified diagram of diffusion policy denoising loop

One reason this is useful is multimodality. Robot tasks usually have more than one good action. A mug can be grasped from different directions. A block can be pushed from different angles.

If a policy average all good choices, sometimes it produce a bad middle action.

Diffusion policy can keep these choices more separate. It can choose one mode and continue with it. This is small sentence, but for manipulation it is a big thing.

Why people cared about it

The first important paper showed diffusion policy for visuomotor robot learning. It used a conditional denoising process for robot actions, and tested the method on simulation and real manipulation tasks.

The official project page also highlight receding horizon control, visual conditioning, and action sequence prediction.

This part is important for control people. The policy does not only produce one action and disappear. It can produce a chunk of future actions. Then robot executes a part of it and plans again.

So it is not model predictive control, but the feeling is a little close to rolling horizon thinking.

For example, in the Push-T task the robot must push a T-shaped block to a target. It is contact-rich, and small errors can change the object motion. Intel also use this task in their embodied intelligence documentation, with both image-based and low-dimensional variants.

Push-T simulation task from Intel diffusion policy documentation

Another interesting point is that diffusion policy did not stay only as one paper. Octo, a generalist robot policy project, also uses diffusion decoding for actions. Octo was trained on a large robot dataset and tries to work with flexible observations, task definitions, and action spaces.

This means diffusion action generation became a part of larger robot-learning direction. Not always the main story, but it is inside the story.

Octo generalist robot policy teaser image

Where it is still annoying

I do not think diffusion policy is a clean final answer.

The first problem is speed. Denoising takes several steps. In robotics this is not only a software detail. If the action comes late, the robot may already be in another condition.

This is more serious when the task is dynamic, or when the robot is limited in compute power.

This is why One-Step Diffusion Policy is interesting. The idea is to distill the multi-step diffusion policy into one-step action generator. Their page says this can improve the action prediction frequency a lot, from about 1.5 Hz to 62 Hz in their reported setting.

For me this number shows the main weakness clearly: original diffusion policy can be powerful, but it can be slow.

One-step diffusion policy distillation pipeline

Another problem is data. Diffusion policy is often closer to imitation learning. It needs good demonstrations. If the demonstrations are narrow, noisy, or not close to real deployment, the policy can learn a narrow behavior too.

It is not magic.

Also, stability and safety are not automatic. A diffusion policy can give a nice action proposal, but the robot still need actuator limits, feedback, state estimation, saturation handling, and safety checks.

The motors do not care that the method is new.

My own reading from control side

For me, the useful reading is not:

diffusion will replace control

That sentence is too simple.

A better view is that diffusion policy can sit above a controller. It can propose an action chunk or a trajectory. Then a classical or model-based controller can track it. There can also be a safety layer between them.

Diffusion policy inside a robot control stack

This is less exciting as a headline. But it is more believable for engineering.

I think this is especially true when robot has strong nonlinear dynamics, contact, or human interaction. In those cases the policy can help with complex behavior, but low-level control and physical limits still matter.

How it can connect to my field

My own background is more about robot control, dynamic simulation, and learning-based compensation. In my M.Sc. thesis, I worked on a cable-driven lower-limb rehabilitation robot. I used computed torque control, then added residual DDPG to reduce tracking error under disturbances and model uncertainty.

So I usually think about learning as a layer beside model-based control, not always instead of it.

For rehabilitation robotics, I would be careful with diffusion policy as the deepest controller. The robot may interact with a human leg, and safety and repeatability are very important.

But I can imagine diffusion policy being useful in higher level. For example, it could generate smooth assistance trajectories, or suggest motion chunks from sensor data, while a safer controller still track the reference.

For cable-driven robots, this may also be interesting. Cable tension limits, feasible workspace, and nonlinear dynamics are not simple.

A diffusion policy could maybe suggest feasible motion patterns if enough good data exist. But I would still want a physics-based layer to check cable tensions and tracking constraints.

For mobile robotics, I see another use. Diffusion policy may be useful for local maneuver generation, like choosing a smooth action sequence when the perception is uncertain.

But for terrain cost, stability, and hard constraints, I still trust explicit planning and control more. Maybe the best system is not one method. It is a stack.

This is why the topic is useful for me. It connects modern robot learning with the question I already care about:

what should be learned, what should be modeled, and where we need safety boundaries?

My view

I don’t think diffusion policy is only a trend word. It is one useful direction in robot learning, especially when the robot action is complex and there are many possible good motions.

But I also don’t think it removes the need for control. In a real robot, action generation is only one part. The robot still need feedback, constraints, safety checks, and physical testing.

For me, the interesting future is not a pure learning system or a pure classical system.

It is probably a stack.

Some part are learned from demonstrations. Some part are designed from physics. Some part are safety filters. Some part are still normal feedback control.

Maybe this is not the most dramatic answer. But it is the answer that feels more realistic for robots.

Image credits

Sources and further reading