Home

Directly editing a VLA's weights improved its performance, with no fine-tuning

I ran the π0.5 DROID policy on two manipulation tasks in simulation. Opus 5.5 analysed why it fails at sorting blocks and fixed it by changing its weights directly. No training and no fine-tuning: four attention heads were turned up 3×. Sorting went from 9 to 27 successes out of 30, and a second task the edit was never designed for got better too.

Colour sorting9 → 27successes out of 30
Cups in basket20 → 27not used to design the edit
Weights changed0.05%4 heads, no training
01

Setup

The policy is the public pi05_droid_jointpos checkpoint, used exactly as released. Both tasks run in Isaac Sim with a Franka arm, a Robotiq gripper and DROID's two cameras, one over the shoulder and one on the wrist. Objects are placed slightly differently in every episode.

Colour sorting scene: two grey bins on a wooden table with two blue and two yellow blocks between them
Task 1 · Colour sorting

“Put the blue blocks in the right bin and the yellow blocks in the left bin.”

Objects
4 blocks, 2 movable bins 10.5 cm tall
Success
every block in its own bin
Time limit
120 s
Cups scene: a black basket, two white cups, white boxes, a carton, a bowl and bananas on a wooden table
Task 2 · Cups in basket

“Put both white cups into the basket.”

Objects
2 cups, a basket, 7 other objects
Success
both cups in the basket
Time limit
60 s

The process

  1. 1Run the original modelon both tasks
  2. 2Diagnose Task 1watch the failures, measure what goes wrong
  3. 3Edit the weightsusing Task 1 only
  4. 4Run the edited modelon both tasks, same 30 placements

The edit was designed using only the colour-sorting task. The cups task played no part in it and was only measured at the end. All before and after results in the Results section come from 30 object placements per task that were not used while designing the edit.

Who did what

Me
Set the question, chose the two tasks, decided to focus on sorting, and reviewed the results.
Opus 5.5
Watched the failures and found the cause, designed and ran every experiment, chose the four heads, edited the weights, and ran the before and after tests. It made the edit by writing and running code that tried candidate changes and applied the one it chose.
02

Results

Task 1 · Colour sorting

Before · original weightsAfter · 4 heads ×3
Before · original weights After · 4 heads ×3
Same object placement, played at 2× speed. Before: the original model catches the bin rim with the second yellow block and drags the bin off the table. After: the edited model lifts higher and sorts all four blocks in 29 s.
Successful episodes (out of 30)
Before
9
After
27
Episodes where a bin was knocked off (out of 30)
Before
16
After
0
Colour sorting · 30 episodesBeforeAfter
Successful episodes927
Blocks sorted, average out of 42.93.9
Episodes with a bin knocked off160
Typical time of a successful episode40 s31 s

Task 2 · Cups in basket

This task was not used at all while designing the edit; it was run at the end only to check nothing broke. It improved instead. Cups are carried higher, as with the blocks, and episodes finish sooner. The gain is smaller than on Task 1.

Before · original weightsAfter · 4 heads ×3
Before · original weights After · 4 heads ×3
Same object placement, played at 2× speed. Before: the original model knocks the second cup over and runs out of time. After: the edited model carries both cups high and finishes in 24 s.
Cups in basket · 30 episodesBeforeAfter
Successful episodes2027
Average episode time44 s32 s
How high cups are carried (typical)23.5 cm28.6 cm
Episodes where the robot grabbed another object1212

The robot still sometimes picks up a white box instead of a cup; the edit does not change that.

03

How Opus 5.5 found the fix

The problem: blocks are carried too low

Opus 5.5 first checked whether the model mixes up the colours. It doesn't: when both bins are on the table it puts almost every block in the right one. The real failure is mechanical. The robot carries the block low, the gripper catches the rim of the bin, and the arm drags the whole bin off the table. After that the task can't be finished.

Height in cm 0 4 8 12 16 bin rim · 10.5 cm Carries that hit a bin: block peaked 8–12 cm above its start carries that hit a bin 8–12 cm Normal carries: block peaked 11–15 cm above its start normal carries 11–15 cm
Peak height of the block, in cm, above where it started on the table, for each carry by the original model. The carries that hit a bin barely reach the rim.

So the fix is simple to state: carry the block higher, without changing how the robot grasps.

The fix: four “lift higher” dials

π0.5 is built from many small parts called attention heads, 288 of them in total. Each one adds its own small contribution to the robot's next move.

  1. 1Test each headOpus 5.5 turned each head off, then doubled it, and measured one thing: how high the robot carries the block.
  2. 2Find the dialsFour heads act like a dial. Turn them down and the carry drops; turn them up and it rises. Grasping stays the same.
  3. 3Turn them upMultiplying all four by 3 raises the carry by about 2.3 cm on average, which lifts most carries clear of the rim. In the full tests no bin was knocked off in 30 episodes.
Four heads turned upCarry heightGrasp height
×2+1.4 cmno change
×3 (used)+2.3 cmno change
×4+3.0 cmstarts to change

Measured while designing the edit, on recorded inputs from the original model. The test results are in the Results section above.

This changes 1.57 million of the model's 3.35 billion weights. The edited model is saved as an ordinary checkpoint and runs with the standard code, with no extra hooks at inference. The tests show what these heads do to the robot's movement; why the model computes it this way internally is a separate question.

Where the edit lives in π0.5

π0.5 is two connected transformers. A large one (2B parameters) reads the camera images, the instruction and the robot's joint positions. A small one (300M, the “action expert”) reads what the large one produced and outputs the next second of movement. Each has 18 layers with 8 heads per layer. Two of the edited heads are in each.

Shoulder cameraimage → 256 tokens
Wrist cameraimage → 256 tokens
Textinstruction + joint positions
↓
Vision-language model · 2B · 18 layers × 8 headsedited: layer 6 head 4, layer 10 head 7
↓
Action expert · 300M · 18 layers × 8 headsedited: layer 13 head 4, layer 14 head 1
↓
Next second of movement15 steps of 7 joint targets + gripper
Vision-language model heads
Action expert heads
edited (output ×3)unchanged
04

Limits

  • One scene per task, in simulation. Other scenes and a real robot are untested.
  • The edited model sometimes puts a blue block in the wrong bin: 4 blocks across its 3 failed sorting episodes. The original rarely did this while both bins were on the table.
  • Opus 5.5 found the heads by testing many candidates against one measurement, not by guessing in one shot.