On Push-T, Failure Demonstrations Are the Ingredient That Makes Entropy Guidance Work
Two conditions were tested: Standard (agent starts in a normal position) and Perturb (agent
starts directly above the block — the setup that, without correction, drives the block away
from the goal). EFP was trained two ways: with a small set of labeled failure trajectories added to its
entropy estimator, and without.
Where the Demonstrations Actually Come From
Neither the "success" nor the "failure" demos are human or learned — both are a hand-written script
that walks the pusher in a straight line toward a fixed sequence of pixel targets a programmer chose in
advance.
detour left/right, clear of block→
go around, below block→
line up behind it→
rise up beneath it→
push straight up, through the goal
That's the "success" route — run twice, mirrored (approach from the left or the right), with a
hand-tuned sideways offset at the final push. Pushing an off-center T-shape makes it spin, so that offset
was tuned by trial and error to keep the block from rotating away from the goal as it's pushed into
place.
The "failure" route is a single bad move by the same script: start directly above the block and
walk straight down. Since the goal sits above where the block starts, this drives it further away, not
closer.
What "Success" Actually Measures
The block always spawns already facing the same angle as the goal — only its position is
wrong at the start. So the script's job is really just "translate the block upward without spinning
it," and success is graded by plain shape overlap: coverage = intersection_area(goal, block) /
goal_area, crossing a 0.35 threshold. There's no notion of "align the T's edges" as a goal in
itself — a well-aligned push just happens to produce high overlap, because the rotation was
already correct going in.
Success % (coverage > 0.35) under both conditions. Standard: EFP+failure 64%, Unguided Flow Policy 24%, everything else 0%. Perturb (adversarial start): EFP+failure 56%, every other method 0%.
Why failure demos matter
Without them, EFP's entropy gradient has never seen what "getting worse" looks like, so under
perturbation it has no signal to correct course, and its guidance term actively fights the flow prior.
Coverage collapses to 0.000 in both conditions, worse than the same network with guidance
switched off (Unguided Flow Policy: 0.143 / 0.010).
Add failure trajectories, and the same architecture becomes the only method that survives the
adversarial start: 64% / 56% success, 0.359 / 0.285 coverage, versus ≤24% / 0% for every baseline.
Standard start
rollouts for all five methods, same seed family
Perturb start (adversarial)
agent starts directly above the block
Each card is a single representative rollout, not a cherry-picked one — for a method scoring, say,
24%, the saved example may well be one of the failing runs. Success itself is defined as coverage
exceeding 0.35 at any point in the 350-step episode, not the block's final resting position, so the
EFP (w/ failure) clips above are trimmed to end at that peak-coverage moment rather than playing out the
full episode (where the policy continues moving and the block can drift again afterward).
How Sensitive Is This to the Guidance Scale?
A separate ablation applies entropy guidance on top of plain BC (rather than a diffusion sampler) and
sweeps the guidance scale from 0.05 to 5.0.
Entropy-guided BC only works in a narrow band around guidance-scale 0.05–0.1 (coverage 0.31–0.41, up to 100% success in Standard); above ~0.3 it collapses to 0% in both conditions. The entropy gradient is a knife-edge correction, not a signal that tolerates being over-applied.
Does This Replicate on Real Human Demonstrations?
Everything above uses a hand-scripted synthetic expert. Re-running the same comparison on the official
Push-T dataset (206 human-teleoperated episodes) tells a different story:
On the real dataset, EFP (w/ and w/o failure, w/ and w/o guidance) and plain Diffusion Policy all reach ~100% success in both Standard and Perturb. Only BC (Mean) and MDN-BC fail — BC's coverage holds up standard (100%) but collapses under perturbation (13%); MDN-BC fails outright.
Standard start
Perturb start (adversarial)
The honest caveat
The baseline actually called "Diffusion Policy" in the literature — a chunked
ConditionalUnet1D trained with DDPM — is exactly what's shown just above, where it
reaches ~100% on the real dataset. The synthetic-task comparison earlier on this page does not
use that architecture: its baseline (labeled "Unguided Flow Policy" throughout this page) is literally
the same lightweight, single-step flow-matching network as EFP itself, with the entropy-guidance term
switched off — no observation-history stacking, no action-chunking, a fresh noise sample drawn
independently at every timestep. Without temporal consistency, nothing stops it from sampling the left
detour at one step and the right detour a few steps later.
So the synthetic headline result is really guided beats unguided — an ablation of EFP
against itself — not a win over the real Diffusion Policy algorithm. That's still a meaningful,
isolated finding about what the guidance term contributes. It's compounded by demonstration diversity
too: 300 hand-scripted episodes are narrower than 206 human ones, which is a second reason the
unguided baseline struggles here but real Diffusion Policy, above, doesn't.
