OpenVLA vs OpenVLA-OFT: A Local Benchmark on LIBERO-Spatial and Robosuite

01 · How OpenVLA Works

How OpenVLA Turns a Picture Into a Motion

OpenVLA is built on a Prismatic vision-language backbone: a frame from the robot's camera is encoded by two fused vision towers, DINOv2 and SigLIP, and projected into the token stream of a 7B-parameter Llama 2 language model alongside the tokenized instruction — e.g. "In: what action should the robot take to pick up the red cube?"

The model was never trained to output text back. Instead, its 7-dimensional continuous action — delta position (x, y, z), delta rotation (roll, pitch, yaw), and gripper open/close — is discretized into 256 bins per dimension, and each bin is mapped onto one of the least-used tokens in the LLM's existing vocabulary. Producing an action means generating 7 of these tokens the same way the model would generate 7 words: one forward pass through the full 7B network per token, each token conditioned on the ones before it.

That autoregressive loop is what makes OpenVLA a strong generalist — it inherits the flexibility of an LLM decoder — and also what makes it slow: 7 sequential 7B forward passes for a single timestep of control, then the whole thing repeats for the next camera frame.

02 · What OFT Changes

Parallel Decoding Instead of Seven Sequential Passes

OpenVLA-OFT is not a new pretrained model — it's a fine-tuning recipe applied to the same OpenVLA backbone, aimed squarely at the two things that make the original hard to deploy: it's slow, and discretizing continuous motion into 256 bins throws away precision the base model never gets back.

The mechanism that changes between the two: OpenVLA feeds each predicted action token back in as input for the next, seven times per timestep; OFT reads the same image once and writes out several timesteps of continuous action in a single pass.

Property	OpenVLA	OpenVLA-OFT
Action decoding	Autoregressive — 7 tokens, sequentially	Parallel — one non-causal forward pass
Action representation	256-bin discretized tokens	Continuous, via L1 regression (or diffusion) head
Actions per query	1 timestep	A chunk of several future timesteps at once
Language grounding	Token-level cross-attention only	+ FiLM conditioning injected into vision features
Ships zero-shot?	Yes — generalist across many robots	No — needs fine-tuning per robot/task setup
Typical control rate	Roughly a few Hz	Reported ~25× higher throughput; real-time on real arms
LIBERO avg. success*	Autoregressive fine-tune, mid-70s%	~97% across all four LIBERO suites

* Figures as reported in the OpenVLA and OpenVLA-OFT papers, for context — not this run. This run's own numbers are in the Results section below.

03 · Setup

What Actually Ran, Here

Two separate setups on this machine, evaluating the two ends of the table above:

	OpenVLA (zero-shot)	OpenVLA-OFT (fine-tuned)
Checkpoint	`openvla/openvla-7b`	`moojink/openvla-7b-oft-finetuned-libero-spatial`
Sim / task	robosuite — `Lift` (pick up the red cube)	LIBERO-Spatial — 10 pick-and-place tasks
Action un-normalization	`bridge_orig` stats (nearest available match — not sim-specific)	Trained specifically for this suite
Episodes run	4 (this reel)	30 (all 10 tasks × 3)
Project	`agenticVLA/demo.py`	`openvla-oft / run_libero_eval.py`

Not an apples-to-apples benchmark

These are two different simulators, two different task families, and OpenVLA never saw LIBERO or robosuite during training — it's improvising with the closest action statistics available (BridgeV2). The comparison below is honest about that: it's "generalist zero-shot" vs. "specialist fine-tuned," which is exactly the trade-off Section 2 describes, not a controlled ablation on one task.

04 · Results

Three Points on the Same Curve

0/4

RANDOM POLICY · LIFT

0/4

OPENVLA ZERO-SHOT · LIFT

29/30

OPENVLA-OFT · LIBERO-SPATIAL

96.7%

OFT OVERALL SUCCESS RATE

No policy, a generalist policy outside its depth, and a policy fine-tuned for exactly this task:

baseline

Random actions — sanity-check of the sim pipeline, no model. Franka arm flails; cube untouched.

stalls

OpenVLA-7b, zero-shot — frontview / sideview / agentview, stitched. Trained on real BridgeV2/RT-X footage, not this sim; reaches, doesn't close the loop.

succeeds

OpenVLA-OFT, fine-tuned — compiled reel from the LIBERO-Spatial eval below. 96.7% success over 30 episodes.

What the simulator actually looks like: five render angles of the robosuite Lift environment used for the OpenVLA zero-shot runs above. Only agentview is fed to the model; the rest are recorded for human inspection.

OpenVLA-OFT Across the LIBERO-Spatial Task Set

Four individual episodes pulled from the full 30-episode run, including the one failure recorded — nothing cherry-picked out.

success

Ep. 1 — bowl between plate & ramekin

success

Ep. 7 — bowl from table center

success

Ep. 13 — bowl in top drawer of cabinet

fail

Ep. 16 — bowl on the ramekin, the one miss

05 · Reading the Result Correctly

What This Does and Doesn't Show

This is not "OFT is 97% better than OpenVLA." It's a specialist fine-tuned for LIBERO-Spatial against a generalist checkpoint that has never seen either simulator. Fine-tune OpenVLA itself on LIBERO and the gap narrows a lot — the OFT paper reports the autoregressive fine-tune lands in the mid-70s% on the same benchmark, still well behind OFT's ~97%, but nowhere near zero.
The zero-shot failure is a domain-gap story, not a capability story. OpenVLA is reaching for the cube; the un-normalization statistics come from real BridgeV2 demonstrations, and MuJoCo's rendering, physics, and camera geometry don't match what those statistics were tuned for.
OFT's real win in the source paper is speed, not just accuracy — parallel decoding plus action chunking is what let a fine-tuned OpenVLA run fast enough for real-time control on a physical ALOHA bimanual rig, something the token-at-a-time original could not keep up with.