OETGBench

Ordered Event-Transition Generation

Beyond Plausible Motion

Benchmarking and Improving Ordered Event-Transition Generation in Few-Step Autoregressive Video Models

Anonymous research demonstration

2,500 benchmark tasks4 event-transition families3 observable phases

From plausible frames to a realized transition

Few-step autoregressive video generation has emerged as a promising paradigm for low-latency streaming synthesis, offering a practical foundation for interactive simulation, embodied agents, and real-time world models. However, these rollout-based distillation methods primarily optimize low-level denoising objectives or distribution matching with the teacher, rather than explicitly enforcing the correct temporal realization of requested event transitions.

We define an event transition as a temporally evolving and visually observable process through which a scene, entity, or interaction departs from an identifiable initial state, undergoes task-relevant intermediate changes, and establishes a target outcome.

01 Onset02 Progression03 Outcome
A bread-baking task with onset, progression and outcome expectations, alongside generated frame sequences from three autoregressive models.
Visual plausibility does not imply ordered event completion. The loaf already appears baked in the first frame, while the requested sequence of scoring, oven insertion, baking, and removal is not fully depicted.

The evaluation framework

OETGBench & OETGEval

Beyond prompts that clearly delineate event transitions, OETGBench provides phase-specific descriptions of successful completion and targeted quality-assessment questions to support fine-grained evaluation.

01 / Tasks

OETGBench

Original generation prompt

Initial state
Requested event transition
Target outcome
OnsetProgressionOutcome

Phase-specific completion descriptions and evaluation questions

02 / Evaluation

OETGEval

Video-level Diagnostics (VLD)Full-video quality and transition-related failure modesVLD Quality ↑   VLD Failure ↓
Window-level Phase Verification (WPV)Independent phase judgments in four overlapping windows
W₀W₁W₂W₃
Ordered phase matching
Phase Presence Score · Event Transition Score · Temporal Order
03 / Post-training

AG-P3T

Frozen-base reference video
Task-matched anchor

Event and phase/order feedback
+ anchor-relative penalties
Positive-advantage gate
Controls positive sample contributions

LoRA optimization
Not post-update acceptance or rollback

Framework re-laid out for the web using the current manuscript terminology. Temporal Order describes coarse temporal compatibility across overlapping windows, not frame-exact ordering. VLD diagnostics and phase/order scores are reported separately.

Qualitative examples

Where plausible motion falls short

The evaluated few-step autoregressive models can produce visually plausible videos while failing to faithfully realize the requested event transition.

Video slots are reserved. Full generated videos will be added here. The descriptions below specify expected transitions, not observations or scores of the missing videos.

Examples are intended to explain individual transitions and failure modes; selected cases are not estimates of category-level failure prevalence.

Inside the benchmark

OETGBench composition

Four task-oriented families organize the dominant requested change: Action Start, Entity Interaction, Object State Change, and Environmental Perturbation.

Distribution of the 2,500 benchmark tasks across the four event-transition families.
A word cloud calculated from the original generation prompts in OETGBench.
Prompt vocabulary. Word size reflects token frequency after lowercasing and stopword removal. This describes vocabulary, not independently validated semantic coverage.
How these statistics were produced

A task, made explicit

Phase targets & evaluation questions

VLM judging instructions

Judgments use visible evidence only. The generation prompt supplies context for VLD, not evidence. Each WPV window may support several phases or none.

YES Clear evidencePARTIAL Weak, incomplete, or ambiguousNO Not supported

For failure questions, YES means the failure is visible; NO means it is not visible. This polarity is opposite to a successful quality or phase judgment. The prompts below are instantiated for the currently selected task; each returned label must be one of YES, PARTIAL, or NO.

Video-level Diagnostics (VLD) — selected task input
Window-level Phase Verification (WPV) — selected task input






Main-paper results
Evaluator Validation


Agreement with the human reference and direct full-video comparisons test different aspects of evaluation. Cross-VLM consistency alone does not establish human-aligned assessment.


Human agreement on phase evidence
150 matched videos from 50 tasks. Each entry is Kendall's τb / Spearman's ρ. Correlations are computed across individual videos, not generator means.
All three VLMs correlate positively with human ratings, but no evaluator is strongest across every phase. Progression is less consistently judged.


Main-paper figure showing cross-VLM and human–VLM correlations and average phase scores.
Human agreement and cross-VLM consistency reproduced from the main-paper figure. Similar average scores can conceal different individual judgments.


Temporal Order: full video vs. window-level verification
Direct full-video VLM assessment compared with OETGEval's four-window phase verification and ordered matching. Values follow the updated main-paper comparison.
OETGEval has the strongest human alignment among the compared evaluators on this evaluation. This supports the usefulness of explicit phase verification; it does not establish perfect or frame-exact temporal understanding. AUROC is a ranking measure, not classification accuracy.






Main-paper results
Generator Evaluation


Producing visible activity remains easier than completing the specified transition. Outcome is the weakest phase for all three evaluated pretrained generators.


Baseline event evaluation: 50 matched tasks per generator, 150 videos total, 81 frames at 16 FPS. Full VBench quality is evaluated separately. Bold values mark the best pretrained-generator values within a column, not statistical significance.
Self-Forcing has the highest baseline Event Transition Score, but its advantage over CausVid is not resolved by the paired confidence interval.


Visual consistency ≠ event completion
Causal Forcing++ ranks highest on the consistency summary but lowest on Event Transition Score; Self-Forcing shows the reverse ranking.
These observations support evaluating event completion separately from visual consistency, without establishing a causal trade-off.
TC is the equal-weight mean of subject consistency, background consistency, temporal flickering, and motion smoothness on the same matched videos. It is not an official VBench aggregate; its absolute scale is not directly comparable with Event Transition Score.






A small post-training study
AG-P3T


We investigate whether OETGEval feedback can guide model improvement through anchor-relative feedback and positive-advantage gating.




Cumulative configurations share a fixed event scorer, four policy rollouts per prompt, and identical group normalization and advantage clipping. Values are from the current main-text A1–A4 table.


Phase/order weighting alone offers limited improvement. Anchor-relative penalties reduce measured failures but also lower VLD Quality. Adding the gate gives the strongest event, outcome, and quality scores, while the ungated penalized variant retains the lowest failure score. The result supports gating as a complement to penalties, not as a guarantee that every quality dimension improves.


The gate filters positive sample advantages without subsequent renormalization. It does not certify post-update model quality. The separate post-training reference in the generator table is not pooled with the pretrained matched-video cohort.