RoboChrono.
GitHub
REAL ROBOTS · STREAMING UNDERSTANDING
A Real Robot Benchmark for
Streaming Task Understanding
The world doesn’t wait for the video to end.
Can a model understand an action as it unfolds?
REAL-WORLD PLATFORMS.
SHARED RESEARCH.
A WORLD IN MOTION
Ten glimpses of real manipulation: making tea, packing, cleaning, and arranging. Watch the same world through different hands.
01 / THE IDEA
A single frame tells you what is visible. An unfolding interaction tells you what is happening. RoboChrono asks models to reason from the observations available up to a queried moment, without looking into the future.
02 / THE BENCHMARK
From identifying the current action to locating it in time, each task isolates a different part of temporal understanding. Six tasks use multiple-choice evaluation; one measures action time localization.
RECOGNITION
Recognize actions, anticipate the next step, and identify a frame within the observed history.
ALIGNMENT
Match observations across cameras and reconstruct the chronological sequence of video frames.
TEMPORAL GROUNDING
Identify when an action starts and ends within the observation history, using explicit temporal boundaries.
A causal observation boundary. Models see only the visual evidence available up to the query time. Future frames stay out of reach.
FOUR EMBODIMENTS / ONE SHARED TASK
Robot grippers, dexterous hands, and egocentric human demonstrations. Explore the shared “Stack Cubes” task across all four embodiment settings.
TIANJI GRIPPER / THREE CAMERA VIEWS
Teleoperated gripper demonstrations capture the same execution from a top camera and two wrist-mounted cameras.
04 / WHAT WE LEARNED
Zero-shot evaluation of 18 vision-language models reveals a capability gap: strong visual matching does not consistently translate into strong temporal ordering.
VISUAL MATCHING → TEMPORAL ORDERING
Accuracy on Frame Matching vs. Frame Ordering.
Paper-reported results (Table 3). Evaluation coverage and aggregation vary by model; see the paper for details.
WHAT IF THE MODEL CANNOT SEE?
Accuracy change when visual observations are removed.
Current Action
Recognition
Next Action
Prediction
Next-action prediction can draw on task and action priors, even when visual evidence is unavailable.
Mean drops across five open-weight models on a balanced 312-stem subset; percentage points (pp).
05 / BUILD WITH ROBOCHRONO
Explore the data, examine the evaluation, and build toward better understanding of real-world robot interaction.
CITE ROBOCHRONO
@misc{wu2026robochrono,
title = {RoboChrono: A Real Robot Benchmark
for Streaming Task Understanding},
author = {Yuzhou Wu and Longteng Fan and
Zimeng Li and Yu Wanchan and Ting Zhang and
Yiyang Ma and Shihao Li and Wei Ying and
Jianbin Qin and Jiajian Jing and Fangwen Chen and
Yifan Wu and Zichen Zhang and Ruiqi Yang and
Weibin Kong and Yihang Xu and Haoran Liu and
Zonghang He and Xuyang Liu and YiFan Xiong and
Siteng Huang and Tao Xu and Zhuo Xu and
Long Chen and Ruoxiang Li},
year = {2026},
note = {Public preprint},
url = {https://github.com/mfan-res/ROBOCHRONO/blob/main/docs/paper/RoboChrono.pdf}
}