As web agents close the gap with humans on benchmarks, one question arises: Do today's agents perform just as well on tomorrow's web?
Introducing TimeWarp
We introduce TimeWarp, a benchmark that emulates the evolving web.
TimeWarp consists of three web environments, each with six UI versions spanning UI design, frontend code, and workflows from different eras of the internet.
Complex, Realistic Tasks
We pair TimeWarp with a set of complex, realistic tasks covering different forms of web navigation.
Experiment Results
Our experiments reveal that vision-based agents are vulnerable to changes, while text-based agents become brittle once fine-tuned on a single version.
Proposing TimeTraj
To address this, we propose TimeTraj, a new annotation method that uses plan distillation to collect trajectories across multiple versions.
By training agents on teacher rollouts using our BC-variant, we achieve substantial performance gains: 20.4% to 37.7% for Qwen-3 4B and 0% to 27.0% for Llama-3.1 8B models.
New Paradigm for Collecting Plans
Our work helps study generalization across web designs and opens a new paradigm for collecting plans rather than trajectories to improve the robustness of web agents.