2026 · Conference paper
Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verification
Benchmarks website development from visual prototypes and written requirements across 100 static-page, 66 interactive-frontend and 27 full-stack tasks, combining workflow-based functional checks with model-judged visual comparison.
Scope and limitations
The full-stack tier is specification-driven, not reconstruction solely by exploring a live website. Workflows test state management and CRUD outcomes, but finite agent judgments do not establish exact backend or durable-database equivalence. The current public evaluator maintains a browser session within each workflow and permits local business-API checks; this is not a universal persistence proof.
Why it fits this collection
A useful comparison for evaluating interactive and stateful website reconstruction, while separating written-specification development from behavior-only inference.
Publication and review details
- Venue
- ICML 2026
- Status
- Conference paper
- Publication / posting date
- September 29, 2026
- Date source
- Publication date stated for PMLR volume 306; the July 6–11 conference dates are not used as the publication date.
- First arXiv submission
- March 27, 2026
- arXiv
- 2603.26648
- Reviewed arXiv version
- v3
The final PMLR paper has seven authors and pages 41393–41415; arXiv v3 is dated 2026-07-20. Current artifact evaluation differs from the paper configuration. Licensing signals conflict: README text says CC-BY-NC-SA-4.0, Hugging Face metadata says Apache-2.0, the project page says CC BY-SA 4.0, and setup.py lists MIT while the repository has no LICENSE file. No uniform artifact license is asserted.
Sources reviewed
Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Discussed in the survey
Primary sources
- https://proceedings.mlr.press/v306/he26n.html
- https://proceedings.mlr.press/v306/
- https://raw.githubusercontent.com/mlresearch/v306/main/assets/he26n/he26n.pdf
- https://arxiv.org/abs/2603.26648
- https://arxiv.org/html/2603.26648v3
- https://vision2web-bench.github.io/
- README.md GitHub · zai-org/Vision2Web · Revision d80a4a3
- setup.py GitHub · zai-org/Vision2Web · Revision d80a4a3
- vision2web/evaluation/prompts.py GitHub · zai-org/Vision2Web · Revision d80a4a3
- vision2web/evaluation/functional_tester.py GitHub · zai-org/Vision2Web · Revision d80a4a3
- https://huggingface.co/datasets/zai-org/Vision2Web/blob/8f03299d92b9bd852e93852d0c21e8a4848ab661/README.md