2024 · Conference paper
Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs
Studies screenshot-to-HTML reconstruction through instruction tuning and two evaluation benchmarks. Web2Code combines generated webpages, refined WebSight and pix2code image/code pairs, and WebSRC-derived question answering. Models produce HTML from a webpage image and instruction; rendered output is compared visually, while a separate benchmark measures screenshot understanding.
Scope and limitations
Evaluation uses 1,198 screenshots and 5,990 yes/no questions. WCGB grades rendered images with GPT-4V; its User Interface and Interactivity category measures element appearance, not executed click paths, persistent state or backend behavior. Synthetic and refined examples do not establish coverage of arbitrary real web applications, original-source equivalence or code quality. The paper reports contact-information issues in 2.7% of a 1,000-item privacy sample, so a blanket claim that the data is free of such information is unsupported.
Why it fits this collection
A foundational visual reconstruction dataset and evaluation method, useful for distinguishing screenshot fidelity from the behavioral guarantees of later website-reconstruction benchmarks.
Publication and review details
- Venue
- NeurIPS 2024 Datasets and Benchmarks
- Status
- Conference paper
- Publication / posting date
- 2024 (exact date not verified)
- Date source
- The official NeurIPS proceedings and publisher-deposited Crossref metadata identify publication year 2024; Crossref provides no finer publication date. The first arXiv submission date is recorded separately.
- First arXiv submission
- June 28, 2024
- arXiv
- 2406.20098
- Reviewed arXiv version
- v2
- DOI
- 10.52202/079017-3560
Published in NeurIPS volume 37, pages 112134–112157; full author names follow the proceedings PDF. The repository still cites the preprint; reviewed arXiv v2 is dated 2024-11-17. The LLaVA-based training subtree carries Apache-2.0. Dataset licensing is inconsistent: the GitHub README and dataset-card frontmatter say MIT, while the same card’s body says CC BY 4.0 with research-only/non-commercial wording. The training metadata requires Python >=3.8 and pins PyTorch 2.0.1; evaluation needs browser rendering and an external vision-model API.
Sources reviewed
Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Discussed in the survey
Primary sources
- https://proceedings.nips.cc/paper_files/paper/2024/hash/cb66be286795d71f89367d596bf78ea7-Abstract-Datasets_and_Benchmarks_Track.html
- https://proceedings.neurips.cc/paper_files/paper/2024/file/cb66be286795d71f89367d596bf78ea7-Paper-Datasets_and_Benchmarks_Track.pdf
- https://arxiv.org/abs/2406.20098v2
- https://arxiv.org/html/2406.20098v2
- https://api.crossref.org/works/10.52202/079017-3560
- README.md GitHub · MBZUAI-LLM/web2code · Revision 48c356a
- web2code/README.md GitHub · MBZUAI-LLM/web2code · Revision 48c356a
- web2code/pyproject.toml GitHub · MBZUAI-LLM/web2code · Revision 48c356a
- web2code/LICENSE GitHub · MBZUAI-LLM/web2code · Revision 48c356a
- code_generation/README.md GitHub · MBZUAI-LLM/web2code · Revision 48c356a
- code_generation/gpt4_vision_evaluation.py GitHub · MBZUAI-LLM/web2code · Revision 48c356a
- webpage_understanding/README.md GitHub · MBZUAI-LLM/web2code · Revision 48c356a
- https://huggingface.co/datasets/MBZUAI/Web2Code/blob/21e587b4509b946e56732fe8c621b17bad631e6b/README.md