# ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents

> Splits screenshot-to-code generation among grounding, planning and generation agents that label UI regions, build a hierarchical layout tree and write HTML/CSS. The pipeline also produces the Screen-10K training pairs used for supervised and reinforcement fine-tuning, and the paper introduces the 1,000-page ScreenBench.

- Authors: Yilei Jiang; Yaozhi Zheng; Yuxuan Wan; Jiaming Han; Qunzhong Wang; Michael R. Lyu; Xiangyu Yue
- Venue: EMNLP 2026
- Status: Conference accepted
- Publication / posting date: 2026 (exact date not verified)
- Date source: The official EMNLP 2026 detailed program lists a main-conference poster, verifying acceptance; no proceedings publication date is asserted.
- First arXiv submission: July 30, 2025
- arXiv: 2507.22827
- Reviewed arXiv version: v2
- Sources reviewed: 2026-10-17
- Added to the catalog: 2026-10-17
- Original paper: <https://arxiv.org/abs/2507.22827>
- PDF: <https://arxiv.org/pdf/2507.22827v2>
- Code: <https://github.com/leigest519/ScreenCoder>
- Project: <https://jimmyzhengyz.github.io/ScreenCoder-Project-Page/>
- BibTeX: <https://awesomerea.org/papers/citations/paper-screencoder.bib>
- Catalog page: <https://awesomerea.org/papers/paper-screencoder/>
- Record JSON: <https://awesomerea.org/papers/paper-screencoder/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-screencoder/index.md)

## Scope and limitations

Output is static HTML/CSS layout; interaction, JavaScript behavior and backend logic are not reconstructed. Results are visual-similarity scores (CLIP and OCR-matched text blocks) on ScreenBench and Design2Code, which do not establish code-level or functional equivalence. The released pipeline calls a hosted model API, Doubao by default. The reviewed text is arXiv v2; the accepted EMNLP version may differ.

## Why it fits this collection

Represents staged, agent-based screenshot-to-code reconstruction for websites, separating visual grounding and layout planning from code synthesis.

## Publication and review notes

The official EMNLP 2026 detailed program lists paper 7281-MAIN as a poster on 2026-10-26; this records acceptance, not proceedings publication. The arXiv v2 comment 'ScreenCoder-v2' is a version label, not a venue, and an earlier ICLR 2026 submission was withdrawn. The abstract-linked repository is Apache-2.0 at the pinned commit and bundles UIED, LLaMA-Factory and VLM-R1 code with their own license files. ScreenBench is published on Hugging Face under Apache-2.0.

## Primary sources

1. <https://arxiv.org/abs/2507.22827>
2. <https://2026.emnlp.org/program/>
3. <https://docs.google.com/spreadsheets/d/1aXGTy_7Xeh-OXIs3iJSbDpUsZ6YhbfA76b17kBZH0Yk/edit?gid=943432490#gid=943432490>
4. <https://openreview.net/forum?id=29VraifZkB>
5. <https://github.com/leigest519/ScreenCoder/blob/22816fdfb4ee5982973f9a09d67498a64823e65d/README.md>
6. <https://github.com/leigest519/ScreenCoder/blob/22816fdfb4ee5982973f9a09d67498a64823e65d/LICENSE>
7. <https://jimmyzhengyz.github.io/ScreenCoder-Project-Page/>
8. <https://huggingface.co/datasets/Leigest/ScreenCoder/blob/b0e49d2a5d937e93f712d4c281d4036ec2652a30/README.md>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.