# UI2App: Benchmarking Visual Interaction Inference in Executable Web Application Generation

> UI2App evaluates runnable React/TypeScript applications generated from 600 screenshots across 95 web applications. It separates build executability, visual fidelity and screenshot-implied interaction behavior, including cross-route state, using a reference interaction taxonomy and public evaluation pipeline.

- Authors: Grace Man Chen; Litao Guo; Yifan Wu; Yiyu Chen; Yenchi Tseng; Sicheng Liu; Yuyu Luo; Ying-Cong Chen
- Venue: arXiv
- Status: Preprint
- Publication / posting date: July 7, 2026
- Date source: First arXiv submission date. The reviewed v2 was submitted on 2026-10-04; it is not a separately verified conference publication.
- First arXiv submission: July 7, 2026
- arXiv: 2607.06306
- Reviewed arXiv version: v2
- DOI: 10.48550/arXiv.2607.06306
- Sources reviewed: 2026-10-17
- Added to the catalog: 2026-10-17
- Original paper: <https://arxiv.org/abs/2607.06306>
- PDF: <https://arxiv.org/pdf/2607.06306v2>
- Code: <https://github.com/chenmancm169/UI2App>
- Project: <https://chenmancm169.github.io/UI2App-ProjectPage/>
- BibTeX: <https://awesomerea.org/papers/citations/paper-ui2app.bib>
- Catalog page: <https://awesomerea.org/papers/paper-ui2app/>
- Record JSON: <https://awesomerea.org/papers/paper-ui2app/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-ui2app/index.md)

## Scope and limitations

Static screenshots underdetermine behavior; the rubric accepts multiple valid implementations rather than proving equivalence to the original application or backend. Auto IIS judges generated source and screenshots without tool execution; human runtime validation covers a 45-application subset. Cross-route state is not necessarily durable persistence across reloads or sessions. Main results use one generation per application-model pair, with repeated runs on a subset.

## Why it fits this collection

Tests reconstruction from observed web interfaces while distinguishing visual copying from inferred interaction logic; its explicit behavior ambiguity makes it a bounded complement to trace-guided reconstruction benchmarks.

## Publication and review notes

Use v2’s 95 applications and 600 screenshots, not v1’s 45/327 counts. The arXiv DOI is registered. Public code is MIT; the dataset and reference annotations are CC BY 4.0, with upstream code and third-party asset rights separate. The documented pipeline needs Python 3.10+, Node/npm, pnpm, Chromium, Tesseract and model access; Auto IIS additionally needs an authenticated Codex CLI.

## Primary sources

1. <https://arxiv.org/abs/2607.06306>
2. <https://arxiv.org/html/2607.06306v2>
3. <https://arxiv.org/pdf/2607.06306v2>
4. <https://api.datacite.org/dois/10.48550/arXiv.2607.06306>
5. <https://chenmancm169.github.io/UI2App-ProjectPage/>
6. <https://github.com/chenmancm169/UI2App/blob/520a59aad0899b1eb61e80351aa1ffbfad9322a4/README.md>
7. <https://github.com/chenmancm169/UI2App/blob/520a59aad0899b1eb61e80351aa1ffbfad9322a4/LICENSE>
8. <https://github.com/chenmancm169/UI2App/blob/520a59aad0899b1eb61e80351aa1ffbfad9322a4/requirements.txt>
9. <https://github.com/chenmancm169/UI2App/blob/520a59aad0899b1eb61e80351aa1ffbfad9322a4/iis/evaluate.py>
10. <https://huggingface.co/datasets/UI2App/UI2App/blob/32e6f6193c5c265cf6d8afcbce12b641e3c2a758/README.md>
11. <https://huggingface.co/datasets/UI2App/UI2App/blob/32e6f6193c5c265cf6d8afcbce12b641e3c2a758/annotations/rii.jsonl>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.