2018 · Conference paper

From UI Design Image to GUI Skeleton: A Neural Machine Translator to Bootstrap Mobile GUI Implementation

Chunyang Chen; Ting Su; Guozhu Meng; Zhenchang Xing; Yang Liu

UI2code translates a mobile UI image into an Android component hierarchy and layout skeleton using a neural encoder-decoder. The paper evaluates structural prediction and a small developer study of implementing layouts from generated skeletons.

Scope and limitations

The output covers component types and hierarchy from a single image; detailed styling and text require developer work. Structural token accuracy and an eight-participant study on five screenshots do not establish complete visual fidelity, navigation, app logic or backend recovery. The main split is screenshot-based; a separate evaluation on 1,208 images from 20 unseen apps provides limited generalization evidence.

Why it fits this collection

A native-app reconstruction foundation that explicitly generates and evaluates GUI structure from visual observations, beyond locating interface elements.

Publication and review details

Venue: ICSE 2018
Status: Conference paper
Publication / posting date: May 27, 2018
Date source: Publisher-deposited Crossref publication date for the ICSE proceedings article.
DOI: 10.1145/3180155.3180240

The author-hosted PDF prints 10.1145/3180155.3180222, which identifies another paper; the DOI here follows publisher-deposited metadata for this title, pages 665–676. The artifact README lists 29,887 screenshots, while the paper reports 185,277 image/skeleton pairs. The pinned legacy Torch/Lua repository has no license file.

Sources reviewed 2026-10-17

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.

Report an outdated entry Suggest a translation fix

Discussed in the survey

Apps: recovering structure without assuming application logic

Scope and limitations

Why it fits this collection

Publication and review details

Discussed in the survey

Primary sources