# WAFFLE: Fine-tuning Multi-Modal Model for Automated Front-End Development > WAFFLE fine-tunes multimodal LLMs to generate HTML/CSS from UI screenshots. A quarter of the decoder's attention heads use a structure-aware mask that limits each element's tokens to its parent, preceding siblings and its own earlier tokens. A contrastive objective aligns image and code embeddings over groups of four mutated WebSight-v0.1 pages. With VLM-WebSight and Moondream2 backbones, the authors report up to 9.00 pp higher HTML-Match and 0.0982 higher CW-SSIM than standard fine-tuning on WebSight-Test, plus gains on every Design2Code metric. - Authors: Shanchao Liang; Nan Jiang; Shangshu Qian; Lin Tan - Venue: ACL 2025 - Status: Conference paper - Publication / posting date: July 2025 - Date source: The official ACL Anthology record gives July 2025. Crossref records only the year; the July 27–August 1 conference dates printed in the PDF are not treated as a publication day. - First arXiv submission: October 24, 2024 - arXiv: 2410.18362 - Reviewed arXiv version: v3 - DOI: 10.18653/v1/2025.acl-long.1208 - Sources reviewed: 2026-10-20 - Added to the catalog: 2026-11-03 - Original paper: - PDF: - Code: - BibTeX: - Catalog page: - Record JSON: - Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-waffle/index.md) ## Scope and limitations Generates static HTML/CSS from a single screenshot; it does not recover a site's original source, scripts, interactions, navigation or backend. HTML-Match strips styles and attributes before requiring pixel-identical renders, while CW-SSIM, CLIP and LLEM are similarity scores; the authors note that these metrics do not fully capture human judgment. Training data and the 500-page WebSight-Test are synthetic WebSight-style pages; Design2Code supplies 484 real-world pages. Only two backbones were fine-tuned, and on Design2Code the Moondream2 model trained with WAFFLE stays below GPT-4o and GPT-4o mini. The Gemini 1.5 Pro baseline answered 384 Design2Code instances. Human ranking used 50 samples per dataset and two annotators. Commercial-model prompting results are historical baselines, not current rankings. ## Why it fits this collection Adds a training method for screenshot-to-HTML website reconstruction that builds DOM hierarchy into attention, complementing the collection's screenshot-to-code benchmarks, datasets and prompting pipelines. ## Publication and review notes Publication identity follows the ACL Anthology record and PDF. Reviewed arXiv v3 of 2026-03-02 (v1 2024-10-24, v2 2025-06-24) is titled “WAFFLE: Finetuning Multi-Modal Models for Automated Front-End Development”; the repository README and model card use “WAFFLE: Multi-Modal Model for Automated Front-End Development” and cite the arXiv preprint. The paper's availability statement links lt-asset/Waffle. The reviewed 2025-05-28 snapshot is BSD-3-Clause and provides VLM-WebSight inference, Selenium/Chrome rendering, the HTML tree-attention utilities, a contrastive dataset class reading a local load_from_disk dataset, and 500 WebSight-Test PNG screenshots without reference HTML. Although README describes a training file, the snapshot has no training script, Moondream2 code, metric implementation or mutated training set; no public copy of the 231,940-pair set was found in the lt-asset Hugging Face organization. Weights are on Hugging Face as lt-asset/Waffle_VLM_WebSight (7B). The GitHub README releases them under Apache-2.0 subject to the base model's terms, while the model card declares bsd-3-clause-clear. Documented dependencies include Python 3.10.14, PyTorch 2.3.0, Transformers 4.41.1, PEFT 0.11.1 and DeepSpeed 0.14.1; the inference script expects a CUDA GPU and has placeholder paths. ## Primary sources 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. ## About this entry Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX. Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.