# ProgramDistill: From Interactive Web Apps to Verifiable Reference-Guided SWE Tasks

> Evaluates recovery of web-app behavior from a live reference with hidden source. It provides 4,063 feature-restoration tasks across 26 applications and a separate 12-application reconstruction setting, scored by replaying dependent browser workflows.

- Authors: Jeonghye Kim; Minseon Kim; Young Jin Kim; Matheus Pereira; Marc-Alexandre Côté; Alessandro Sordoni; Xingdi Yuan; Zhengyan Shi
- Venue: arXiv
- Status: Preprint
- Publication / posting date: September 16, 2026
- Date source: arXiv v1 submission history; the earlier author blog date is separate.
- First arXiv submission: September 16, 2026
- arXiv: 2609.18805
- Reviewed arXiv version: v1
- DOI: 10.48550/arXiv.2609.18805
- Sources reviewed: 2026-10-17
- Added to the catalog: 2026-10-17
- Original paper: <https://arxiv.org/abs/2609.18805>
- PDF: <https://arxiv.org/pdf/2609.18805v1>
- Code: <https://huggingface.co/datasets/microsoft/ProgramDistill>
- Project: <https://microsoft.github.io/debug-gym/blog/2026/09/programdistill/>
- BibTeX: <https://awesomerea.org/papers/citations/paper-programdistill.bib>
- Catalog page: <https://awesomerea.org/papers/paper-programdistill/>
- Record JSON: <https://awesomerea.org/papers/paper-programdistill/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-programdistill/index.md)

## Scope and limitations

Tests cover finite, model-mined behaviors in self-contained apps, including SaaS clones. Persistence failures can be detected through replayed workflows, but passing does not establish all backend or database invariants. Evaluated agents receive structured browser observations without screenshots; visual fidelity is not measured. Public verifiers and reference solutions are not a contamination-resistant test set.

## Why it fits this collection

Directly studies reference-guided behavioral reconstruction beyond screenshot-to-HTML, including dependent state changes and fresh verification of submitted implementations.

## Publication and review notes

The DOI resolves and DataCite records registration on 2026-09-17, despite arXiv’s stale pending banner. Separately, the paper was accepted to the non-archival Workshop on Interpreting Agent Behavior (IAB) at NeurIPS 2026, scheduled for December 12, 2026; its OpenReview record was published on 2026-10-04. That record and the workshop listing put Xingdi Yuan before Alessandro Sordoni; the authors here follow arXiv v1. The public release contains executable benchmark packages, not the mining, task-generation or orchestration pipeline. ProgramDistill-authored artifacts declare MIT; included applications and assets retain upstream terms.

## Primary sources

1. <https://arxiv.org/abs/2609.18805>
2. <https://arxiv.org/html/2609.18805v1>
3. <https://microsoft.github.io/debug-gym/blog/2026/09/programdistill/>
4. <https://huggingface.co/datasets/microsoft/ProgramDistill/blob/dc782f7003ae00738260996d02d77920d08a5db1/README.md>
5. <https://huggingface.co/datasets/microsoft/ProgramDistill/blob/dc782f7003ae00738260996d02d77920d08a5db1/tasks/full-reconstruction/README.md>
6. <https://huggingface.co/datasets/microsoft/ProgramDistill/blob/dc782f7003ae00738260996d02d77920d08a5db1/tasks/full-reconstruction/baserow_baserow/tests/scorer/trace_schema.py>
7. <https://huggingface.co/datasets/microsoft/ProgramDistill/blob/dc782f7003ae00738260996d02d77920d08a5db1/tasks/full-reconstruction/baserow_baserow/tests/scorer/trace_replay_validator.py>
8. <https://api.datacite.org/dois/10.48550/arXiv.2609.18805>
9. <https://openreview.net/forum?id=FbKJFN0R5L>
10. <https://github.com/iab-agents/iab-agents.github.io/blob/0437015a568d8d541e277833c68ffd213b117c53/src/App.jsx>
11. <https://github.com/iab-agents/iab-agents.github.io/blob/0437015a568d8d541e277833c68ffd213b117c53/src/data/siteData.js>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.