# Assemblage: Automatic Binary Dataset Construction for Machine Learning > Assemblage is a distributed system that crawls GitHub and vcpkg, builds C/C++ projects under many compiler configurations, and records source-to-binary mappings such as function addresses, source code, comments and PDB or DWARF debug information. Run on AWS for about a year, it produced 890k Windows PE binaries built with Visual Studio 2015–2022 across optimization levels, architectures and Debug/Release modes, plus 428k Linux ELF binaries from GCC and Clang, in 29 configurations. Case studies use the data for compiler provenance with PassTell, function similarity with a graph neural network and jTrans, and function-boundary identification comparing XDA with IDA. - Authors: Chang Liu; Rebecca Saul; Yihao Sun; Edward Raff; Maya Fuchs; Townsend Southard Pantano; James Holt; Kristopher Micinski - Venue: NeurIPS 2024 Datasets and Benchmarks - Status: Conference paper - Publication / posting date: 2024 (exact date not verified) - Date source: The official NeurIPS proceedings and publisher-deposited Crossref metadata identify publication year 2024; Crossref provides no finer publication date. The first arXiv submission date is recorded separately. - First arXiv submission: May 7, 2024 - arXiv: 2405.03991 - Reviewed arXiv version: v2 - DOI: 10.52202/079017-1871 - Sources reviewed: 2026-10-27 - Added to the catalog: 2026-11-03 - Original paper: - PDF: - Code: - Project: - BibTeX: - Catalog page: - Record JSON: - Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-assemblage/index.md) ## Scope and limitations The binaries are builds of open-source GitHub and vcpkg projects, many of them small student projects or utilities, so they do not represent commercial, packed or obfuscated software. Assemblage does not independently scan for malware; the authors rely on GitHub’s policy against malware and plan scanning for a later version. The reported build success rate was about 30% on Windows and 7% on Linux. Public datasets include only repositories with permissive licenses, so the downloadable copy is a subset of the paper’s corpus, and repository links in the metadata tie binaries to authors’ GitHub profiles. The case studies retrain existing models under their original protocols with minimal changes; they show Windows/Linux generalization gaps, such as jTrans MRR of 0.52 on Windows against 0.83 on Linux, rather than new analysis methods. ## Why it fits this collection Adds a dataset paper for Windows PE binaries with source and debug mappings, complementing the Linux ELF function corpora used by decompilation research in this section. ## Publication and review notes Crossref and the NeurIPS proceedings page give the eight-author order, volume 37 and pages 58698–58715; Crossref records only the year. The proceedings page’s 2024-12-16 citation date also appears on another NeurIPS 2024 paper page, so year precision is kept. Reviewed arXiv v2 (2024-11-02), which carries the NeurIPS comment; v1 dates from 2024-05-07. At the reviewed f8ac584 commit (2026-08-08), the repository README says the code is MIT-licensed and the published dataset is a permissively licensed subset; pyproject.toml declares MIT, but there is no LICENSE file and GitHub detects none. The README also says the codebase was refactored in July 2026 and extended with a Rust worker, that Windows/MSVC builds sit under backend/assemblage/legacy/ and need a Windows host, and that a pre-refactor branch keeps the earlier code. The Hugging Face cards for changliu8541/Assemblage_PE, whose public copy has 91k Windows PE binaries, and Assemblage_LinuxELF declare license “other” and ask users to follow each source repository’s license. ## Primary sources 1. 2. 3. 4. 5. 6. 7. 8. 9. ## About this entry Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX. Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.