{ "added_at": "2026-11-03", "arxiv_id": "2404.09836", "arxiv_version_reviewed": "v3", "authors": [ "Xiuwei Shang", "Shaoyin Cheng", "Guoqiang Chen", "Yanming Zhang", "Li Hu", "Xiao Yu", "Gangyang Li", "Weiming Zhang", "Nenghai Yu" ], "bibtex_url": "https://awesomerea.org/papers/citations/paper-llm-binary-understanding.bib", "citation": { "container_title": "2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)", "pages": "1-12", "publisher": "IEEE", "sources": [ "https://api.crossref.org/works/10.1109/ICSME58944.2024.00012" ] }, "code_url": null, "date_basis": "IEEE’s Crossref deposit gives published/print and issued date 2024-10-06, which is also the first day of the conference listed there (6–11 October). This entry uses the publisher-deposited date and does not derive it from the conference schedule or the arXiv history.", "date_basis_zh": "IEEE 提交的 Crossref 元数据将 published／纸质出版及 issued 日期记为 2024-10-06，这也是其中所列会议（10 月 6–11 日）的首日。本条目采用出版方提交的日期，并非依据会议日程或 arXiv 历史推定。", "date_precision": "day", "description_en": "Benchmarks 16 code and general LLMs on function name recovery and summarization of stripped binaries. The benchmark takes 12 popular C projects built for x86-64, strips and decompiles 28 binaries with IDA Pro, aligns 81,185 functions with source through DWARF, and keeps 2,000 functions after expert review. Zero-shot LLMs are compared with SymLM, NER, BinT5 and HexT5. Code Llama 34B leads name recovery with token-level F1 of 27.59, and ChatGPT leads summarization with BLEU-4 7.37. The study also measures few-shot prompting, LoRA fine-tuning of two 7B models on GNU binaries, and a case study on a stripped open-source Linux remote-access trojan.", "description_zh": "在剥离符号二进制的函数名恢复与代码摘要两项任务上，评测 16 个代码领域与通用领域的大语言模型。基准选取 12 个热门 C 项目编译为 x86-64，剥离并用 IDA Pro 反编译 28 个二进制，借助 DWARF 将 81,185 个函数与源码对齐，经专家审核后保留 2,000 个函数。零样本大语言模型与 SymLM、NER、BinT5 和 HexT5 进行比较。Code Llama 34B 在函数名恢复上以 27.59 的词元级 F1 领先，ChatGPT 在摘要任务上以 7.37 的 BLEU-4 领先。研究还考察了少样本提示、两个 7B 模型基于 GNU 二进制的 LoRA 微调，以及针对一个剥离符号的开源 Linux 远程访问木马的案例研究。", "doi": "10.1109/ICSME58944.2024.00012", "entry_url": "https://awesomerea.org/papers/paper-llm-binary-understanding/", "entry_url_zh": "https://awesomerea.org/zh/papers/paper-llm-binary-understanding/", "evidence_urls": [ "https://api.crossref.org/works/10.1109/ICSME58944.2024.00012", "https://arxiv.org/abs/2404.09836v3", "https://arxiv.org/pdf/2404.09836v3", "https://arxiv.org/html/2404.09836v3" ], "first_arxiv_date": "2024-04-15", "id": "paper-llm-binary-understanding", "json_url": "https://awesomerea.org/papers/paper-llm-binary-understanding/record.json", "kind": "research", "markdown_url": "https://awesomerea.org/papers/paper-llm-binary-understanding/index.md", "markdown_url_zh": "https://awesomerea.org/zh/papers/paper-llm-binary-understanding/index.md", "metadata_notes": "Crossref establishes the nine-author order, DOI, ICSME 2024 proceedings title and pages 1–12. Reviewed arXiv v3 (2024-10-24), whose comment reads “to be published in ICSME 2024”; v1 and v2 date from 15 and 16 April 2024. The IEEE Xplore page refused automated requests. The paper gives no download link for the benchmark, fine-tuning data or prompts, and no release was found. The evaluated pseudocode depends on IDA Pro, a commercial decompiler.", "metadata_notes_zh": "Crossref 确认九位作者顺序、DOI、ICSME 2024 论文集名称及第 1–12 页。审阅了 arXiv v3（2024-10-24），其注释为“to be published in ICSME 2024”；v1 与 v2 分别提交于 2024 年 4 月 15 日和 16 日。IEEE Xplore 页面拒绝自动化请求。论文未给出基准、微调数据或提示词的下载链接，也未找到相应发布。所评测的伪代码依赖商业反编译器 IDA Pro。", "pdf_url": "https://arxiv.org/pdf/2404.09836v3", "project_url": null, "publication_date": "2024-10-06", "publication_status": "conference-paper", "record_type": "paper", "runtime_tested": false, "scope_limitations_en": "The benchmark covers x86-64 builds with each project’s default compiler and optimization settings and IDA Pro pseudocode only, so the results do not establish behavior on other architectures, optimization levels, decompilers or obfuscated code, which the authors exclude. Reference summaries were generated by ChatGPT from source and screened by three experts, and ChatGPT is also an evaluated model. BLEU-4, METEOR and Rouge-L measure overlap with those references, which the authors say may not fit binary summarization. Name recovery uses token-level precision, recall and F1 rather than exact matches. Fine-tuning used 30,000 functions from O0 x86-64 GNU builds. Results describe model versions released between mid-2023 and early 2024, including gpt-3.5-turbo-16k.", "scope_limitations_zh": "基准仅覆盖按各项目默认编译器与优化设置构建的 x86-64 二进制及 IDA Pro 伪代码，因此结果不能说明在其他架构、优化级别、反编译器或混淆代码上的表现；作者明确未考虑混淆。参考摘要由 ChatGPT 根据源码生成，并经三位专家筛查，而 ChatGPT 本身也是被评测的模型。BLEU-4、METEOR 与 Rouge-L 衡量的是与这些参考摘要的重合度，作者也指出这些指标未必适合二进制代码摘要。函数名恢复采用词元级精确率、召回率与 F1，而非完全匹配。微调使用来自 O0 级 x86-64 GNU 构建的 30,000 个函数。结果对应 2023 年中至 2024 年初发布的模型版本，包括 gpt-3.5-turbo-16k。", "targets": [ "binary" ], "title": "How Far Have We Gone in Binary Code Understanding Using Large Language Models", "title_zh": "大语言模型理解二进制代码的能力已走多远", "url": "https://ieeexplore.ieee.org/document/10795058", "venue": "ICSME 2024", "verification": "source-reviewed", "verified_at": "2026-10-20", "why_fits_en": "Adds a benchmark of LLMs on two core reverse-engineering subtasks, recovering function names and summarizing decompiled functions, with comparisons against earlier learned symbol-recovery models.", "why_fits_zh": "补充针对两项核心逆向子任务（恢复函数名与概括反编译函数）的大语言模型基准，并与早期基于学习的符号恢复模型进行比较。", "year": 2024 }