# Reveree: Diagnosing LLM Reverse-Engineering Agents

> Evaluates reverse-engineering agent traces through flag recovery, eight analysis milestones and action profiles. Deterministic checks and an outcome-blinded LLM judge help locate failures beyond a single solve-rate score.

- Authors: Hadjer Benkraouda; Hongyu Cai; Berkay Celik; Gang Wang
- Venue: arXiv
- Status: Preprint
- Publication / posting date: September 1, 2026
- Date source: First arXiv submission; no reviewed venue publication is established.
- First arXiv submission: September 1, 2026
- arXiv: 2609.01185
- Reviewed arXiv version: v1
- DOI: 10.48550/arxiv.2609.01185
- Sources reviewed: 2026-10-20
- Added to the catalog: 2026-10-20
- Original paper: <https://arxiv.org/abs/2609.01185>
- PDF: <https://arxiv.org/pdf/2609.01185v1>
- BibTeX: <https://awesomerea.org/papers/citations/paper-reveree.bib>
- Catalog page: <https://awesomerea.org/papers/paper-reveree/>
- Record JSON: <https://awesomerea.org/papers/paper-reveree/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-reveree/index.md)

## Scope and limitations

Uses 88 public CTF challenges and nine models; all four prompting strategies are compared on four models. Evaluation fixes the D-CIPHER harness and uses one attempt per setting. One expert checks 15 traces; agreement on judged comprehension stages is 53/60. Recall and surface-perturbation probes do not eliminate contamination.

## Why it fits this collection

Adds process-level evaluation for binary-analysis agents.

## Publication and review notes

Reviewed arXiv v1 and registered DOI. The paper announces a release, but the reviewed manuscript does not identify a dedicated artifact URL.

## Primary sources

1. <https://arxiv.org/abs/2609.01185v1>
2. <https://arxiv.org/html/2609.01185v1>
3. <https://api.datacite.org/dois/10.48550/arxiv.2609.01185>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.