# CrackMeBench: Binary Reverse Engineering for Agents

> Evaluates agents on executable-checked CrackMe tasks: infer validation logic from binaries and submit an accepted input or artifact, while recording tool use, time, and scoring attempts.

- Authors: Isaac David; Arthur Gervais
- Venue: arXiv
- Status: Preprint
- Publication / posting date: May 11, 2026
- Date source: arXiv v1 submission
- First arXiv submission: May 11, 2026
- arXiv: 2605.10597
- Reviewed arXiv version: v1
- Sources reviewed: 2026-10-16
- Added to the catalog: 2026-10-16
- Original paper: <https://arxiv.org/abs/2605.10597>
- PDF: <https://arxiv.org/pdf/2605.10597>
- BibTeX: <https://awesomerea.org/papers/citations/paper-crackmebench.bib>
- Catalog page: <https://awesomerea.org/papers/paper-crackmebench/>
- Record JSON: <https://awesomerea.org/papers/paper-crackmebench/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-crackmebench/index.md)

## Scope and limitations

The v0 study uses 12 generated tasks and eight public calibration tasks, Linux x86-64 ELF binaries, a shell interface, and a five-minute budget. Its compact educational tasks do not establish performance on large applications. No paper-linked public artifact URL was verified.

## Why it fits this collection

Directly measures whether an autonomous reverse-engineering workflow produces a verifiably accepted result.

## Publication and review notes

No conference venue is stated in the reviewed arXiv record. Tool repositories cited in its bibliography are not treated as the paper's own code.

## Primary sources

1. <https://arxiv.org/abs/2605.10597>
2. <https://arxiv.org/html/2605.10597v1>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.