# CREBench: Evaluating Large Language Models in Cryptographic Binary Reverse Engineering

> Benchmarks cryptographic binary reverse engineering across algorithm recognition, key or IV extraction, wrapper reconstruction, and flag recovery using 432 controlled challenges.

- Authors: Baicheng Chen; Yu Wang; Ziheng Zhou; Xiangru Liu; Juanru Li; Yilei Chen; Tianxing He
- Venue: COLM 2026
- Status: Conference paper
- Publication / posting date: 2026 (exact date not verified)
- Date source: Publication year from the official COLM 2026 OpenReview record and its BibTeX; no finer publication date is asserted.
- First arXiv submission: April 4, 2026
- arXiv: 2604.03750
- Reviewed arXiv version: v2
- Sources reviewed: 2026-10-17
- Added to the catalog: 2026-10-16
- Original paper: <https://openreview.net/forum?id=tLYdf8hWJB>
- PDF: <https://arxiv.org/pdf/2604.03750v2>
- Code: <https://github.com/wangyu-ovo/CREBench>
- Project: <https://jams-zhou-james.github.io/CREBench/>
- BibTeX: <https://awesomerea.org/papers/citations/paper-crebench.bib>
- Catalog page: <https://awesomerea.org/papers/paper-crebench/>
- Record JSON: <https://awesomerea.org/papers/paper-crebench/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-crebench/index.md)

## Scope and limitations

The corpus combines 48 algorithms, three insecure key-use scenarios, and three difficulty levels. It excludes professional-grade obfuscators such as Tigress and O-LLVM; results should not be generalized to heavily protected arbitrary binaries.

## Why it fits this collection

Provides a focused, decomposed test of cryptographic behavior recovery, with a verified official implementation.

## Publication and review notes

The COLM 2026 OpenReview record has the same title, seven authors and abstract as arXiv v2 (2026-08-06), whose header reads 'Published as a conference paper at COLM 2026' and which the official project calls its camera-ready version. OpenReview’s 2026-07-08 publication timestamp coincides with the project’s acceptance announcement and is not used as a publication date; the October 6–9, 2026 conference and its Poster Session 5 listing are event details. The repository BibTeX still cites arXiv. This is unrelated to the creativity-evaluation paper also named CreBench.

## Primary sources

1. <https://openreview.net/forum?id=tLYdf8hWJB>
2. <https://colm.cc/Conferences/2026/AcceptedPapers>
3. <https://colm.cc/Conferences/2026>
4. <https://arxiv.org/abs/2604.03750>
5. <https://arxiv.org/html/2604.03750v2>
6. <https://arxiv.org/pdf/2604.03750v2>
7. <https://github.com/wangyu-ovo/CREBench>
8. <https://github.com/wangyu-ovo/CREBench/blob/9fc4016538a0bab8c230732a9f788a61ed43addf/README.md>
9. <https://jams-zhou-james.github.io/CREBench/>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.