# SCDBench: A Benchmark for LLM-Based Smart Contract Decompilers

> Benchmarks LLM-based EVM decompilation on 600 source-verified Ethereum Solidity contracts, sampled in easy, medium and hard thirds from 772,736 unique bytecode–source pairs after near-duplicate filtering. Each task supplies EVM assembly and optional four-byte selector hints and asks for self-contained Solidity with compiler settings. Outputs are scored in four cumulative stages: format completeness, compilation, ABI recovery, and semantic consistency, where 227,383 recorded test cases for 14,553 public functions are replayed on the recompiled output. With one compiler-feedback repair call, GPT-5.3-Codex compiled 90.3% of contracts, while the best semantic results were 29.4% of functions and 42 of 600 contracts recovered perfectly, both by Claude Opus 4.7.

- Authors: Kaihua Qin; Dawn Song; Arthur Gervais
- Venue: arXiv
- Status: Preprint
- Publication / posting date: May 27, 2026
- Date source: arXiv v1 submission date; no later version, journal reference or proceedings publication was found.
- First arXiv submission: May 27, 2026
- arXiv: 2605.29059
- Reviewed arXiv version: v1
- Sources reviewed: 2026-10-20
- Added to the catalog: 2026-11-02
- Original paper: <https://arxiv.org/abs/2605.29059>
- PDF: <https://arxiv.org/pdf/2605.29059v1>
- BibTeX: <https://awesomerea.org/papers/citations/paper-scdbench.bib>
- Catalog page: <https://awesomerea.org/papers/paper-scdbench/>
- Record JSON: <https://awesomerea.org/papers/paper-scdbench/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-scdbench/index.md)

## Scope and limitations

Replay compares return data, reverts, logs and touched storage on generated inputs that reach 77.8% average code coverage; passing it is not proof of equivalence, and replay needs an Ethereum archive node. The contracts are source-verified Ethereum Solidity from compiler v0.4 to v0.8, so unverified, obfuscated, Vyper or other-chain bytecode is not covered. Three models are evaluated zero-shot through OpenRouter: Claude Opus 4.7 and GPT-5.3-Codex at high reasoning effort, and GLM-5 with and without extended thinking. Results and quoted costs describe those 2026 snapshots. Gigahorse and Heimdall-rs are scored only on selector and prototype recovery because they do not emit compilable Solidity; DiSCo and the fine-tuned decompiler of David et al. are discussed but not run.

## Why it fits this collection

Adds a shared benchmark with execution-based semantic checks for smart-contract decompilers, complementing EVM decompilation papers that evaluate on separate datasets and metrics.

## Publication and review notes

Reviewed arXiv v1, the only version, which is labelled a preprint and has no comment, journal reference or DOI. The paper says the benchmark dataset and evaluation artifacts, including prompts, scripts and model outputs, are provided, and its broader-impact appendix describes a released dataset, but v1 gives no URL. GitHub and Hugging Face searches found no release. Compilation and replay use solc and Foundry.

## Primary sources

1. <https://arxiv.org/abs/2605.29059v1>
2. <https://arxiv.org/pdf/2605.29059v1>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.