# Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase

> Studies development of the Extractor reverse-engineering tool through commits, available session logs and code-provenance tracing. Taxonomies separate instruction intent, change triggers and agent claims, exposing errors corrected before commits that ordinary repository history misses.

- Authors: Douglas Leith
- Venue: arXiv
- Status: Preprint
- Publication / posting date: September 24, 2026
- Date source: First arXiv submission; the reviewed version is v1, not a verified proceedings publication.
- First arXiv submission: September 24, 2026
- arXiv: 2609.29744
- Reviewed arXiv version: v1
- Sources reviewed: 2026-10-20
- Added to the catalog: 2026-10-20
- Original paper: <https://arxiv.org/abs/2609.29744>
- PDF: <https://arxiv.org/pdf/2609.29744v1>
- Project: <https://github.com/doug-leith/extractor-code-development>
- BibTeX: <https://awesomerea.org/papers/citations/paper-extractor-development-reliability.bib>
- Catalog page: <https://awesomerea.org/papers/paper-extractor-development-reliability/>
- Record JSON: <https://awesomerea.org/papers/paper-extractor-development-reliability/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-extractor-development-reliability/index.md)

## Scope and limitations

One codebase with human-directed work, no human-coded comparison. The 14.3% figure is 15 of 105 audited generation events, including test defects and stale tests, not a defect rate per source line or in shipped code. LLM scoring has sampled human calibration; transcripts are incomplete and the archive omits per-response evidence bundles. This companion study does not independently corroborate the related schema-recovery results.

## Why it fits this collection

Adds provenance and claim-auditing methods for agent-built reverse-engineering tools.

## Publication and review notes

The author's pinned reproducibility-archive README documents lost sessions and unreproducible response-scoring evidence. The repository exposes an archive and Git-history material. No repository-wide license was verified; the project link points to the artifact landing page.

## Primary sources

1. <https://arxiv.org/abs/2609.29744>
2. <https://arxiv.org/html/2609.29744v1>
3. <https://github.com/doug-leith/extractor-code-development/blob/4f74f798e58073c23386a1516b582ed6827bb190/README.md>
4. <https://github.com/doug-leith/ios-protobuf-decoding/blob/9a4a688c52686f82c1a86e386848edc4eb671746/README.md>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.