# web-crawler ReverseAgent

> web-crawler includes ReverseAgent for investigating website JavaScript and request signing. It hooks fetch, XHR, cookies, crypto.subtle and webpack activity in a browser, captures traffic, separates webpack modules and uses an LLM to reimplement a discovered signature algorithm in Python.

- Type: Tool
- Targets: Website
- Category: Website analysis tools
- Relationship: Foundation tool
- Sources reviewed: 2026-11-06
- Added to the catalog: 2026-11-06
- Original resource: <https://github.com/xiaomaozjj666/web-crawler>
- Primary source: <https://github.com/xiaomaozjj666/web-crawler/blob/36dca34845702822a136a1ea60de7444aa687aee/README.md>
- Catalog page: <https://awesomerea.org/resources/web-crawler-reverse-agent/>
- Record JSON: <https://awesomerea.org/resources/web-crawler-reverse-agent/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/resources/web-crawler-reverse-agent/index.md)

## Requirements & limits

- **Prerequisites:** Python 3.10 or newer.
- **Prerequisites:** ReverseAgent uses CamoufoxFetcher and a compatible LLM service; optional browser dependencies are required for that path.
- **License:** MIT license.

## Primary sources

1. <https://github.com/xiaomaozjj666/web-crawler/blob/36dca34845702822a136a1ea60de7444aa687aee/README.md>
2. <https://github.com/xiaomaozjj666/web-crawler/blob/36dca34845702822a136a1ea60de7444aa687aee/docs/architecture.md>

## More in Websites

- [Mirrorframe](https://awesomerea.org/resources/mirrorframe/index.md) · Tool
- [unwebpack-sourcemap](https://awesomerea.org/resources/unwebpack-sourcemap/index.md) · Tool
- [Wirebrowser](https://awesomerea.org/resources/wirebrowser/index.md) · Tool
- [Thug](https://awesomerea.org/resources/thug/index.md) · Tool
- [spy-debugger](https://awesomerea.org/resources/spy-debugger/index.md) · Tool
- [Dinosaur runner: recovering one acceleration rule](https://awesomerea.org/resources/rea-dinosaur-speed-case/index.md) · Case study

## About this entry

The description was reviewed against primary sources.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.