# Glaurung Models

> Glaurung Models contains tokenizer and model experiments for executable analysis. Its documented 64K BPE tokenizer encodes compiled PE and ELF bytes using Latin-1, with x86 and ARM64 training coverage; an accompanying analysis compares learned SSE and AVX instruction patterns across two tokenizers.

- Type: Research
- Targets: Binary
- Category: Native analysis tools
- Relationship: Learning resource
- Sources reviewed: 2026-11-06
- Added to the catalog: 2026-11-06
- Original resource: <https://github.com/mjbommar/glaurung-models>
- Primary source: <https://github.com/mjbommar/glaurung-models/blob/1819020f447d55f55b629b78ba96502edae71715/glaurung_binary_tokenizer_001_README.md>
- Catalog page: <https://awesomerea.org/resources/glaurung-models/>
- Record JSON: <https://awesomerea.org/resources/glaurung-models/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/resources/glaurung-models/index.md)

## Requirements & limits

- **Prerequisites:** The documented tokenizer workflow uses the tokenizers and transformers Python packages and Latin-1 conversion of binary bytes.
- **Testing evidence:** The document reports an average of 2.849 bytes per token for the 64K tokenizer.
- **License:** No licence file in the repository.

## Primary sources

1. <https://github.com/mjbommar/glaurung-models/blob/1819020f447d55f55b629b78ba96502edae71715/glaurung_binary_tokenizer_001_README.md>
2. <https://github.com/mjbommar/glaurung-models/blob/1819020f447d55f55b629b78ba96502edae71715/SSE_AVX_ANALYSIS.md>

## More in References and learning

- [CrackMes](https://awesomerea.org/resources/crackmes-writeups/index.md) · Learning resource
- [Binary BPE Tokenizer for Executable Analysis](https://awesomerea.org/resources/binary-bpe-tokenizer-research/index.md) · Research
- [Assassin's Creed III online service docs](https://awesomerea.org/resources/ac3-online-service-docs/index.md) · Documentation
- [AC4 docs](https://awesomerea.org/resources/ac4-docs/index.md) · Documentation
- [eRepublik API Documentation](https://awesomerea.org/resources/erepublik-api-documentation/index.md) · Documentation
- [Garmin Strength Workout API](https://awesomerea.org/resources/garmin-strength-workout-api/index.md) · Documentation

## About this entry

The description was reviewed against primary sources.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.