# Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search

> GIF-MCTS generates executable Python transition and reward models from natural-language environment descriptions and offline state-transition trajectories, using prediction feedback to generate, improve and repair code. CWMB evaluates 18 reinforcement-learning environments and downstream planning; a separate experiment uses a simplified Read to Fight Monsters grid world.

- Authors: Nicola Dainese; Matteo Merler; Minttu Alakuijala; Pekka Marttinen
- Venue: NeurIPS 2024
- Status: Conference paper
- Publication / posting date: 2024 (exact date not verified)
- Date source: The final NeurIPS proceedings and publisher-deposited Crossref record identify 2024. No publication month or day is asserted; conference event dates are not substituted for publication dates.
- First arXiv submission: May 24, 2024
- arXiv: 2405.15383
- DOI: 10.52202/079017-1933
- Sources reviewed: 2026-10-17
- Added to the catalog: 2026-10-17
- Original paper: <https://proceedings.neurips.cc/paper_files/paper/2024/hash/6f479ea488e0908ac8b1b37b27fd134c-Abstract-Conference.html>
- PDF: <https://proceedings.neurips.cc/paper_files/paper/2024/file/6f479ea488e0908ac8b1b37b27fd134c-Paper-Conference.pdf>
- Code: <https://github.com/nicoladainese96/code-world-models>
- Project: <https://sites.google.com/view/code-world-models/home>
- BibTeX: <https://awesomerea.org/papers/citations/paper-code-world-models.bib>
- Catalog page: <https://awesomerea.org/papers/paper-code-world-models/>
- Record JSON: <https://awesomerea.org/papers/paper-code-world-models/record.json>
- Chinese version: [简体中文](https://awesomerea.org/zh/papers/paper-code-world-models/index.md)

## Scope and limitations

Assumes deterministic, fully observable dynamics and sufficiently detailed descriptions; this is not recovery from pixels or binaries alone. Accuracy uses the replay transitions supplied during synthesis, alongside planning-return evaluation, rather than proving correctness on all unseen states or full-engine equivalence. RTFM fixes the manual across episodes and keeps monsters stationary. CWMB also includes non-game control and physics tasks; some planning returns are unavailable and complex continuous-state models remain difficult.

## Why it fits this collection

Shows how documented rules and observed transitions in game-like environments can become executable behavior models with test feedback. It complements game reconstruction without claiming recovery of original engines, graphics or assets.

## Publication and review notes

Reviewed the final proceedings paper, pages 60429–60474. Its WorldCoder baseline follows the first arXiv version, not later revisions. The Apache-2.0 repository provides prompts, replay-buffer files and evaluation scripts; bundled RTFM carries CC BY-NC 4.0. The pinned evaluator balances reward classes, grants partial credit over next-state components, skips terminal next-state checks, and accepts nonempty valid-action subsets for RTFM; continuous states use 1e-5 relative/absolute tolerance. The planning script defaults to two episodes, while the paper reports ten. Reproduction needs the Linux-oriented Conda/Python 3.10 environment, Gymnasium/MuJoCo and model access.

## Primary sources

1. <https://proceedings.neurips.cc/paper_files/paper/2024/hash/6f479ea488e0908ac8b1b37b27fd134c-Abstract-Conference.html>
2. <https://proceedings.neurips.cc/paper_files/paper/2024/file/6f479ea488e0908ac8b1b37b27fd134c-Paper-Conference.pdf>
3. <https://api.crossref.org/works/10.52202/079017-1933>
4. <https://arxiv.org/abs/2405.15383>
5. <https://sites.google.com/view/code-world-models/home>
6. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/README.md>
7. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/LICENSE>
8. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/RTFM/LICENSE>
9. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/environment.yml>
10. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/src/mcts/gif_mcts/simulators/simulator_cwm.py>
11. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/src/cwm_benchmark/generate_env.py>
12. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/src/experiments/eval_cwm_planning.py>
13. <https://github.com/nicoladainese96/code-world-models/blob/e17e787b41137793bcef9f860bdfe9e6fc07b768/data/prompts/rtfm/generate.md>

## About this entry

Reviewed against the paper and its primary sources. Cite the original authors using the linked paper or BibTeX.
Original editorial content: MIT. Linked projects, papers and assets retain their own rights and licenses.