{
  "added_at": "2026-10-20",
  "additional_evidence_urls": [
    "https://github.com/buzzer-re/Rikugan/blob/a2ba6691baa0a7ace3bce9fe834a4f6f7610b77d/README.md",
    "https://github.com/buzzer-re/Rikugan/blob/a2ba6691baa0a7ace3bce9fe834a4f6f7610b77d/LICENSE",
    "https://github.com/wangyu-ovo/CREBench/blob/9fc4016538a0bab8c230732a9f788a61ed43addf/README.md"
  ],
  "category": "learning",
  "description_en": "Matthieu Breuil’s Synacktiv article explains tool-calling and code-executing agent designs and describes Frontkick, a CodeAct agent that runs generated Python inside Binary Ninja. It compares Frontkick with the open-source Rikugan plugin on two CREBench cryptographic binary tasks, then attributes most of Frontkick’s advantage to a prompt that listed installed Python libraries rather than to its architecture. It is a small evaluation write-up with per-run results, not a released tool or a general benchmark finding.",
  "description_zh": "Matthieu Breuil 在 Synacktiv 发表的文章讲解了调用预定义工具与执行生成代码两类智能体设计，并介绍在 Binary Ninja 内运行生成的 Python 代码的 CodeAct 智能体 Frontkick。文章在 CREBench 的两个密码二进制任务上将 Frontkick 与开源插件 Rikugan 对比，随后指出 Frontkick 的优势主要来自提示词中列出已安装 Python 库的做法，而非其架构本身。这是一篇附逐次运行结果的小规模评估文章，不是已发布的工具，也不是普遍性的基准结论。",
  "entry_url": "https://awesomerea.org/resources/frontkick-rikugan-crebench-comparison/",
  "entry_url_zh": "https://awesomerea.org/zh/resources/frontkick-rikugan-crebench-comparison/",
  "evidence_url": "https://www.synacktiv.com/en/publications/ufc-666-frontkick-vs-rikugan",
  "group": "learn",
  "id": "frontkick-rikugan-crebench-comparison",
  "json_url": "https://awesomerea.org/resources/frontkick-rikugan-crebench-comparison/record.json",
  "kind": "learning-resource",
  "markdown_url": "https://awesomerea.org/resources/frontkick-rikugan-crebench-comparison/index.md",
  "markdown_url_zh": "https://awesomerea.org/zh/resources/frontkick-rikugan-crebench-comparison/index.md",
  "name": "Frontkick vs Rikugan: a CREBench agent comparison",
  "name_zh": "Frontkick 与 Rikugan：基于 CREBench 的智能体对比",
  "record_type": "resource",
  "relationship": "learning",
  "requirements": [
    "The article on Synacktiv’s publications site is dated 13/10/2026. It describes Frontkick as a CodeAct agent that executes generated Python against Binary Ninja’s API, with a recipe-retrieval corpus, a single-level subagent and truncation of older turns; no download, repository or license for Frontkick is linked. Rikugan is an MIT-licensed community agent plugin for IDA Pro and Binary Ninja built on schema-validated tools, while Vector 35’s closed-source Sidekick was excluded because its internals are not accessible.",
    "The comparison used Claude Sonnet 5.5 with CREBench’s AES-128-CBC and ChaCha20 cases in the hardcode_plain/O0 setting, supplied as stripped binaries, with four runs per case for each agent and a 1M-token budget. Each run had to identify the algorithm and key and reimplement decryption to recover the flag. The article does not state the Binary Ninja, Rikugan or Frontkick versions or publish the run transcripts it inspected.",
    "In the published run table Frontkick fully solves six of eight runs and Rikugan none; the means are about 86 versus 31 out of 100 and about 0.60M versus 0.95M tokens, which the article summarizes as roughly half the tokens and nearly three times the score. On AES-128-CBC, Rikugan repeatedly tried a Python crypto package that was not installed and fell back to a manual implementation, whereas Frontkick’s prompt listed the installed libraries. The author therefore calls the outcome a draw driven by prompt design, and proposes a rerun with aligned prompts.",
    "Two tasks, one model and eight runs per agent cannot support general conclusions about either agent, about tool-calling versus code-executing designs, or about other CREBench settings."
  ],
  "requirements_zh": [
    "这篇发表于 Synacktiv 出版物页面的文章标注日期为 2026-10-13。文章介绍 Frontkick 是一个 CodeAct 智能体，针对 Binary Ninja 的 API 执行生成的 Python 代码，配有脚本示例检索语料、单层子智能体以及对较早轮次的截断机制；文中没有给出 Frontkick 的下载、仓库或许可证。Rikugan 是采用 MIT 许可、面向 IDA Pro 与 Binary Ninja 的社区智能体插件，基于经过模式校验的工具；Vector 35 的闭源 Sidekick 因无法了解其内部机制而未纳入对比。",
    "对比使用 Claude Sonnet 5.5，选取 CREBench 中 hardcode_plain/O0 设置下的 AES-128-CBC 与 ChaCha20 两个用例，以去除符号的二进制文件形式提供；每个智能体每个用例运行四次，token 预算为 100 万。每次运行都需识别算法和密钥，并重新实现解密流程以取得 flag。文章未说明 Binary Ninja、Rikugan 或 Frontkick 的版本，也未公开其查看过的运行记录。",
    "按文中公布的逐次运行表，Frontkick 在八次运行中有六次完全解出，Rikugan 一次也没有；两者平均得分约为 86 对 31（满分 100），平均 token 用量约为 60 万对 95 万，文章将其概括为 token 约减半、成功得分接近三倍。在 AES-128-CBC 上，Rikugan 反复尝试调用未安装的 Python 密码库，失败后改为手工实现算法，而 Frontkick 的提示词列出了已安装的库。作者因此认为结果是由提示词设计造成的平局，并提议在对齐提示词后重新测试。",
    "两个任务、一个模型、每个智能体八次运行，不足以得出关于任一智能体、关于工具调用与代码执行两类设计，或关于 CREBench 其他设置的普遍结论。"
  ],
  "targets": [
    "binary"
  ],
  "url": "https://www.synacktiv.com/en/publications/ufc-666-frontkick-vs-rikugan",
  "verification": "source-reviewed",
  "verified_at": "2026-10-19"
}