PaperBanana: an unofficial multi-agent pipeline for turning method text into paper figures

The gap PaperBanana targets: method sections that need figures nobody wants to draw

A paper's methodology figure is usually drawn by hand in a vector editor or glued together from matplotlib panels, then redrawn every time the architecture changes. PaperBanana's stated purpose is to take a text description of a method and produce a publication-quality diagram or statistical plot from it. The README frames the audience as "AI Scientists", and the surrounding topics list (arxiv, neurips, academic-diagrams) makes the intent explicit: researchers writing machine learning papers who want a first draft of a figure rather than a finished one. The repository is an unofficial, community-driven implementation of the paper "PaperBanana: Automating Academic Illustration for AI Scientists" by Dawei Zhu, Rui Meng, Yale Song, Xiyu Wei, Sujian Li, Tomas Pfister and Jinsung Yoon, and it carries an explicit disclaimer that it is not affiliated with or endorsed by the original authors or Google Research, and that the implementation "may differ from the original system". That sentence matters more than it looks. If you are citing the method in a paper, the artifact you are citing is the arXiv paper, not this code. This repository is a reimplementation with extensions, and the extensions (slide generation, a Gradio studio, batch manifests, an MCP server) are not in the original paper's scope.

Two phases, one refinement loop, and an input optimization layer in front

The README describes the core as a "two-phase multi-agent pipeline with iterative refinement". It does not enumerate the agents or their interfaces, so the only architecture claim that can be verified from the supplied material is the shape: multiple agents, two distinct phases, and a loop that can run again with feedback. Two adjacent features fill in the picture. First, an "input optimization layer for better generation quality", which sits ahead of the pipeline and rewrites or enriches the caller's text before any agent sees it. Second, "auto-refine mode and run continuation with user feedback", which implies the pipeline persists intermediate state between runs so a user can comment on a draft and resume rather than restart. The practical consequence is that the prompt you write is not the prompt the model receives. If the output drifts from your intent, the input optimization layer is the first place to look, and the README does not document a switch to disable it. Providers are selected by configuration rather than by code: OpenAI (the README names GPT-5.2 and GPT-Image-1.5), Azure OpenAI or Foundry via OPENAI_BASE_URL, Google Gemini, and Atlas Cloud. A VLM handles understanding and a separate image model handles rendering, which is why the Gemini overrides split into GOOGLE_VLM_MODEL and GOOGLE_IMAGE_MODEL. That split is the design decision to notice: the quality of a figure depends on two models from potentially two vendors, and swapping the image model alone changes the output without touching the reasoning.

Install, key configuration, and the first generate call

Installation is a single pip command, `pip install paperbanana`, or a source install with extras: `pip install -e ".[dev,openai,google]"`. Python 3.10 or newer is required. Credentials come from a .env file created by `cp .env.example .env`, with OPENAI_API_KEY and GOOGLE_API_KEY as the two main keys. Azure endpoints are handled by pointing OPENAI_BASE_URL at a resource URL of the form https://<resource>.openai.azure.com/openai/v1. Gemini has three optional overrides: GOOGLE_BASE_URL for a proxy, GOOGLE_VLM_MODEL (the example sets gemini-2.5-flash) and GOOGLE_IMAGE_MODEL (the example sets gemini-3-pro-image-preview). There is also a wizard, `paperbanana setup`, which the README presents as the Gemini path. Generation is a single subcommand: `paperbanana generate --input examples/sample_inputs/transformer_method.txt --caption "Overview of our encoder-decoder ..."`. The README also gives a Docker path, building with `docker build -t paperbanana .` and running with `-e GOOGLE_API_KEY` plus a volume mount that maps an input file to /work/method.txt read-only and a host outputs directory to /work/outputs. Note that the Docker example passes GOOGLE_API_KEY while the prerequisites section lists OpenAI first; both are supported, but the .env keys, the install extras and the default model names are three separate places where the provider choice is expressed, and they can disagree. Confirm which one your run actually used before you blame the model for a bad figure.

Batch manifests, PDF context, and the surfaces beyond the CLI

Beyond single calls, the README lists batch generation from a YAML or JSON manifest for multiple diagrams in one run, and a separate `paperbanana plot-batch` command that runs many statistical plots from one manifest with a CSV or JSON file per item. There is PDF input support for methodology context, gated behind an optional extra (`paperbanana[pdf]`, PyMuPDF) and with per-page selection, so you can point the pipeline at a draft and pick the pages that describe the method. PaperBanana Studio is a local Gradio UI started with `paperbanana studio`, covering diagrams, plots, evaluation, batch and a run browser. For editor workflows there is an MCP server, registered under the name io.github.llmsresearch/paperbanana, and Claude Code skills for `/generate-diagram`, `/generate-plot` and `/evaluate-diagram`. The presence of an evaluation surface and a run browser suggests the intended loop is draft, inspect, refine, rather than one-shot generation. The batch manifest is the feature most likely to matter in practice: a paper with six figures and four plots is a batch job, not twelve interactive sessions, and the manifest is the only documented way to express that.

Where the pipeline breaks down, and when it is the wrong tool

The most concrete limitation is stated by the project itself: it is unofficial and may differ from the original system. If your goal is to reproduce the paper's results, this repository is not the reference implementation. A second limitation is structural. The pipeline routes through hosted models from OpenAI, Azure, Gemini or Atlas Cloud, so the method text you feed it leaves your machine. For unpublished work under review, or for anything under a data agreement that forbids third-party processing, that alone disqualifies the tool regardless of output quality. A third is that image generation models are not deterministic in the way vector drawing is. A diagram produced this way is a raster image, and the README never claims a vector or editable output format. Redrawing a slightly wrong box in a generated raster is often slower than drawing the figure from scratch, which inverts the value proposition for simple architectures. Fourth, the input optimization layer rewrites your text before generation and is not documented as optional, so a bad result is hard to attribute. Fifth, the provider matrix is wide but the documentation is not uniform across it: the prerequisites section leads with OpenAI, the .env example leads with a Gemini wizard, and the model identifiers in the override block are examples rather than a compatibility table. There is no stated evaluation of how well any single provider performs, and the supplied material contains no accuracy or quality numbers at all.

What it is not: a diagram-as-code tool, and the difference that makes

The obvious alternative for architecture figures is a declarative diagram tool such as Mermaid or Graphviz, or TikZ if you are already writing LaTeX. The difference is not quality, it is where the specification lives. In Mermaid or Graphviz you write nodes and edges; the renderer is deterministic, the source is diffable, and a reviewer can see exactly what changed between two versions of the figure. PaperBanana inverts that: you write prose, a VLM interprets it, an image model renders it, and the artifact is a bitmap with no source of truth beyond the text you supplied. That makes PaperBanana better at layout and visual style, which is where hand-written diagram code is tedious, and worse at reproducibility and incremental edits, which is where diagram code wins. A reasonable division is to use PaperBanana for the first draft of a complex figure whose composition you have not settled, then rebuild the final version in TikZ or Graphviz once the structure stops changing. The other alternative is the original PaperBanana release from the paper's authors, if and when it is available; this repository is explicitly a community reimplementation and the README does not claim parity with it.

Maintenance, licensing, and what an upgrade actually costs you

The repository is MIT licensed, which permits commercial and academic use, modification and redistribution provided the copyright notice and permission notice are retained. That is a permissive license with no copyleft obligation, and it is the same license the README badge advertises. Two practical notes, offered as observations rather than legal advice: MIT covers the code in this repository, not the figures you generate, and it does not cover the upstream paper or any assets the project links to, such as the logo image hosted on the original authors' site. If you redistribute generated figures, the terms that apply are your model provider's, not MIT's. On maintenance, the release history shows v0.2.0 and v0.3.0 one day apart in June 2026, a dataset mirror tagged bench-data-v1, and a last push in September 2026, which is consistent with an actively developed project but also with rapid churn. The upgrade cost is concentrated in two places: the .env keys and the model identifiers. Model names in the override block are versioned strings (gemini-2.5-flash, gemini-3-pro-image-preview), and provider deprecations will force edits there rather than in your code. Pin the package version in your environment and re-run a known input after each bump, because the input optimization layer and the refine loop both sit between your text and the output, and a change in either can shift results without any change on your side.

Editorial conclusion

Adopt it if you already hold a Gemini or OpenAI key and want a scriptable, MIT-licensed way to draft methodology figures and statistical plots from text, with batch manifests and an MCP server for editor integration. Do not adopt it if you need a figure that survives a reviewer's reproducibility check without manual redrawing, if you cannot send unpublished method text to a hosted model, or if you need a vendor-neutral abstraction you control. Before installing, verify which provider your credentials actually reach, since the README's install extras, .env keys and default model names are three separate places where the choice is made, and confirm the current GOOGLE_VLM_MODEL and GOOGLE_IMAGE_MODEL values against your own account rather than the example block.

Official sources

1. Issues
2. License: MIT
3. llmsresearch/paperbanana on GitHub
4. README
5. Releases