# VIGA rebuilds a scene by writing code, rendering it and arguing with the result > VIGA is a research code agent for programmatic visual reconstruction, built as an analysis by synthesis loop where one self-reflective agent alternates between generating Blender code and verifying the rendered output against a target image. The cost of trying it is the environment: four conda environments, a GPU, two API keys and a segment anything checkpoint. **Fugtemypt123/VIGA** — VIGA: Vision-as-Inverse-Graphics Agent - Repository: https://github.com/Fugtemypt123/VIGA - Stars: 1,333 · Forks: 126 - Language: Python - License: MIT - Published: 2026-10-19 · Updated: 2026-10-19 · Language: en - Canonical page: https://hysenlabs.com/projects/fugtemypt123-viga ## Vision treated as inverse graphics VIGA stands for Vision-as-Inverse-Graphics Agent, and the framing is the interesting part of the name. Forward graphics goes from a program to a picture. Inverse graphics goes the other way, from a picture to a program, and VIGA treats that as a code generation problem rather than a perception problem. The project describes itself as an analysis by synthesis code agent for programmatic visual reconstruction, and it approaches the task through an iterative loop of generating, rendering and verifying scenes against target images. The output is not a mesh and not a texture set. It is a program, a Blender Python script, that produces the scene when run. That choice has consequences for who should care. A perception system that outputs geometry is useful to a graphics pipeline. A system that outputs a program is useful to a person who wants to edit the scene afterwards, because the scene is described in a language they can change. It is also what makes the task evaluable, since the rendered result can be compared against the target image automatically. The audience is therefore a research group working on multimodal agents or on 3D content generation, and a developer who wants to understand how an agent can be made to correct itself. The project is a research release rather than a product: there is a project page, a paper on arXiv with a full citation block, a benchmark dataset published on Hugging Face, and a documentation set of three pages. The repository layout matches that framing. There are agents/, models/, prompts/, tools/, runners/, evaluators/, requirements/, data/, docs/ and utils/ directories alongside a main.py, which is the shape of a research codebase where the interesting parts are the prompts, the tools the agent may call, and the harnesses that run and score it. ## One agent, two roles, and a memory that grows The mechanism is a single self-reflective agent that alternates between two roles. The Generator writes and executes scene programs, using tools for planning, code execution, asset retrieval and scene queries. The Verifier examines rendered output from multiple viewpoints, identifies visual discrepancies, and provides feedback for the next iteration. The documentation describes the overall cycle as write, run, compare, revise, and the project is explicit that this loop is self-correcting and requires no finetuning. That last point is the one to weigh carefully. No finetuning means no training run, no dataset of trajectories, no weights to host, and the agent works with a model that already exists. What it also means is that the improvement comes from the loop and the prompts rather than from anything the project trained, so the ceiling is set by the base model's ability to write plausible Blender code in the first place. For a research artefact that is a reasonable trade, since the claim being tested is about the loop, not about a new model. The agent maintains an evolving contextual memory containing plans, code diffs and render history. This is the mechanism that makes the loop iterative rather than a sequence of independent attempts. The plan says what the agent intends to build, the diffs show what changed since the last iteration, and the render history lets the verifier refer to earlier attempts. Without that memory the agent would keep making the same class of mistake, and the diagnostics that the project publishes as a trajectory image are a picture of exactly that state over time. The roles are also mapped onto tools, which is how a code agent stays concrete: planning, executing code, retrieving assets and querying the scene are separate capabilities the agent chooses between, rather than one long prompt that asks for everything. ## Five modes, five different outputs The supported domains are listed as a table, and the variation across them is the clearest statement of what the project can do. BlenderBench is multi-step 3D graphics editing at levels one to three, and the output is Blender Python. BlenderGym covers single-step 3D graphics editing with the same output, and it comes from a separate project by different authors. SlideBench is a different problem entirely, 2D slide and document layout synthesis, with the output being PowerPoint rather than a 3D scene, which is a good sign of generality since nothing in the loop is specific to 3D. The two custom modes are the ones that make the system interesting to use on your own material. A custom static scene is single-view 3D reconstruction with a Blender scene as output, and a custom dynamic scene is a 4D dynamic scene with physics, producing a Blender animation. The dynamic mode is the harder of the two by an order of magnitude, since the agent has to get motion and simulation right rather than a single frame, and the sample data for it lives in data/dynamic_scene with an artist example that shows the expected format. Note what the outputs are. In every case the deliverable is source code in a specific tool's scripting language, Blender Python or PowerPoint. That is consistent with the inverse graphics framing: the agent reconstructs the scene by writing the program that a person would have written, and the render is the evidence that the program works. ## The environment is the hardest part The quickstart is a list of environment steps, and the number of them is the honest characterisation of this project. It needs Conda, and for the 3D modes an NVIDIA GPU with CUDA support is recommended. The repository and its submodules come first, along with a segment anything checkpoint: ```bash git clone https://github.com/Fugtemypt123/VIGA-release.git && cd VIGA-release git submodule update --init --recursive # download sam module wget -P utils/third_party/sam https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth ``` Then four separate conda environments, because the agent and the tools it drives have incompatible dependencies: ```bash conda create -n agent python=3.10 -y && conda activate agent pip install -r requirements/requirement_agent.txt conda create -n blender python=3.11 -y && conda activate blender pip install -r requirements/requirement_blender.txt ``` The blender environment additionally installs Infinigen through its own interactive script, run with INFINIGEN_MINIMAL_INSTALL=True, and the documentation notes that the errors from that script can be ignored as long as the expected blender path appears. Two more environments follow for the segmentation work, one for sam on Python 3.10 and one for sam3d on Python 3.11 with its own install script. Two configuration files finish the setup. The API keys are copied from an example file and filled in with an OpenAI key and a Meshy key: ```bash cp utils/_api_keys.py.example utils/_api_keys.py ``` And the conda location is configured from its own example, pointing at ~/anaconda3/envs by default through a CONDA_BASE variable that can be overridden with the VIGA_CONDA_BASE environment variable: ```bash cp utils/_path.py.example utils/_path.py ``` Two external services are therefore involved in a run, the language model and the asset service, which is worth knowing before you start a batch evaluation. ## Running the agent and supplying your own scenes A run is one Python command from the agent environment, and the arguments are the interesting part: ```bash conda activate agent python runners/dynamic_scene.py --task=artist --model=gpt-5 --generator-tools=tools/blender/exec.py,tools/generator_base.py,tools/initialize_plan.py,tools/sam3d/init.py --prompt-setting=init ``` The runner is chosen by the task, dynamic_scene.py for the 4D dynamic mode, and the task argument selects the sample within it, artist in this case. The model argument names the language model the generator and verifier will use. The generator-tools argument lists the tools the generator is allowed to call, in this case the blender execution tool, the base generator tool, the plan initialisation tool and the sam3d initialisation tool, which is a direct control over the agent's action space. The prompt-setting argument selects which prompt configuration is used, init here, and the prompts/ directory in the repository is where those live. Supplying your own material is a directory convention rather than a flag. Custom data goes into data/dynamic_scene/, following the format of the data/dynamic_scene/artist example, which means the input format is defined by example rather than by a specification, so the example directory is the reference you should read before you try your own images. For batch work there is a runners/README.md documenting the batch execution options, which is where you would look to run the same configuration across a whole benchmark set rather than a single task. ## Evaluation harness, submodules, and a research release cadence The repository has an evaluators/ directory, which is the part that makes this usable as a research instrument rather than a demo. A reconstruction agent that renders a scene can be scored automatically, since the render and the target image are both available, and an evaluation harness is what turns a demo into a result you can put in a paper. The benchmarks the project points at, BlenderBench is published separately on Hugging Face and SlideBench has its own repository, and a third-party benchmark repository, BlenderGym, is supported as well. The presence of a.gitmodules file and the submodule update step in the quickstart tells you that parts of the toolchain live in other repositories and are pinned, which is normal for a research code release that wraps several large third-party projects. The licence is MIT, with the LICENSE file at the repository root, and the citation block in the documentation gives the paper title, the author list, the year and the arXiv identifier, along with the primary class cs.CV. A project that asks you to cite it is usually a project whose value is partly the paper, so if you build on the code you should cite it. The maintenance picture is the main caveat. The repository is not archived, and the last push was on 2026-05-06, with no GitHub releases, so there is no version to pin and no changelog to check. There is a documentation set of three pages, architecture, requirements and runners, which is enough to get started and thin if you want to understand a specific tool in depth. The practical advice for anyone evaluating it is to read docs/architecture.md before the code, since the tools and prompts directories are where the design lives, and to run the artist example first, because it is the one the project ships with a known expected format and the one your own data should be modelled on. ## Conclusion VIGA suits a research team studying agent driven 3D reconstruction or evaluating a model on the BlenderBench, BlenderGym and SlideBench tasks, and it suits anyone who wants to see the generator and verifier loop in a readable codebase. It is the wrong choice if you want a finished scene reconstruction tool, because the repository is a research release with no packaged installer, no releases and a five month old last push, and a poor match if you cannot run multiple conda environments with an NVIDIA GPU and supply both an OpenAI and a Meshy API key. Before evaluating, set up the agent, blender, sam and sam3d environments as documented, put your own images in data/dynamic_scene following the artist example, and check which model the runner defaults to before reading the numbers. ## FAQ ### What is VIGA and how does it work? VIGA is a Vision-as-Inverse-Graphics Agent, an analysis by synthesis code agent for programmatic visual reconstruction. A single self-reflective agent alternates between a Generator that writes and runs scene programs and a Verifier that inspects renders from several viewpoints, in a write, run, compare, revise loop that needs no finetuning. ### What environment does VIGA need to run? Conda with four separate environments, one for the agent on Python 3.10, one for blender on Python 3.11, one for sam and one for sam3d, plus an NVIDIA GPU with CUDA for the 3D modes, a segment anything checkpoint, and an OpenAI and a Meshy API key configured in utils/_api_keys.py. ### Which tasks and benchmarks does VIGA support? BlenderBench for multi-step 3D editing, BlenderGym for single-step 3D editing, SlideBench for 2D slide and document layout synthesis with a PowerPoint output, plus custom static scene reconstruction and a custom 4D dynamic scene mode with physics that outputs a Blender animation. ### How do I run VIGA on my own images? Place your data in data/dynamic_scene/ following the format of the data/dynamic_scene/artist example, then run the runner for your mode from the agent conda environment with the task, model, generator-tools and prompt-setting arguments. ## Sources - [Fugtemypt123/VIGA on GitHub](https://github.com/Fugtemypt123/VIGA) - [Issues](https://github.com/Fugtemypt123/VIGA/issues) - [License: MIT](https://github.com/Fugtemypt123/VIGA/blob/main/LICENSE) - [README](https://github.com/Fugtemypt123/VIGA/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/fugtemypt123-viga