Modly: a desktop shell for local image-to-3D model extensions
Desktop app to generate 3D models from images or prompt using local AI — runs entirely on your GPU
At a glance
- What is it?
- Modly is an Electron and TypeScript desktop app that turns photos into 3D meshes by running open source generation models on your own GPU. Its real design decision is that the models live outside the app, in separately installed GitHub extensions.
- Who is it for?
- Adopt Modly if you want image-to-3D generation to stay on your own machine and you are willing to install model extensions separately from the app. Skip it if you need a single-installer experience or you are on Intel macOS, which the platform notes do not cover.
- Can I use it commercially?
- Check first. The repository uses a licence we do not classify automatically, so read its LICENSE file before any commercial use.
- Is it still maintained?
- Yes. The repository last received commits 2 days ago.
- What is it written in?
- Mainly TypeScript, according to GitHub's language statistics.
Answers come from the project's GitHub data, last synced on September 15, 2026, and from our analysis. They are not legal advice.
DEEP OPEN-SOURCE ANALYSIS
The problem is not generation, it is where the weights run
Image-to-3D tools generally fall into two groups. Hosted services send your photo to someone else's GPU and return a mesh, which is fine until the subject is under NDA or the service changes its pricing. Local pipelines keep the photo on your machine but usually mean cloning a research repository, resolving CUDA or MPS versions, and writing your own glue between the model and a viewer. Modly targets the second group and tries to remove the glue.
The README describes it as a desktop application for Windows, Linux, and Apple Silicon macOS that turns a photo into a 3D model using open source AI models running entirely on your GPU. The audience is therefore people who already have a capable GPU and would rather not rent one. That includes game and prop artists working on unreleased assets, and engineers who want mesh generation available as a local step in a scripted pipeline rather than an API call. The repository topics list self-hosted and ai-local, which matches the framing.
One thing worth noting about the positioning: Modly does not ship the generation models itself. The models arrive as extensions, covered below. So the app is closer to a host environment than to a bundled model release.
An Electron front end, a FastAPI backend, and models that live in other repositories
The repository is TypeScript first, and the development instructions split cleanly into a JS side and a Python side. `npm run dev` starts the app, and step two of the getting started guide creates a virtual environment under `api/` and installs `requirements.txt` there. The CLI documentation refers to starting the FastAPI backend, which is the Python process the desktop app talks to. So the data flow runs from the Electron UI down to a local HTTP service, which in turn drives the generation model.
The extension system is the part that shapes everything else. According to the README, each extension is a GitHub repository containing a `manifest.json` plus the runtime entry files required by its type. Extensions come in two kinds: model extensions and process extensions. Model extensions expose model nodes, and after installation you download the model or one of its variants. Process extensions, in the README's wording, are ready once installation and setup complete.
Five official extensions are listed: Hunyuan3D 2 Mini, its Turbo and Fast variants, TripoSG, and Trellis2 GGUF. Three of the five are variants of the same base model, which suggests the practical choice is mostly about speed versus quality within one family rather than across architectures. The README does not state which GPU vendors or VRAM thresholds each extension needs, and that is a real gap for anyone deciding before installing.
Workflows are the user-facing abstraction. The README's example is Image to Generate Mesh to Add to Scene, with a note that each step must be connected. The platform notes add that workflow wiring is validated before run, and that invalid graphs stay in place and surface inline or toast warnings instead of dropping the current mesh view. That is a small but deliberate choice: a broken graph does not cost you the mesh you were already looking at.
Getting it running, and the CLI that skips the UI entirely
The README gives two paths. The first is to download an installer from the Releases page for Windows, Linux, or Apple Silicon macOS. The second is to clone the repository and run `launch.bat` on Windows or `./launch.sh` on Linux and macOS. For development, the sequence is `npm install`, then inside `api/` creating a venv and running `pip install -r requirements.txt`, then `npm run dev` from the repository root. The test commands are listed as `npm test`, a type check via `./node_modules/.bin/tsc --noEmit -p tsconfig.node.json`, and `npm run build`. Packaging for Apple Silicon uses `npm run package:mac`.
Installing a model happens in the UI, not the terminal. You go to the Models page, click Install from GitHub, paste the HTTPS URL of the extension repository, and confirm. If the extension exposes model nodes, you then download a model or one of its variants.
For scripted use there is a stdlib-only CLI under `tools/modly-cli/`, and it talks to a running desktop app rather than starting one. The canonical commands are `health`, `model`, `workflow-run`, `capability`, and `process-run`. The convenience wrapper is `generate`:
python tools/modly-cli/agent.py generate --image ./input.png --output ./export.glb
That command posts to `/workflow-runs/from-image`, polls the returned run, exports the mesh when asked, and puts recovery metadata such as `workflow-run status ...` and `workflow-run cancel ...` into the JSON it prints. The README is unusually explicit about what is not canonical: `legacy` wraps the older `/generate/*` endpoints, `dev serve-api` and `dev ensure-server` start only the FastAPI backend and do not prove the Electron bridge is ready, and the `experimental comfy-image` and `experimental generate-from-workflow` commands are external ComfyUI orchestration helpers. Hidden aliases like `status`, `export`, and `batch` still parse but are not presented as root commands. `tools/modly-cli/SKILL.md` holds the output contract.
The extension boundary is also the failure boundary
Because models are installed from third-party GitHub repositories, the app's capability depends on code that is not in this repository. The README says each extension is a GitHub repository with a manifest and runtime entry files, but it does not describe a signing step, a version compatibility rule, or what happens when an extension expects an API the installed app no longer has. Given that releases are still labelled Beta and v0.4.0 through v0.4.2 landed roughly two months apart, extension drift is a plausible failure mode rather than a hypothetical one. If you install an extension and the model node fails to appear, the first thing to check is whether that extension targets the app version you are running, and the README does not tell you how to confirm that.
There is a second boundary worth naming. The CLI documentation states that `dev serve-api` and `dev ensure-server` start only the FastAPI backend and, in its own words, do not prove Electron/Desktop bridge readiness. That means a health check against the Python service can pass while the desktop side is not usable. If you are wiring Modly into automation, `health` is the command to trust, not a successful backend start.
The wrong-tool case is straightforward. If you want one installer that produces meshes without further setup, Modly's extension model works against you, because the model download is a separate step performed in the UI. And if you are on Intel macOS, the platform notes say macOS support targets Apple Silicon only, so this is not the tool for that machine.
How this differs from a bundled pipeline like ComfyUI
ComfyUI is the obvious comparison, and Modly itself acknowledges it by shipping `experimental comfy-image` and `experimental generate-from-workflow` commands. The difference is where the graph lives. ComfyUI is a general node graph editor: you assemble the pipeline yourself, and image-to-3D is one graph among many you can build. Modly fixes the graph shape around a small set of named steps (the README's example is Image to Generate Mesh to Add to Scene) and pushes the variability into installable extensions instead. You choose a model by installing an extension, not by rewiring nodes.
That trade is real in both directions. Modly's workflow validator, which keeps an invalid graph in place and reports inline or toast warnings, is a feature a general graph editor does not give you by default. Against that, a ComfyUI user can swap in a custom sampler or a post-processing node that no Modly extension exists for. The `experimental generate-from-workflow --workflow <name> --output <path>` command is the bridge between the two worlds, and the README describes its behaviour precisely: when the ComfyUI workflow produces a downloadable 3D asset, the CLI downloads it directly, while image-only workflows fall back to Modly's own image-to-3D generation. That fallback is the tell. Modly is not trying to be a general orchestrator, it is trying to be a predictable one.
If your requirement is a specific research model with a specific checkpoint that nobody has packaged as a Modly extension, ComfyUI or a direct Python pipeline is the shorter path, and writing a Modly extension is the longer one.
Licence, forking, and what the NOASSERTION label means here
The repository metadata reports the licence as NOASSERTION, while the README's own License section states MIT and points at a LICENSE file. Those two signals disagree, and the README is the more specific of the two. Treat the discrepancy as something to resolve by reading the LICENSE file in the repository before you rely on either.
The README attaches a condition that goes beyond a bare MIT grant. It states that if you fork the project and build your own app from it, you must credit the original project and its creator, with a specific attribution line: Based on Modly by Lightning Pixel. The README then says this is a requirement of the MIT license. That framing is worth pausing on, because MIT's standard text does not impose an attribution requirement on derivative applications in those terms. Whether the requirement is enforceable, and how it interacts with the LICENSE file, is a question for a lawyer, not for this article. What is clear from the README is the author's intent, and anyone forking Modly should assume they are expected to carry that credit line.
Note also that the extensions are separate repositories. Their licences are not described in this README, and the models they download may carry their own terms. Installing an extension is not the same as accepting the terms of the model weights it fetches.
Upgrade cost and who should install this
The maintenance picture visible in the material is a Beta app with three releases over roughly two and a half months, the most recent being v0.4.2 in late August 2026, with the last push to the default branch in early September 2026. That cadence means upgrades will be frequent enough that you should expect to re-check extensions after each one, since the README gives no compatibility guarantee between app versions and extension manifests.
The upgrade surface itself is small if you installed from the Releases page: a new installer, then a check that your installed extensions still expose their model nodes. If you run from source, the cost is the two-step update, `npm install` plus `pip install -r requirements.txt` inside `api/`, followed by the `npm test` and `tsc --noEmit` checks the README lists. There is no migration or database concern described, and the workspace holds meshes rather than a persistent project format that the README documents.
Adopt Modly if local-only generation matters to you, if your GPU is on Windows, Linux, or Apple Silicon, and if you are comfortable treating model installation as a separate step. Do not adopt it if you need a fully bundled offline installer, if you are on Intel macOS, or if your target model has no extension. Verify first by installing one official extension, running the Image to Generate Mesh to Add to Scene workflow, and reading the Settings/Logs/Errors panel, which the README names as the place generation problems surface.
Editorial conclusion
Adopt Modly if you want image-to-3D generation to stay on your own machine and you are willing to install model extensions separately from the app. Skip it if you need a single-installer experience or you are on Intel macOS, which the platform notes do not cover. Before committing, install one official extension from the Models page, run the Image to Generate Mesh to Add to Scene workflow end to end, and check the Settings/Logs/Errors panel for anything the workflow validator flagged.
Community notes