screenshot-to-code: turning pixels into Tailwind and React with a FastAPI backend

What it actually does and who should care

The project takes a screenshot, a mockup, a Figma design, or a screen recording and returns runnable frontend code. The supported output stacks are HTML plus Tailwind, HTML plus CSS, React plus Tailwind, Vue plus Tailwind, Bootstrap, and Ionic plus Tailwind. That breadth is the main selling point: a single tool covers both static marketing pages and component-based apps. The intended audience is developers who want a starting point for a UI, not a finished product. The hosted version at screenshottocode.com exists for people who do not want to manage API keys, but the open source code is the focus here. If you are evaluating this for a team, the key question is whether you already have access to paid model APIs, because the tool does nothing without them.

How the pipeline is wired: FastAPI, React, and a headless browser

The architecture is a React/Vite frontend talking to a FastAPI backend. The user drops an image into the frontend, and the backend sends it to one of several AI models depending on which provider keys are configured. The generation flow is not a single call. According to the README, the app can extract real assets like logos from the screenshot, and that extraction is powered by Gemini. For image editing, background removal, and image generation, it uses Replicate. The backend also has an optional screenshot preview feature: once Chromium is installed via Playwright, the agent renders its own generated page in a headless browser and visually checks the result. If Chromium is missing, the tool simply skips that step, which means the visual check is skipped (the Settings dialog shows whether preview is available). That is a design choice worth noting: the preview is an enhancement, not a hard dependency, so a minimal setup will produce code without any visual verification.

Getting it running: Poetry, pnpm, and four API keys

The setup is explicit and mostly copy-paste. The backend lives in the backend directory and uses Poetry. You create a .env file with keys for OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY, and REPLICATE_API_KEY. The README says you need at least one of the first three, but it strongly recommends Gemini and Replicate for best accuracy. After writing the keys, you run poetry install, then poetry run playwright install chromium to install the browser used by the screenshot preview. On Linux, the command includes --with-deps to install system libraries with sudo. Then you start the backend with poetry run uvicorn main:app --reload --port 7001. The frontend is a separate pnpm project: cd frontend, pnpm install, pnpm dev, and the app appears at http://localhost:5173. There is also a Docker path: write the keys to a root .env file and run docker-compose up -d --build. The Docker setup is not for development because file changes will not trigger a rebuild. The README also mentions that OpenAI, Anthropic, and Gemini keys can be set through the frontend settings dialog, but Replicate must be in backend/.env. That asymmetry is a small friction point.

Model mix matters more than the code generation itself

The quality of the output depends less on the tool's own logic and more on which models you feed it. The README lists Gemini 3 Flash Preview and Gemini 3.1 Pro Preview as the best models, with GPT-5.5 and GPT-5.4 Mini, plus Claude Opus 4.6 and 4.8 as options. With a single key, the app uses only that provider's models. With multiple keys, it automatically picks a stronger mix per variant. The implication is that your results will vary wildly depending on your budget and provider access. The README goes so far as to say that running with Ollama open-source models is not recommended due to poor-quality results. That is a direct warning: this project is a wrapper around commercial APIs, not a standalone code generator. If your organization cannot use external model providers for privacy or cost reasons, this tool is likely the wrong choice.

A real limitation: the preview tool and the video mode are conditional

Two features in the README are not guaranteed to work out of the box. The screenshot preview, which renders the generated page in a headless browser, only activates if Chromium is installed. The README says the Settings dialog shows whether the preview is available on your backend. If you skip the Playwright install step, you lose that verification loop. The second conditional feature is video mode: turning a screen recording into a functional prototype. The README states that a GEMINI_API_KEY is required for video mode. That means users who only have OpenAI or Anthropic keys cannot use the recording feature at all. These are not bugs, but they are constraints that are easy to miss in the marketing copy. Before relying on either feature, you should check the Settings dialog in the running app to confirm the backend has the necessary pieces.

Alternatives and how they differ

The most direct alternative is to use a hosted tool like v0 by Vercel or similar design-to-code services. The difference is control: screenshot-to-code is self-hostable, so you keep the application code on your own infrastructure, although the images still go to whichever model provider you configure, and you can modify the backend or frontend. Hosted tools are easier to start with, but they lock you into their model choices and their pricing. Another alternative is to skip the AI wrapper entirely and use a traditional HTML-to-code tool like Anima or a Figma plugin that exports code from design files. Those tools do not generate code from arbitrary screenshots; they work from structured design files with layers and constraints. screenshot-to-code is more flexible in input type, but it depends on the model interpreting the pixels correctly, which is a fundamentally different and less deterministic process. If you need reproducible output from a design system, a Figma export plugin is the safer bet.

Maintenance and licensing: MIT, but not free to run

The project is licensed under MIT, so you can use, modify, and redistribute it without royalty obligations, as long as you preserve the license notice. That is a permissive license, but the operational cost is not zero. You need paid API keys for at least one model provider, and the README recommends Replicate for image editing, which is another paid service. The maintenance burden is on you: the README points to a GitHub issue comment for backend setup errors and to Troubleshooting.md for getting an OpenAI API key, which suggests that the setup is not always smooth. The project has an active default branch and is not archived, but there are no recent releases listed, so you are essentially tracking the main branch. That means you should expect occasional breaking changes when you pull updates. The frontend dependencies are managed with pnpm, and the backend with Poetry, so you need to keep both toolchains in sync. If you are not comfortable maintaining a two-part app with external API dependencies, the hosted version is the lower-effort path.

Editorial conclusion

Adopt screenshot-to-code if you need a self-hostable pipeline that turns visual designs into frontend code across six stacks, and you are willing to pay for at least one model provider key and, ideally, Gemini plus Replicate for asset extraction and image editing. Skip it if you want a fully offline tool or if you expect high accuracy from a single weak model, since the README itself warns that Ollama models produce poor results. Before committing, verify that your chosen provider keys work with the current model list, confirm that the Chromium preview tool is installed on your backend, and check the README FAQ and Troubleshooting.md for setup help, especially on Windows where UTF-8 encoding in .env files is a documented pitfall.

Frequently asked questions

Official sources

1. Official documentation
2. Official README
3. Project repository