# Firecrawl web agent: a research agent you can fork > The firecrawl/web-agent repository ships the open-source foundation of Firecrawl's hosted research agent, built on the Deep Agents harness with skills, parallel subagents and structured output. It is aimed at developers who want the loop under their own control, and it carries the rough edges of a repository with no tagged releases. **firecrawl/web-agent** — 🔥 Open-source web data agent optimized for structured web research - Repository: https://github.com/firecrawl/web-agent - Website: https://firecrawl.dev - Stars: 1,248 · Forks: 158 - Language: TypeScript - License: MIT - Published: 2026-10-19 · Updated: 2026-10-19 · Language: en - Canonical page: https://hysenlabs.com/projects/firecrawl-web-agent ## The hosted agent has an open source foundation Firecrawl runs a research-grade autonomous agent at firecrawl.dev/app/agent, powered by Spark 1 models tuned for structured web research. The firecrawl/web-agent repository is the open-source foundation of that hosted product. The README is explicit about the intent: fork it, swap models, add skills, and deploy however you want. That framing matters because it identifies the actual user. This is not a library you add to a project. It is a repository you copy and modify, aimed at developers who want the research loop, the tool wiring and the output handling in their own codebase and their own deployment, rather than calling a hosted agent over the network. The repository is organised as a directory of two things: agent-core for the core agent logic, orchestrator, skills and tools, and agent-templates for deployment templates covering Next.js, Express and a library integration. The top level is otherwise thin, holding the README, the licence and a small internal directory. This is a scaffold you read and copy from, not a package you install and call. The description calls it an open-source web data agent optimized for structured web research, and the mechanics below are what that phrase amounts to in practice. ## Deep Agents supplies the loop, the subagents and the skill loader The runtime is not written from scratch. The harness is Deep Agents from LangChain, and the README credits it with three specific things: the plan, act, observe and repeat loop, parallel subagent spawning through a task tool, and on-demand SKILL.md loading. The agent itself combines web tools with a model in a loop where it plans, acts, observes and repeats until the task is done. The distinction between the harness and the product is worth keeping straight. Deep Agents provides the agent loop, subagent spawning, skills loading and context management. The agent-core package wires Firecrawl's tools into that runtime and adds structured output and streaming on top. The tool set is short and specific: Search, Scrape, Interact for browser automation, and bash. These are delivered through the firecrawl-aisdk package, the Vercel AI SDK integration for Firecrawl. Interact is the reason the setup step in this repository is unusual, and it comes back later in the article. Because the harness is a third party project, the upgrade path is not entirely yours to control. ## Skills are plain SKILL.md files found in one directory Skills are described as reusable SKILL.md playbooks. The discovery rule is concrete: they are auto-discovered from agent-core/src/skills/definitions/, and loaded on demand through the Deep Agents skills middleware. Two properties follow from that. First, a skill is a file, not a code module, so adding one means dropping a markdown document into a directory rather than writing and registering a class. That is a low cost for a capability, and it is also a plain text surface with no version control beyond the repository itself. Second, because loading is on demand, the full set of skills is not in the context at all times, which is the reason the harness lists context management as its own responsibility. For a team building on this repository, the practical benefit is that domain procedures can be authored in prose and versioned alongside the agent. The cost is that the loading rules, the ordering, and the arguments a skill can accept are decided by the Deep Agents middleware, and this repository's README does not document them. The examples table lists a With Skills example under agent-core/examples, so the intended shape of a skill file is available in the repository even if the README does not spell out the format. ## Each subagent gets its own browser session Subagents are spawned through the Deep Agents task tool, and the README describes them as parallel workers for independent tasks. Each one has its own tool set and its own session state, with an isolated interact browser session given as the example. That last point is the mechanism that makes parallel web work viable. If two workers shared one browser session, interleaved navigation and form state would corrupt each other, and the obvious tasks for a research agent, reading several sites at once or running several extraction jobs at the same time, would serialise by accident. Per-worker session state turns parallelism into something the runtime can actually promise. The agent-core examples include a Parallel Subagents example, which is the file to read if you are wiring this yourself. The limit is worth stating too. Subagents work on independent tasks, and the parallelism buys throughput rather than shared reasoning. A subagent that discovers a contradiction with another subagent's finding has no obvious channel to report it, and the README does not describe one. If your research question needs cross checking between sources, that arbitration has to happen above the subagent layer. ## Scaffolding an agent takes two CLI commands The project documents a two step start. The first installs the Firecrawl CLI and authenticates, and it enables the browser because the agent needs it for the Interact tool: ```bash # 1. Install the Firecrawl CLI and authenticate npx -y firecrawl-cli@latest init -y --browser # 2. Scaffold an agent project firecrawl create agent -t next ``` You should end up with a Next.js agent project that includes the chat UI, streaming, skills, subagents and structured output. Two other templates are available through the same command, and both are named in the template table: an Express template for an API server with skills, subagents and structured output, and a library template that gets you the agent core on its own. ```bash firecrawl create agent -t express ``` The browser flag in the init step is the detail to notice. The agent exposes Interact as a first class tool, and the tool list includes bash as well, so the default tool set expects a sandboxed environment with a browser and a shell rather than a bare Node process. The scaffolding command pins itself to the latest CLI release, which is convenient for a first run and something to control later if you intend to keep the project. ## Structured output runs through formatOutput and a bash sandbox The output path has two halves, and the naming is specific enough to be worth using as search terms in the code itself. Results are shaped through formatOutput, which returns JSON, and data processing is done through bashExec, a set of bash tools powered by just-bash from Vercel Labs. Using a shell for post processing is a design decision with clear trade-offs. It gives an agent a general way to reshape data, count rows, filter and reformat without every transformation needing a bespoke tool, and shell output is compact compared with the token cost of a large JSON blob. It also hands the agent a general execution surface, which is why the tool list in the README groups bash alongside the web tools rather than burying it. The agent-core examples list a Structured output example and a Streaming example, and the Next.js template is described as including streaming, so the streaming path is a supported part of the stack rather than an add on. The README does not document what just-bash permits inside the sandbox, which is the question to answer before letting an agent run shell commands over data it fetched from the open web. Treat the sandbox boundaries of that dependency as part of your own review. ## Each layer is a separate install, and the lower ones are enough for many jobs The stack is drawn as a ladder, and each rung is a separate package. At the bottom sits the REST API, usable from any language. Above it is the Firecrawl SDK, described as the core API client for Scrape, Search, Crawl and Extract. Above that is the Firecrawl AI SDK, which exposes Search, Scrape and Interact as Vercel AI SDK tools. Above that sits agent-core, the orchestrator. Above it are the two application templates. ```bash npm i @mendable/firecrawl-js ``` ```bash npm i firecrawl-aisdk ``` That layering is the real alternative to adopting the agent. If your requirement is to fetch and parse pages inside an application you already control, the SDK layer gives you Search, Scrape, Crawl and Extract without an agent loop, without a model, and without skills or subagents. The agent layers add value when the task is open ended and the sequence of steps is not known in advance, when you want the plan, act and observe behaviour to handle the ordering, and when the result should arrive as structured JSON rather than as text you parse yourself. So the honest comparison is not between two agents. It is between an agent that decides what to fetch and a client that fetches exactly what you told it to. Pick the SDK when the steps are known, the agent when they are not. ## No tagged releases and a last push in April 2026 Two facts about the state of the repository should shape any adoption decision. First, the repository has no GitHub releases, so there is no version to pin, no changelog to read, and no upgrade path other than the source tree itself. The scaffold command pulls the latest CLI at the time of the run, which gives you a moving target in a different way. Second, the last push was on 2026-04-19. That date should be treated as the boundary of the code you would be forking. The licence is MIT, which is the simplest case to reason about: the code can be copied, modified and shipped inside a product without copyleft obligations. That is not legal advice, it is simply the shape of the grant. Two further gaps are visible. The hosted agent runs on Spark 1 models, and the repository does not document which model the open-source core defaults to or how a different model is wired in, even though swapping models is presented as a headline reason to fork. ## Conclusion Adopt firecrawl/web-agent if you need a self-hosted research agent over Firecrawl's search, scrape and browser tools and want to fork the loop, replace the model, or add your own SKILL.md playbooks. Do not adopt it if you only need Firecrawl's API primitives, since the Firecrawl SDK or the AI SDK package give you those without a harness, and do not treat it as a stable dependency while the repository publishes no GitHub releases. Verify first that the Deep Agents harness and the Spark model endpoint still match what your stack expects, and confirm the browser enabled init path works in your environment, since the setup step requires it. ## FAQ ### How do I create a new Firecrawl web agent project? Install and authenticate the Firecrawl CLI with the browser flag, then scaffold with firecrawl create agent -t next. The Next.js template includes the chat UI, streaming, skills, subagents and structured output, and Express and library templates are also available. ### What harness does the Firecrawl web agent use? It uses Deep Agents from LangChain, which provides the plan, act and observe loop, parallel subagent spawning through a task tool, on-demand SKILL.md loading and context management. The agent-core package wires Firecrawl's tools into that runtime. ### Where do I add my own skills? Skills are reusable SKILL.md playbooks that are auto-discovered from agent-core/src/skills/definitions/ and loaded on demand through the Deep Agents skills middleware. ### Do I need the full agent to use Firecrawl's scraping features? No. The Firecrawl SDK is the core API client for Scrape, Search, Crawl and Extract, and the Firecrawl AI SDK exposes Search, Scrape and Interact as Vercel AI SDK tools, both usable without the agent layers. ### What licence is the firecrawl/web-agent repository under? It is released under the MIT licence, and the repository README states the licence as MIT. ### Does the web agent repository publish versions? No. The repository has no GitHub releases, and the last push was on 2026-04-19, so there is no tagged version to pin to. ## Sources - [firecrawl/web-agent on GitHub](https://github.com/firecrawl/web-agent) - [Issues](https://github.com/firecrawl/web-agent/issues) - [License: MIT](https://github.com/firecrawl/web-agent/blob/main/LICENSE) - [Project website](https://firecrawl.dev) - [README](https://github.com/firecrawl/web-agent/blob/main/README.md) --- Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/firecrawl-web-agent