# Nuxt Robots: controlling crawlers, indexers and AI scrapers from one module

> The @nuxtjs/robots module generates a robots.txt, adds the matching X-Robots-Tag header and meta robots tag, detects bots including AI crawlers, and lets you decide whether a given path can be indexed, all with sensible defaults and a handful of configuration lines.

**nuxt-modules/robots** — Tame the robots crawling and indexing your Nuxt site.

- Repository: https://github.com/nuxt-modules/robots
- Website: https://nuxtseo.com/robots
- Stars: 528 · Forks: 54
- Language: TypeScript
- License: MIT
- Published: 2026-10-22 · Updated: 2026-10-22 · Language: en
- Canonical page: https://hysenlabs.com/projects/nuxt-modules-robots

## What the module actually controls

Nuxt Robots is a module for configuring the robots crawling your site with minimal config and best practice defaults, and the readme is clear that there are two core features rather than one. The first is telling crawlers which paths they can and cannot access using a robots.txt file. The second is telling search engine crawlers what they can show in search results from your site, using a meta name robots content index tag and an X-Robots-Tag HTTP header. That second one is the part that trips people up, because robots.txt and the robots meta tag do different jobs. One says which URLs may be fetched, the other says what may be shown for a URL that was fetched, and a page can be crawled and still be excluded from the index. Handling both from one configuration is the reason the module exists rather than a static file in your public directory. The documentation sits at nuxtseo.com/robots, the licence is MIT, and the project publishes v7.0.2, with v7.0.1 and v7.0.0 the day before.

## The feature list, taken one at a time

The five features are each a distinct thing you would otherwise implement yourself. You can merge in your existing robots.txt or programmatically create a new one, which matters for a site that already has hand-written rules and does not want to lose them. There is automatic X-Robots-Tag header and meta robots meta tag, so the per-page directives are emitted for you rather than left to a manual meta tag in every template. There is bot detection, with optional fingerprinting, which is the piece that turns the module from a static generator into something that can react to who is asking. Non-production environments are disabled from being indexed, which is a small default that prevents a staging deployment from appearing in search results. And AI crawlers are controlled using the Content-Signal and Content-Usage directives, which is the part of the module aimed at the current generation of scrapers rather than at search engines.

## Installing it

Installation is the standard Nuxt module command.

```bash
npx nuxt module add robots
```

There is one version note that matters if you are on an older stack. Using Nuxt 2, you are asked to use the v3.x tag instead, because the current line targets modern Nuxt. The package itself is published as @nuxtjs/robots and the package manifest shows it is an ES module with three export paths, the module itself, a util entry point and a content entry point, with types resolved from the dist/types.d.mts file. The engines field is specific, requiring Node ^22.22.3 or ^24.15.0 or 26.0.0 or newer, and the package manager declared is pnpm at 12.8.1.

```json
  "engines": {
    "node": "^22.22.3 || ^24.15.0 || >=26.0.0"
  },
```

If you are working with an AI agent rather than typing it yourself, the project points at a skill for @nuxtjs/robots hosted on skilld.dev, which is the same skill repository pattern the module badge advertises.

## The three build scripts, and what they reveal

The scripts in the manifest describe the shape of the project. The build script runs nuxt-module-build in three stages, build with stub, then prepare, then build, and finally copies the devtools directory into dist with a node one-liner using cpSync with the recursive flag. That last step is the interesting one, because it means the module ships a devtools panel, and the copy is done separately from the module build because the devtools code is plain files rather than something nuxt-module-build compiles.

```json
    "build": "nuxt-module-build build --stub && nuxt-module-build prepare && nuxt-module-build build && npm run client:build",
```

The development script runs nuxt dev against the playground directory, and there is a separate dev:build that runs nuxt build on the same playground. The prepare:fixtures script is the one to read if you want to know how the module is tested, because it prepares five separate fixtures in one command.

```json
    "prepare:fixtures": "nuxt prepare playground && nuxt prepare test/fixtures/basic && nuxt prepare test/fixtures/i18n && nuxt prepare test/fixtures/content-v3 && nuxt prepare test/fixtures/static-headers",
```

The fixture names tell you the feature surface. There is a basic fixture, an i18n fixture because a multilingual site has different rules per locale, a content-v3 fixture because the content entry point has its own version, and a static-headers fixture because the header behaviour is a separate code path from the file behaviour.

## The examples, and what they show

The repository carries three example directories, examples/basic, examples/custom-rules and examples/i18n, and each maps to one of the things the module does. The basic one is the default behaviour, and the custom-rules one is where you see how a path is told apart from another, which is the case you hit as soon as one part of your site is public and another is not. The i18n one is the interesting case, because a site with several locales has a question that has no single answer, whether a crawler should be allowed to reach every locale or only some of them, and the module has an example dedicated to answering it. The readme also links a default demo on StackBlitz, opened at the public/_robots.txt file, which is a quick way to see what the generated file looks like without setting anything up.

## What the module puts in the repository

The top level of the repository shows what a Nuxt module project needs and this one has all of it. There is src/ for the module code, test/ with the fixtures, docs/ and examples/ and playground/ for the demo, devtools/ for the panel that gets copied into the distribution, and scripts/ for the release tooling. The build configuration is in build.config.ts, the TypeScript configuration in tsconfig.json, and the test runner is vitest with a vitest.config.ts. There is a pnpm-workspace.yaml and a pnpm-lock.yaml, and an .nuxtrc file alongside a .npmrc, which are the two places where the package manager and the Nuxt CLI read their settings. There is also a skills/ directory, which is what the skilld.dev badge refers to, and an .attw.json, the Are The Types Wrong configuration, which is the check that the published types resolve correctly for consumers. The security policy lives in SECURITY.md and the changelog in CHANGELOG.md, and eslint.config.mjs is the lint setup.

## Checking the result, and the limits of what robots.txt can do

The readme suggests two things to do after configuring. If you already have a robots.txt configured, you can verify what crawlers and AI bots can access with the free robots.txt validator the project links on its site, and you can monitor crawl and indexing status with the paid tier of the same product. That is the sensible workflow, because the module merges your existing file with the generated rules, and a merge is exactly the kind of thing that produces a result nobody reads carefully. There is also the guide for people new to robots or SEO, the Controlling Web Crawlers page, which the readme points at before anything else for anyone who is not sure whether they need these features at all. And the practical warning is one of ordering rather than content. If you configure the module to disable indexing on non-production environments, that protection only helps if the environment is identified correctly, so the environment configuration is worth getting right before you deploy rather than after a staging URL turns up in a search result.

## Conclusion

Nuxt Robots fits a Nuxt team that wants crawler control without hand-maintaining a robots.txt, and fits anyone whose pages differ in what search engines and AI crawlers should be allowed to see, since the module expresses that as per-path rules and emits the header, the meta tag and the text file together. It does not fit a Nuxt 2 project, because the module points Nuxt 2 users at the v3.x tag instead. It also does not fit a team that wants to control AI crawlers with a single switch and nothing else, since the feature list is explicit that AI crawlers are handled through the Content-Signal and Content-Usage directives, which is a deliberate choice rather than a blanket block. Before you deploy, set the environment block so non-production sites are not indexed, and verify the result with the validator the project links, because a robots file that looks right in the source can still be wrong once merged with an existing one.

## FAQ

### How do I install the Nuxt Robots module?

Run npx nuxt module add robots, which installs the @nuxtjs/robots dependency into your project. If you are on Nuxt 2, use the v3.x tag of the module instead. The package requires Node ^22.22.3 or ^24.15.0 or 26.0.0 or newer.

### What is the difference between robots.txt and the X-Robots-Tag header in this module?

The robots.txt file tells crawlers which paths they may and may not access, while the X-Robots-Tag header and the meta robots tag tell search engine crawlers what they may show in search results for a page. The module handles both, and the meta tag is emitted automatically rather than added by hand in every template.

### Does Nuxt Robots handle AI crawlers?

Yes. One of the listed features is controlling AI crawlers using the Content-Signal and Content-Usage directives. The module also offers bot detection with optional fingerprinting, and non-production environments are disabled from being indexed by default.

### Can I keep the robots.txt I already have?

Yes. Merging in your existing robots.txt is the first listed feature, and you can also create a new one programmatically. After configuring, you can check what crawlers and AI bots can access with the robots.txt validator linked from the project documentation.

## Sources

- [License: MIT](https://github.com/nuxt-modules/robots/blob/main/LICENSE)
- [nuxt-modules/robots on GitHub](https://github.com/nuxt-modules/robots)
- [Project website](https://nuxtseo.com/robots)
- [README](https://github.com/nuxt-modules/robots/blob/main/README.md)
- [Releases](https://github.com/nuxt-modules/robots/releases)

---

Hysen Labs editorial analysis, written from the project's own repository and release notes. Cite the canonical page: https://hysenlabs.com/projects/nuxt-modules-robots