Artemis: a modular vulnerability scanner built for reporting

What Artemis is built to do

Artemis is a modular vulnerability scanner. The README describes it as the tool that powers CERT PL scanning activities, doing two things: checking various aspects of website security, and building easy-to-read messages ready to be sent to the scanned organisations.

That second half is the distinguishing part. A scanner that produces a list of findings has to be interpreted by a human before it becomes useful. Artemis goes further and produces the message itself, including the recommendation. The README shows what that looks like:

> The following addresses contain version control system data: > > - https://subdomain1.example.com:443/.git/ > > Making a code repository public may allow an attacker to learn the inner workings of a system, and if it contains passwords or API keys - also gain unauthorized access. Such data shouldn't be publicly available.

And for outdated software, it names the version found and tells the recipient what to do about it.

The README says this export capability is what enabled CERT Polska to notify entities in its constituency about hundreds of thousands of vulnerabilities. That is the use case the project was designed around: scanning at national scale, where the operator of the scanner is not the owner of the system being scanned, and the output has to be addressed to a stranger.

The licence is BSD-3-Clause, and the README includes a specific note for contributors: by contributing to Artemis, you agree that the BSD 3-Clause Licence applies to your input automatically, without additional declarations.

Modules split by licence, and why extra modules live elsewhere

The scanner is modular, and each module implements one check. The README points to the documentation for the up-to-date list of features, and to a dedicated page on writing your own module.

The packaging decision is worth understanding. The README says that if you want to use additional modules that were not included because of non-BSD-compatible licences, you should browse the Artemis-modules-extra repository. The reason is the licence policy, and the repository enforces it in code: `pyproject.toml` contains a `liccheck` configuration with an `authorized_licenses` list covering 0bsd, bsd, Apache 2.0, MIT, ISC, LGPL, MPL, the unlicense and several SPDX-style combinations, and a separate `authorized_packages` section for specific packages whose metadata names the licence badly, such as `dkimpy` described as too generic, `karton-core` and `libmailgoose` recorded as BSD without proper metadata, and `anytree` as Apache 2.

So a dependency whose licence is not on the list will fail a check rather than slip into a build. That is a real constraint for anyone extending Artemis: a module that pulls in a permissively licensed but non-listed library will not pass the licence gate.

The `liccheck` package itself appears in the frozen requirements at version 0.9.2, which confirms the check runs as part of the toolchain rather than as an occasional manual review.

Running Artemis means running a web app, a queue and a database

The docker-compose file in the repository root is where the architecture becomes clear. It defines services including `redis`, `s3mock`, `postgres` and the Artemis service itself.

Redis is pinned to `redis:7.2.4` and runs with `--io-threads 4 --io-threads-do-reads yes`. The comment on that pin is unusually candid: do not upgrade to 7.3 or later because of its non-free licence, until the licensing situation becomes clearer or an open-source fork appears. It also sets a `stop_grace_period` of 2m, because Redis saves the dataset on shutdown, which for a large task queue can take longer than the default 10 seconds after which Docker kills the container.

`postgres:16` runs with `max_connections=2000`, and `s3mock` is the Adobe S3 mock, initialised with a bucket named `bucket` and capped at 250M of memory, with a healthcheck that wgets port 9090.

The Artemis service itself uses a YAML anchor so the image can be overridden in one place:

x-artemis-build-or-image:
  &artemis-build-or-image
  image: "${ARTEMIS_BUILD_IMAGE:-certpl/artemis:latest}"

That single indirection is what lets the development mode build the image locally instead of pulling it from Docker Hub.

One piece of configuration in the compose file is a fix for a subtle bug and is worth reading as an example of the operational care in this project. The `dns_opt` sets `ndots:0` on the Artemis service, with a comment explaining that without it, a host whose `/etc/resolv.conf` contains a search domain causes every lookup of a short hostname such as `s3mock` to first try the qualified variant against the external nameserver, generating hundreds of thousands of spurious DNS queries per day, tracked as issue 2310.

So Artemis is not a single binary you run against a URL. It is a service, with a web interface, a job queue and a database behind it, and that is the main thing to understand before you try to evaluate it.

Development mode builds the image and live-reloads the web interface

The development entry point is a single script:

./scripts/start --mode=development

It copies the example environment file `env.example` to `.env` if that file does not exist, so the first run needs no manual configuration step, and you then configure the settings in `.env` according to your needs. The README highlights `CUSTOM_USER_AGENT` as the variable for customising the user agent, alongside other parameters, with the full list in the configuration section of the documentation.

The user agent point deserves emphasis. A scanner that identifies itself honestly is a scanner you can responsibly run against systems you do not own, because the operator of the target can identify and contact you. A scanner that hides is a different ethical proposition, and the fact that the documentation calls out the variable as a first-class configuration point suggests the project wants the honest setting to be the easy one.

In development mode the image is built locally from the code you are developing rather than downloaded from Docker Hub, and for the `web` component you can see the results of code modifications on the page without reloading the entire container.

For a quick start without touching the code, the README links a Quick Start page in the documentation. That is the path to use if your goal is to see a report rather than to extend the scanner.

Tests, PID 1 and why unittest-parallel is mandatory

The test entry point is:

./scripts/test

The README then attaches a warning that is unusual enough to be worth repeating, because it is the kind of detail that silently doubles your test suite's runtime. Avoid using `python -m unittest` directly as the `command:` in `docker-compose.test.yaml`. When Python runs as PID 1, subprocesses, especially Go binaries like Nuclei, suffer from slow or degraded I/O, because PID 1 handles signals and stdout and stderr differently. The instruction is to use unittest-parallel instead, since its worker processes avoid the PID 1 issue.

That tells you two things at once. First, the test suite runs the scanner's modules as real subprocesses, including Nuclei, so the tests are integration tests of the tool rather than unit tests of pure functions. Second, the project has hit a containerisation problem serious enough to document a workaround for it, and it is the kind of problem that appears only after you have built a scanner that shells out to other scanners.

The repository carries several compose files for different purposes: `docker-compose.test.yaml`, `docker-compose.test-e2e.yaml` and the main `docker-compose.yaml`, so there is an end-to-end tier on top of the unit and integration tiers.

Code formatting is handled with `pre-commit`, which runs the linters and formatters and is also executed on CI to verify the code is formatted properly. Locally you run it over everything with `pre-commit run --all-files`, and you can install it as a commit hook with `pre-commit install`. The `pyproject.toml` sets black with a line length of 120 and isort with the black profile, so the formatting is standardised and the pre-commit hook is the enforcement.

Pinned dependencies, pip-tools and the mypy configuration

Dependencies are managed with `pip-tools`. The procedure is explicit: modify `requirements.in`, or `docs/requirements.in` for documentation dependencies, run `./scripts/compile-requirements` to generate the frozen `requirements.txt` files, and commit both the `.in` and the `.txt` files.

The `pip-tools` section of `pyproject.toml` explains why the configuration looks the way it does. `no-header` prevents pip-compile from emitting a Python-version-specific header, which would differ between developer machines, CI on Python 3.11 and the Dependabot runner, causing the pre-commit hook to perpetually flag `requirements.txt` as modified. `no-annotate`, `strip-extras` and `quiet` round out the settings.

That is a small detail with a large payoff. A requirements file that churns on every commit trains contributors to ignore the pre-commit hook, and once the hook is ignored the formatting and linting guarantees are gone.

The frozen requirements are extensive, with aiohttp, fastapi, flask, jinja2, mako, pydantic, dnspython, paramiko, minio, boto3, aioboto3, cryptography, argon2-cffi, pycryptodome, liccheck, prometheus-client, opentelemetry-api and psutil among the pins. The presence of prometheus-client and opentelemetry-api says the scanner emits metrics and traces, and the presence of both fastapi and flask, with templates in a `templates/` directory and migrations under `migrations/` with an `alembic.ini`, says the web side has a database schema that evolves.

Type checking is configured separately through a `.mypy.ini` at the root, and the `artemis/` package directory holds the implementation.

Where Artemis is the wrong tool, and the legal line to respect

The clearest wrong case is a single-target check. If you have one server and want to know whether it exposes a `.git` directory, Artemis is a large operational commitment: a web interface, a task queue, a database, an S3 mock for object storage, and a Docker compose file. A point scanner does that job in a fraction of the setup.

The second wrong case is a need for stability. The README says it plainly: Artemis is experimental software, under active development, use at your own risk. The release history supports the impression of a fast-moving project, with v4.0.0 published on 2026-10-02, a jump from v3.6.0 in February 2025. A major version moving that quickly implies breaking changes to your configuration and your module code.

The third wrong case is legal rather than technical, and it is the one to be most careful about. Artemis is a vulnerability scanner designed to be pointed at systems its operator does not own. The CERT use case, scanning a national constituency, comes with legal authority behind it. Running the same tool against systems you have no authorisation to test is a different act, and the tool's own documentation does not grant permission. A team deploying this internally should establish the authorisation for each target before scanning.

What the project gives you on the reporting side is genuinely unusual. The combination of a modular check runner with automatic prose output aimed at the site owner, the licence-gated module packaging and the national-scale deployment record is a design you cannot easily replicate with a general purpose scanning tool that emits JSON findings.

Editorial conclusion

Artemis suits national CERT teams, CSIRTs and security researchers who need to scan a constituency and then produce findings an ordinary administrator can act on, and who are prepared to write or maintain modules. It does not suit a team that wants a one-command web vulnerability check, since running it means operating a web interface, a task queue and a database, and it does not suit anyone who needs a stability guarantee, because the project describes itself as experimental software under active development. Before deploying it, read the module documentation to see which checks are bundled and which live in Artemis-modules-extra, confirm that your own scanning of third-party systems has the authorisation the law of your jurisdiction requires, and check the `CUSTOM_USER_AGENT` setting in your `.env` so that your traffic is identifiable to the operators of the sites you scan.

Frequently asked questions

Official sources

1. CERT-Polska/Artemis on GitHub
2. License: BSD-3-Clause
3. Project website
4. README
5. Releases