ScubaGear: assessing a Microsoft 365 tenant against CISA's Secure Configuration Baselines

The compliance gap ScubaGear was built to close

CISA publishes Secure Configuration Baseline documents for a set of Microsoft 365 products, and those documents describe policy. Turning policy into a defensible statement about a specific tenant is the hard part: an administrator has to know which setting maps to which control, query it, and record the answer somewhere a reviewer can read. ScubaGear exists to do that mapping mechanically. It is a PowerShell module that verifies whether a Microsoft 365 tenant's configuration conforms to the policies in the SCuBA Secure Configuration Baseline documents, and it is explicitly aimed at M365 administrators rather than at developers or general security teams.

The scope is broad in product terms and narrow in purpose. The baselines cover Microsoft Entra ID, the Security Suite, Exchange Online, Power BI, Power Platform, SharePoint and Teams. A separate document lists removed policies, which matters if you have been tracking a control that was dropped from a baseline. The controls are mapped to NIST SP 800-53 and to MITRE ATT&CK, so the output can be tied back to frameworks your organisation may already report against. What ScubaGear does not do is change anything. Every run is read-only against the tenant, which is the right default for a tool that a compliance function will want to run repeatedly.

The three-step mechanism: Graph queries, Rego policies, three report formats

The README describes a three-step process, and the separation is the most interesting design decision in the project. Step one is PowerShell code that queries M365 APIs for configuration settings. Step two hands those settings to Open Policy Agent, which compares them against Rego security policies written per baseline document. Step three writes the comparison as HTML, JSON and CSV.

Putting the policy logic in Rego rather than in PowerShell means the rules are data evaluated by a general policy engine. That is why the repository carries a .regal directory, the linter configuration for Rego, alongside the PowerShell tree. It also means the assessment logic and the collection logic can be reviewed separately, and that a baseline change is a policy change rather than a rewrite of the collection code. The cost is a second language in the toolchain: anyone who wants to read a rule or adjust it has to be comfortable with Rego and with OPA's evaluation model.

The output split is practical rather than decorative. The HTML report (a sample BaselineReports.html ships in the repository) is the artefact you hand to a reviewer. ScubaResults.json is the structured form for parsing into another system. ScubaResults.csv is for spreadsheet analysis. Because the same run produces all three, you do not have to choose between a human-readable report and machine-readable data.

Installing ScubaGear from PSGallery and running a first assessment

The README states that ScubaGear is installed from PSGallery, and the install instructions assume a PowerShell 5 terminal on a Windows computer. The module name is ScubaGear.

# Install ScubaGear
Install-Module -Name ScubaGear

After the module is in place, the dependencies are installed with a second cmdlet rather than by hand. The README calls this the minimum required set of dependencies, which is worth noting because Open Policy Agent is part of the pipeline.

# Install the minimum required dependencies
Install-ScubaDependencies

The version check is the fastest way to confirm the module resolved correctly before you touch a tenant.

# Check the version
Invoke-SCuBA -Version

The first real assessment is a single command with a wildcard product selector. This queries every supported product, so expect it to take a while and to require an account with the necessary permissions.

# Assess all products (basic command)
Invoke-SCuBA -ProductNames *

One constraint applies before you run that: the README warns that if you are running v2.0.0 with interactive login against a non-commercial tenant such as gcc or gcchigh, you must include the -M365Environment parameter. It also states that a future release will auto-detect the environment and the parameter will not be necessary. Until then, a government-cloud tenant needs the parameter set explicitly.

The two-run workflow and the YAML configuration file

ScubaGear's documented workflow is iterative, and the first run is deliberately not a verdict. Run it without a configuration file and it generates a baseline template of your environment's current settings. That run makes no changes. It tells you what your tenant looks like today, which is useful but is not the same as telling you whether it is compliant, because there is nothing yet to compare against.

After you review and edit the generated configuration file, you run ScubaGear again with that file as input. Only then does it compare intended settings against the actual environment and raise discrepancies. The README is direct about what to do with the gaps: either update the tenant configuration or document risk acceptances in the YAML file using exclusions, annotations or omissions. Those three mechanisms are the tool's answer to the reality that not every baseline control will be implemented, and that a documented acceptance is a better outcome than a permanently red report.

The YAML file also carries the customisation: which products, baselines and rules apply to your environment. For administrators who would rather not hand-edit YAML, the project ships a configuration UI launched with Start-ScubaConfigApp. It presents a step-by-step setup wizard covering the configuration options, validates input with a live YAML preview, integrates with Microsoft Graph for user and group selection, and imports and exports existing configuration files. The README positions it for users who prefer a visual interface over command-line tools. That is a reasonable split: the wizard for the first setup, the file itself for version control.

Where ScubaGear is the wrong tool

ScubaGear is a Windows PowerShell 5 module installed from PSGallery. If your administrators work on macOS or Linux, or if your organisation has standardised on PowerShell 7 cross-platform tooling, the documented install path does not match your environment. The README gives no alternative installation route.

The second limitation is structural. This is a point-in-time assessment that you run, not a service that watches the tenant. Nothing in the described architecture schedules itself, alerts on drift, or reacts to a configuration change between runs. If your requirement is continuous posture monitoring with alerting, ScubaGear produces the evidence but not the cadence.

The third is the permission surface. The prerequisite section is referenced repeatedly in the README rather than spelled out in the overview, and the assessment queries M365 APIs across seven product areas. An account that can read all of that is a privileged account. Running the assessment is therefore itself something to plan, not something to do casually from a personal admin login.

Finally, ScubaGear reports. It does not remediate. Every gap it finds becomes work for a human, and the README's own guidance points you toward either changing tenant configuration or writing an acceptance into YAML. Teams expecting a one-click fix will be disappointed by design.

ScubaGear against Monkey365 and ScubaGoggles

The most common comparison is with Monkey365, which appears alongside ScubaGear in the searches people run. The difference is the reference point. ScubaGear evaluates a tenant against CISA's SCuBA Secure Configuration Baseline documents, and its Rego policies are written per baseline. The output is framed as conformance to a published government baseline, with mappings to NIST SP 800-53 and MITRE ATT&CK. A general-purpose M365 security assessment tool answers a different question: what is exposed, rather than does this tenant match this specific baseline. If your reporting obligation names the SCuBA baselines, that framing is the whole point; if you want broad posture discovery, it is a constraint.

ScubaGoggles is the sibling project for Google Workspace, and the naming is not accidental. Same assessment model, different cloud. That matters for organisations running both M365 and Google Workspace, because it means two tools with two configuration formats rather than one. It also means the ScubaGear documentation will not help you with a Workspace tenant, and vice versa.

Maintenance, upgrade path and the CC0-1.0 licence

The repository is not archived, and the last push was on 2026-09-24, which is recent. The release cadence visible in the repository shows v1.7.0 and v1.7.1 in February 2026 and v1.8.0 in May 2026. The README's own note on upgrading is explicit: after installing ScubaGear, use the built-in update functions and features when updating to the latest version, and it points to an update guide under docs/installation/update.md. That is a deliberate position against reinstalling from PSGallery each time, and it is worth following because the dependencies installed by Install-ScubaDependencies are part of the runtime.

The README also references v2.0.0 in the context of the -M365Environment parameter, which means the documentation is tracking a version ahead of the most recent release listed. If you pin to a released version, check whether the parameter guidance applies to the version you actually installed.

The licence is CC0-1.0. That is a public domain dedication rather than a permissive software licence, and it is unusual for a code project: CC0 was written for content, not for software, and it says nothing about patents or about the warranty and liability disclaimers that software licences typically carry. CC0-1.0 does remove attribution requirements, so you can embed or modify the module without a notice obligation. Whether that is the right fit for your organisation's open source policy is a question for your legal team, not something this article can settle.

Editorial conclusion

Adopt ScubaGear if you administer a Microsoft 365 tenant and need a repeatable, evidence-producing check against CISA's Secure Configuration Baselines, and if you can run PowerShell on Windows with the required Graph permissions. Do not adopt it as a continuous monitoring service or as a substitute for a remediation tool: it reports, it does not fix. Before the first run, confirm the module version you install, the M365 environment parameter for non-commercial tenants, and which Graph permissions the assessment account already holds.

Frequently asked questions

Official sources

1. cisagov/ScubaGear on GitHub
2. License: CC0-1.0
3. Project website
4. README
5. Releases