Maester: A PowerShell test harness for Microsoft 365 security configuration

What problem Maester solves

Maester addresses a recurring gap for Microsoft 365 tenants: security configuration drifts silently. An admin can set conditional access policies, enable MFA, or lock down legacy authentication, but nothing reminds them when a setting changes. Maester turns security configuration into a set of automated tests that run on a schedule or in a pipeline. It is built for administrators who manage Microsoft 365 and want a continuous check rather than a manual review. The README frames it as a way to 'stay in control' of security configuration, which is a fair description of the intent. The framework is PowerShell-based and uses Pester as its test engine, so it fits environments where PowerShell is already the operational language.

How the test mechanism works

Maester does not invent a new testing paradigm. It wraps Pester, the standard PowerShell testing framework, and adds a set of pre-written tests for Microsoft 365 security settings. The README describes two core commands: Connect-Maester establishes a session with the tenant, and Invoke-Maester runs the tests in the current folder. Tests are stored as files under a directory such as ~/maester-tests. The framework also provides Install-MaesterTests to populate that folder with the default test suite, and Update-MaesterTests to refresh it when new tests are published. Results can be exported in CSV, Excel, HTML, JSON, or Markdown, and notifications can be sent to email, Teams, or Slack. That means the output is not locked to the console; it can be fed into reporting or alerting systems.

Getting started: installation and first run

Installation is a single PowerShell command: Install-Module -Name Maester -Scope CurrentUser. After that, you create a test directory and install the bundled tests: md ~/maester-tests, cd ~/maester-tests, then Install-MaesterTests. The README notes that Pester will be installed if needed, which is a convenience for fresh environments. To run, you connect and invoke: Connect-Maester, then Invoke-Maester. For national cloud tenants, Connect-Maester accepts an -Environment parameter with values Global, China, USGov, or USGovDOD. The default is Global. This is a concrete, scriptable workflow that can be scheduled or run manually. The commands are simple enough that a non-expert can follow the README, but the underlying logic assumes you understand PowerShell sessions and module loading.

CI/CD integration and the GitHub Action

Maester is published as a GitHub Action, which makes it usable in a GitHub workflow without writing custom PowerShell. The README points to a separate repository, maester365/maester-action, and includes a migration notice: the old action name maester365/maester is deprecated, and users should switch to the new one. The action integrates with GitHub features like artifact upload and workflow summaries, so test results can be attached to a run and viewed in the Actions tab. This is a practical way to run Maester on a schedule or on pull requests. The deprecation notice is a useful detail because it shows the project is actively changing how it distributes the action. If you adopt Maester, you should use the new action repository from the start.

Building from source and module structure

For contributors, Maester's source lives in the powershell/ and tests/ folders. The build script is ./build/Build-LocalMaester.ps1, which builds and validates the module, unloads any existing Maester module, and imports the local build into the current session. There is also a -BuildReport switch to rebuild and embed the report template. The README warns that the module/ folder is a build artifact: it is ignored by git and never committed. Built modules are attached to GitHub Releases and published to the PowerShell Gallery from CI. This is a clean separation between source and distribution, but it means you cannot just clone the repo and run the module directly. You must either install from the gallery or run the build script. That is a minor friction point for users who want to inspect the exact code they run.

Known limitation: ExchangeOnlineManagement v3.9.2

The README carries a prominent warning about a known issue with the ExchangeOnlineManagement module. It recommends not using version 3.9.2 because many users experience connection errors, while previous versions are generally reliable. The warning is explicit that this is an issue with the ExchangeOnlineManagement module, not Maester itself. This is a real constraint for any tenant that relies on Exchange Online connections in their tests. If you use Maester to test Exchange-related security settings, you may need to pin the ExchangeOnlineManagement module to an earlier version. This is the kind of dependency friction that can slow adoption, because it adds an extra step to environment setup. The warning is honest, which is good, but it also signals that Maester's tests are only as reliable as the modules they depend on.

Alternatives and how they differ

The most direct alternative is writing your own Pester tests without Maester. Pester is a general-purpose PowerShell testing framework, and you can write scripts that call Microsoft Graph or Microsoft 365 cmdlets directly. The difference is that Maester provides a curated set of tests and a wrapper for connection and reporting. With raw Pester, you own the entire test suite, including the connection logic and output formatting. Another alternative is Microsoft's own compliance and security center, which offers built-in reports and alerts but does not let you define custom tests in code. Maester sits between those two: it gives you a test framework with a predefined starting point, but you can still add custom Pester tests. The trade-off is that Maester adds a layer of abstraction, so you need to learn its commands and test layout, whereas raw Pester is more flexible but requires more setup.

Maintenance and license considerations

Maester is released under the MIT license, which permits commercial use, modification, and redistribution with minimal restrictions. The project is actively maintained, with recent preview releases and a regular release cadence. The README includes commands to update both the module and the test files: Update-Module Maester -Force, Import-Module Maester, and Update-MaesterTests -Path ~/maester-tests. This suggests that the test suite evolves as Microsoft changes security features, so you need to run those updates periodically to keep coverage current. The preview releases (e.g., 2.2.40-preview) indicate that new tests may arrive in preview first, which is a consideration if you prefer stable versions. The maintenance cost is not zero: you must track module updates, test updates, and any dependency issues like the ExchangeOnlineManagement warning. For a security tool, that ongoing upkeep is the price of staying current with Microsoft's ever-changing tenant configuration surface.

Editorial conclusion

Maester suits Microsoft 365 administrators who already use PowerShell and Pester and want a scheduled, repeatable check on security settings. It is less ideal for teams without PowerShell expertise or those needing a no-code compliance dashboard. Before adoption, verify that your tenant's conditional access, MFA, and identity policies are covered by the built-in tests, and check the known ExchangeOnlineManagement v3.9.2 issue if you rely on Exchange connections. Start by installing the module, running Install-MaesterTests, and reviewing the generated test files to see which checks match your environment.

Official sources

1. Official documentation
2. Official README
3. Project repository
4. Release notes