AWS ParallelCluster turns a cloud account into an HPC cluster

A cluster, a filesystem and a scheduler, in one command

AWS ParallelCluster is described as an AWS supported open source cluster management tool that makes it easy to deploy and manage High Performance Computing clusters in the AWS cloud, and it is built on the open source CfnCluster project.

What it sets up is specific. It automatically creates the required compute resources and a shared filesystem, and it offers a choice of batch schedulers, AWS Batch and Slurm. The filesystem is the part that separates this from a script that launches instances: an HPC job needs somewhere shared to read input from and write output to, and that is what the tool provisions alongside the compute fleet.

The stated use case is deliberately broad. It facilitates both quick start proof of concepts and production deployments, and the README gives a concrete example of what that means at the higher end, building a genomics portal that automates the entire DNA sequencing workflow on top of ParallelCluster. A portal on top of a cluster is a different level of ambition from launching a handful of nodes, and the tool is meant to be the layer underneath rather than the application itself.

The project is written in Python, and the repository layout shows the split between the parts. There is a cli directory, an api directory, a cloudformation directory, pc_support for the packaging and support code, util, tests, and the third-party licence files LICENSE.txt, NOTICE.txt and THIRD-PARTY-LICENSES.txt at the top level.

The supported status matters too, and the README states it plainly: this is an AWS supported tool, not a community project that happens to work with AWS.

The setup path, and the key pair you cannot skip

The quick start is a sequence of shell steps, and the first prerequisite is stated as important: you will need an Amazon EC2 Key Pair to be able to complete the following steps, with a link to the official AWS guide.

The Python environment comes first, and the README notes that ParallelCluster 3.0.0 and later requires Python 3.7 or newer:

python3 -m pip install --upgrade pip
python3 -m pip install --user --upgrade virtualenv
python3 -m virtualenv ~/hpc-ve
source ~/hpc-ve/bin/activate

Then the command line tooling, the AWS CLI and Node.js, and the reason for Node is worth noting because it is not obvious from a Python project:

$ pip3 install awscli

Node.js is required by the AWS CDK library used by ParallelCluster. So the toolchain is Python, the AWS CLI, and Node, and the Node requirement comes from the infrastructure-as-code layer rather than from the CLI itself.

ParallelCluster itself installs with:

$ pip3 install aws-parallelcluster

and then the AWS credentials and default region are set with aws configure, which prompts for the access key ID, the secret access key, the default region name and the default output format. The example output in the README shows us-east-1 as the default region.

The key pair requirement is the one detail that catches people out, because the cluster creation needs to log in to the head node. The README also notes that you can import a public key instead, and that during configuration you must select a region that contains the key pair you intend to use.

The configure wizard asks about networking, and it can build the VPC

The configuration step is an interactive wizard, pcluster configure, which walks through each parameter and prints the allowed values, and the example in the README shows the region list with sixteen entries in the version being documented.

$ pcluster configure --config /dir/cluster-config.yaml

The wizard asks two questions that matter most for a first cluster, and both offer to create things for you. The first is whether to automate VPC creation, and entering n means you already have a VPC suitable for the cluster, while leaving it as y lets the tool create one on the fly. The second is the subnet, and the README describes the same choice being offered again: you can select a valid subnet ID for both the head node and the compute nodes, or let the tool set everything up.

The subnet configuration prompt offers two layouts, a head node in a public subnet with the compute fleet in a private subnet, or the head node and the compute fleet in the same public subnet. The first is the arrangement most teams want and the second is the one that will let compute nodes reach the internet, which is convenient for pulling packages and wrong for a cluster holding real data.

At the end, the wizard writes the file and prints the command to create the cluster, so the configuration file is a plain YAML document you can commit and edit. That matters for a team, because the next person does not have to answer the wizard again.

Creating the cluster and getting inside it

Creation is a single command with a cluster name and the configuration file, and the README then shows how to log in once it has finished:

$ pcluster create-cluster --cluster-name myfirstcluster --cluster-configuration /dir/cluster-config.yaml

$ pcluster ssh --cluster-name myfirstcluster

Inside the cluster, the scheduler's own command shows the compute hosts:

$ sinfo

sinfo is Slurm's job and node information command, so this only makes sense once you are logged into a cluster running Slurm, which is the default path through the wizard.

Two points about the lifecycle are worth flagging even though the README does not spell them out as caveats. First, the cluster name is the handle for every subsequent command, so it needs to be a name you can live with, because it is what pcluster ssh and the other operations will refer to. Second, the head node and compute fleet are created by the tool with the names and options you chose in the wizard, and the compute fleet scales according to the configuration, so a first cluster that comes up with several nodes in a public subnet is a configuration choice that is visible in the bill.

The README also notes that you can edit the configuration file after the wizard writes it, and that the getting started guide in the user documentation covers the steps in more detail.

Documentation in ten languages, and where support actually lives

The documentation arrangement is one of the strongest parts of this project, and the README is explicit about why. It says the team has been working hard to greatly improve the documentation, that it is now published in 10 languages, and that this is one of the many benefits of being hosted on AWS Docs. The Getting Started Guide is the entry point it points at for new users.

That last point is a design decision with consequences. Because the documentation lives on AWS Docs rather than in the repository, it inherits the AWS review and publication process, which is why it can be translated into ten languages at once and why it stays in step with the service side of the platform. The cost is that a page lives outside the project, so feedback on it goes through a feedback link at the bottom of each page, and changes to the docs themselves are made in a separate repository, the awsdocs/aws-parallelcluster-user-guide.

Support has two more channels. The README asks for bugs and feedback to be opened as a GitHub issue in the project, and it points to an active AWS HPC forum on the AWS re:Post site.

There is also a managed alternative mentioned in a callout, and it is worth reading as a fork in the road. AWS Parallel Computing Service, or PCS, is a managed service for running and scaling HPC workloads on AWS using Slurm, and the README raises it for anyone looking for managed Slurm. The difference in approach is the whole point: ParallelCluster is a tool you install and operate yourself, and it gives you the CloudFormation templates and the CLI, while PCS is a service AWS runs for you, and the operational burden moves to the provider.

For maintenance, the repository is not archived and the last push was on 2026-10-19, with recent releases v3.16.1 on 2026-09-08, v3.16.0 on 2026-08-20 and v3.15.1 on 2026-05-28. The default branch is develop. The licence is Apache-2.0, with the Apache 2.0 badge and a LICENSE.txt at the root, alongside NOTICE.txt and THIRD-PARTY-LICENSES.txt.

What a cluster costs, and where this is the wrong tool

Two limits deserve plain statements.

The first is that the tool creates real, billable infrastructure, and the wizard makes several of those choices for you. The compute fleet, the shared filesystem, the head node and the VPC all end up in your account, and the default subnet option in the wizard puts the head node and the compute nodes in the same public subnet. A cluster running in a public subnet with several nodes is a configuration a reviewer at any organisation with a security function will ask about, and the more private option is one keystroke away in the wizard but only if you know to reach for it.

The second is the coupling to AWS. The tool is built on AWS resources and on the AWS CDK library, the head node is reached with ssh through the AWS CLI, and the whole model is a cluster in an AWS account. The name change in version 2.0.0, when CfnCluster became AWS ParallelCluster, and the release of several new features alongside it, is the point at which the project stopped being a general cluster tool and became an AWS one.

That is not a flaw, it is the deal, and for a team already committed to AWS it is the right deal, since the alternative is assembling CloudFormation templates, a shared filesystem, an AMI with a scheduler installed, and a fleet scaling policy by hand. What it does mean is that the operational surface is real: the tool creates resources, and a team that wants a cluster it can hand to researchers has to decide who owns the patching, the key management and the cost monitoring.

The genomics portal example is the reference for how a team builds on top of a cluster, and the repository's own tests and cloudformation directories are where a team looking for a pattern to copy will look.

Editorial conclusion

AWS ParallelCluster suits a team that needs a working HPC cluster in AWS this week, either to run a proof of concept or to run production work with a shared filesystem and a batch scheduler in place. It is the wrong tool if you want a cluster that is portable across clouds, since the configuration model is built on AWS resources and on the AWS CDK library for the infrastructure it creates, and it is the wrong tool if you want a managed service rather than a tool you operate yourself, since the README points managed Slurm users to AWS Parallel Computing Service. Before you adopt it, work through the Getting Started Guide in the ten-language documentation, and note that an EC2 key pair is a hard prerequisite, since the configuration wizard will not complete without one in the selected region.

Frequently asked questions

Official sources

1. aws/aws-parallelcluster on GitHub
2. License: Apache-2.0
3. Project website
4. README
5. Releases