AWS ParallelCluster - AWS ParallelCluster User Guide
AWS ParallelCluster User Guide (v3)
AWS ParallelCluster
Copyright ? 2024 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
AWS ParallelCluster
AWS ParallelCluster User Guide (v3)
AWS ParallelCluster: AWS ParallelCluster User Guide (v3)
Copyright ? 2024 Amazon Web Services, Inc. and/or its affiliates. All rights reserved.
Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon.
AWS ParallelCluster
Table of Contents
AWS ParallelCluster User Guide (v3)
What is AWS ParallelCluster ........................................................................................................... 1 Pricing ............................................................................................................................................................. 1
Setting up AWS ParallelCluster ...................................................................................................... 2 Setting up an AWS account ....................................................................................................................... 2 Sign up for an AWS account ................................................................................................................ 2 Create a user with administrative access ........................................................................................... 3 Create a key pair .......................................................................................................................................... 4 Installing the AWS ParallelCluster CLI ..................................................................................................... 4 Install AWS ParallelCluster in a virtual environment (recommended) ......................................... 5 Installing AWS ParallelCluster in a non-virtual environment using pip ....................................... 7 Install AWS ParallelCluster as a standalone application ................................................................. 8 Steps to take after installation ................................................................................................................. 9 Installing the AWS ParallelCluster UI ..................................................................................................... 10 Install the AWS ParallelCluster UI ..................................................................................................... 11 Create a custom domain ..................................................................................................................... 13 Amazon Cognito user pool options .................................................................................................. 16 Identify the AWS ParallelCluster and AWS ParallelCluster UI version ....................................... 18 Update the AWS ParallelCluster UI to a new AWS ParallelCluster version ............................... 19 AWS ParallelCluster UI costs .............................................................................................................. 19 Getting started ........................................................................................................................................... 19 Configure and create a cluster with the AWS ParallelCluster CLI ............................................... 20 Configure and create a cluster with the AWS ParallelCluster UI ................................................. 30 Connect to a cluster ............................................................................................................................. 31 Multiple user access to clusters .............................................................................................................. 32 Create an Active Directory .................................................................................................................. 33 Create a cluster with an AD domain ................................................................................................ 33 Log in to a cluster integrated with an AD domain ........................................................................ 36 Running MPI jobs .................................................................................................................................. 37 Example AWS Managed Microsoft AD over LDAP(S) cluster configurations ............................. 38 Best practices .............................................................................................................................................. 42 Best practices: head node instance type selection ........................................................................ 42 Best practices: network performance ............................................................................................... 42 Best practices: budget alerts .............................................................................................................. 44 Best practices: moving a cluster to a new AWS ParallelCluster minor or patch version ......... 44
iii
AWS ParallelCluster
AWS ParallelCluster User Guide (v3)
Moving from AWS ParallelCluster 2.x to 3.x ........................................................................................ 45 Custom Bootstrap Actions .................................................................................................................. 45 AWS ParallelCluster 2.x and 3.x use different configuration file syntax .................................... 46 Inclusive language ................................................................................................................................ 52 Scheduler Support ................................................................................................................................ 52 AWS ParallelCluster CLI ....................................................................................................................... 53 IMDS configuration update ................................................................................................................. 56
Supported Regions for AWS ParallelCluster ......................................................................................... 56 Using AWS ParallelCluster ............................................................................................................ 58
AWS ParallelCluster UI .............................................................................................................................. 59 AWS Lambda VPC configuration in AWS ParallelCluster ................................................................... 60 AWS Identity and Access Management permissions in AWS ParallelCluster .................................. 62
AWS ParallelCluster Amazon EC2 instance roles ............................................................................ 63 AWS ParallelCluster example pcluster user policies ............................................................... 63 AWS ParallelCluster user example policies for managing IAM resources .................................. 78 AWS ParallelCluster configuration parameters to manage IAM permissions ............................ 84 Network configurations ............................................................................................................................ 99 AWS ParallelCluster in a single public subnet .............................................................................. 100 AWS ParallelCluster using two subnets ......................................................................................... 102 AWS ParallelCluster in a single private subnet connected using AWS Direct Connect ......... 103 AWS ParallelCluster with AWS Batch scheduler ........................................................................... 104 AWS ParallelCluster in a single subnet with no internet access ............................................... 106 Login nodes ............................................................................................................................................... 112 Custom bootstrap actions ...................................................................................................................... 115 Configuration ....................................................................................................................................... 118 Arguments ............................................................................................................................................ 121 Example cluster with custom bootstrap actions .......................................................................... 121 Example of updating a custom bootstrap script for IMDSv2 .................................................... 123 Example of updating a configuration for IMDSv1 ....................................................................... 124 Working with Amazon S3 ...................................................................................................................... 125 Examples ............................................................................................................................................... 125 Working with Spot Instances ................................................................................................................ 126 Scenario 1: Spot Instance with no running jobs is interrupted ................................................. 126 Scenario 2: Spot Instance running single node jobs is interrupted .......................................... 127 Scenario 3: Spot Instance running multi-node jobs is interrupted ........................................... 127 Schedulers supported by AWS ParallelCluster .................................................................................. 127
iv
AWS ParallelCluster
AWS ParallelCluster User Guide (v3)
Slurm Workload Manager ................................................................................................................. 127 AWS Batch ............................................................................................................................................ 190 Shared storage ......................................................................................................................................... 198 Configure shared storage ................................................................................................................. 201 Working with shared storage ........................................................................................................... 204 Quotas ................................................................................................................................................... 207 Tagging ....................................................................................................................................................... 208 Monitoring AWS ParallelCluster and logs ........................................................................................... 211 Integration with Amazon CloudWatch Logs ................................................................................. 213 Amazon CloudWatch dashboard ..................................................................................................... 216 Amazon CloudWatch alarms for cluster metrics .......................................................................... 218 AWS ParallelCluster configured log rotation ................................................................................ 221 pcluster CLI logs ............................................................................................................................. 222 Amazon EC2 console output logs ................................................................................................... 223 Retrieve AWS ParallelCluster UI and AWS ParallelCluster runtime logs .................................. 224 Retrieving and preserving logs ........................................................................................................ 226 AWS CloudFormation custom resource ............................................................................................... 229 Provider stack hosted by AWS ParallelCluster ............................................................................. 230 Cluster resource .................................................................................................................................. 231 Cluster operations .............................................................................................................................. 234 Troubleshooting stacks that include the AWS ParallelCluster custom resource .................... 235 Elastic Fabric Adapter ............................................................................................................................. 235 Enable Intel MPI ....................................................................................................................................... 236 AWS ParallelCluster API ......................................................................................................................... 238 AWS ParallelCluster API Documentation ....................................................................................... 238 Deploy with AWS CLI ......................................................................................................................... 239 Updating the API ................................................................................................................................ 241 Invoking AWS ParallelCluster API ................................................................................................... 242 Accessing the API logs and metrics ................................................................................................ 244 AWS ParallelCluster for Terraform ....................................................................................................... 245 Connect to the head node through NICE DCV .................................................................................. 246 NICE DCV HTTPS certificate ............................................................................................................. 246 Licensing NICE DCV ............................................................................................................................ 246 Using pcluster update-cluster .................................................................................................. 247 Update Policy: definitions ................................................................................................................. 247 pcluster update-cluster examples ...................................................................................... 250
v
................
................
In order to avoid copyright disputes, this page is only a partial summary.
To fulfill the demand for quickly locating and searching documents.
It is intelligent file search solution for home and business.