创建适用于AWS Parallel Cluster的自定义YAML文件。

0

【以下的问题经过翻译处理】 我正在尝试按照此处的AWS Parallel Cluster上运行FDS/SMV的教程:https://aws.amazon.com/blogs/compute/fire-dynamics-simulation-cfd-workflow-using-aws-parallelcluster-elastic-fabric-adapter-amazon-fsx-for-lustre-and-nice-dcv/,我到了需要设置一个配置文件的步骤,其包含以下数据:

[aws]
aws_region_name = <AWS-REGION>

[global]
sanity_check = true
cluster_template = fds-smv-cluster
update_check = true

[vpc public]
vpc_id = vpc-<VPC-ID>
m ster_subnet_id = subnet-<SUBNET-ID>

[cluster fds-smv-cluster]
key_name = <Key-Name>
vpc_settings = public
compute_instance_type=c5n.18xlarge
m ster_instance_type=c5.xlarge
initial_queue_size = 0
max_queue_size = 100
scheduler=slurm
cluster_type = ondemand
s3_read_write_resource=arn:aws:s3:::fds-smv-bucket-unique*
placement_group = DYNAMIC
placement = compute
base_os = alinux2
tags = {"Name" : "fds-smv"}
disable_hyperthreading = true
fsx_settings = fsxshared
enable_efa = compute
dcv_settings = hpc-dcv

[dcv hpc-dcv]
enable = m ster

[fsx fsxshared]
shared_dir = /fsx
storage_capacity = 1200
import_path = s3://fds-smv-bucket-unique
imported_file_chunk_size = 1024
export_path = s3://fds-smv-bucket-unique

[aliases]
ssh = ssh {CFN_USER}@{M STER_IP} {ARGS}

我无法创建一个通过Parallel Cluster create-cluster接受的YAML文件。它返回以下错误:

{
  "message": "Bad Request: Configuration must be a valid YAML document"
}

我试图使用AWS Parallel configure向导(https://docs.aws.amazon.com/parallellecluster/latest/latest/ug/install-v3-configuring.html)创建YAML文件,像教程要求的那样在FSX中共享的S3存储桶。我尝试从向导创建的配置文件开始并进行编辑,以使其看起来像文档中的YAML文件,但是它仍然返回相同的错误。

在此处查看我编辑的yaml文件:

Region: us-east-2
Image:
  Os: alinux2
HeadNode:
  InstanceType: c5.xlarge
  Networking:
    SubnetId: subnet-032f3e6409362aff2
  Ssh:
    KeyName: MyKeyPair1
  DisableSimultaneousMultithreading: true
Scheduling:
  Scheduler: slurm
  SlurmQueues:
  - Name: queue1
    CapacityType: ONDEMAND
    ComputeResources:
    - Name: c5n18xlarge
      InstanceType: c5n.18xlarge
      MinCount: 0
      MaxCount: 100
	  Efa:
	    Enabled: true  
    Networking:
      SubnetIds:
      - subnet-032f3e6409362aff2
    Iam:
      S3Access:
        - BucketName: fds-smv-jts-bucket-1
          EnableWriteAccess: True
      AdditionalIamPolicies:
        - Policy: arn:aws:s3:::fds-smv-jts-bucket-1*
SharedStorage:
  - MountDir: /fsx
    StorageType: FsxLustre  
	FsxLustreSettings:
      StorageCapacity: 1200
      ImportedFileChunkSize: 1024
      ExportPath: s3://fds-smv-jts-bucket-1
      ImportPath: s3://fds-smv-jts-bucket-1
Tags:
  - Key: String
    Value: fds-smv
DevSettings
  ClusterTemplate: fds-smv-cluster

关于如何使用本教程请求的所有数据创建适当的YAML文件的任何想法?

谢谢你!

1 Antwort
0

【以下的回答经过翻译处理】 在你的yaml文件中,DevSettings键后缺少分号。我使用yamllint(https://pypi.org/project/yamllint/)来检查我的yaml文件,在验证yaml时可能会有所帮助。

另外,PCluster 3包括一个配置转换器(https://docs.aws.amazon.com/parallelcluster/latest/ug/pcluster3-config-converter.html)。建议你尝试对你的 3.x 之前的配置文件执行该转换器。

你也可以在这里找到Parallel Cluster 3配置文件格式:https://docs.aws.amazon.com/parallelcluster/latest/ug/cluster-configuration-file-v3.html。我认为你要找的是与FSX系统关联的导入路径和导出路径,在3.x配置文件中在这里(https://docs.aws.amazon.com/parallelcluster/latest/ug/SharedStorage-v3.html#yaml-SharedStorage-FsxLustreSettings-ImportPath) 。

profile picture
EXPERTE
beantwortet vor 5 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen