Questions on yml for simple EMR cluster(1 master+1 core)

Question

I am drafting a yaml for simple EMR cluster. It will be loaded to stack and run.

But I am tried to launch my yml with servicerole without any attached policies and jobflowrole without any policies. It said invalid servicerole.
Am I missing sth? Are there any requirments on servicerole and jobflowrole?
Any syntax issues?
Updates: The below is working.
```
---
Parameters:
  EmrReleaseLabelParameter:
    Type: String
    Default: emr-6.10.0

Resources:       
  EMRCluster:
    Type: AWS::EMR::Cluster
    Properties:
      Name: EMR_Two_Node_Cluster-20231108
      ReleaseLabel: !Ref EmrReleaseLabelParameter
      Applications:
        - Name: Hadoop
        - Name: Spark
        - Name: Zeppelin
      Instances:
        MasterInstanceGroup:
          Name: Master
          InstanceCount: 1
          InstanceType: c6g.xlarge 
          Market: ON_DEMAND
        CoreInstanceGroup:
          Name: Core
          InstanceCount: 1
          InstanceType: c6g.xlarge  
          Market: ON_DEMAND
      ServiceRole: !Ref EmrRole
      JobFlowRole: !Ref EmrEc2InstanceProfile
      LogUri: 's3://emr-log-123456789/123456789/'             #created before hand

EmrRole:     #servicerole
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2008-10-17
        Statement:
          - Sid: ''
            Effect: Allow
            Principal:
              Service:
                 - elasticmapreduce.amazonaws.com
            Action: 'sts:AssumeRole'
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole'

EmrEc2InstanceProfile:
    Type: AWS::IAM::InstanceProfile
    Properties:
      Path: /
      Roles:
        - !Ref EmrEc2Role

EmrEc2Role:     #refered by instanceProfile
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2008-10-17
        Statement:
          - Sid: '' 
            Effect: Allow
            Principal:
              Service:
                 - ec2.amazonaws.com
            Action: 'sts:AssumeRole'
      Path: /
      ManagedPolicyArns:
        - 'arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role'

```
Thanks.

Accepted Answer

Hello,

Yes, JobFlowRole and ServiceRole are mandatory to be included when provisioning the cluster as they are required to interact with other AWS services like EC2, S3 and emr services. These roles assume the other AWS service like below mentioned, Please include the roles in the below format and retry the execution,

```
    ServiceRole:
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Statement:
                    - Effect: Allow
                      Action: sts:AssumeRole
                      Principal:
                          Service:
                              - elasticmapreduce.amazonaws.com
            ManagedPolicyArns:
                - arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceRole

JobFlowRole:
        Type: AWS::IAM::Role
        Properties:
            AssumeRolePolicyDocument:
                Statement:
                    - Effect: Allow
                      Action: sts:AssumeRole
                      Principal:
                          Service:
                              - ec2.amazonaws.com
            ManagedPolicyArns:
                - arn:aws:iam::aws:policy/service-role/AmazonElasticMapReduceforEC2Role

```
 
Please refer the example mentioned in this document - https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-emr-cluster.html#aws-resource-emr-cluster--examples

Answer

Hello,

Glad it worked for you. For vpc subnet, refer the below,
Mention the subnet under instances branch that will take the appropriate uniquely identified subnet under a vpc,

```
Instances:
        Ec2SubnetId: < refer the SubnetId >
```

VpcId might require when you specify the security group properties which tagged to specific vpc.

```
   SecurityGroup:
        Type: AWS::EC2::SecurityGroup
        Properties:
            GroupDescription: "Allow SSH from anywhere"
            VpcId: !Ref VpcId
            SecurityGroupIngress:
                - IpProtocol: tcp
                  FromPort: 22
                  ToPort: 22
                  CidrIp: 0.0.0.0/0
```

Example: https://github.com/wwbrannon/emrcfn/blob/master/emrcfn.yaml

Questions on yml for simple EMR cluster(1 master+1 core)

相关内容