AWS CDK - AWS EMR (creating task instances)

0

Hello,

I was trying to create emr cluster using aws_cdk.aws_emr module, but I could not figure out a way to add task instances. It is possible using aws_cdk.aws_stepfunctions_tasks, but then it does not have manged_scaling_policy as available in aws_cdk.aws_emr .

If anyone here could point me towards a way to achieve it, I would really appreciate it.

Thank you, Manjil

Manjil
feita há 2 anos850 visualizações
2 Respostas
0

Hello,

You can use aws_cdk.aws_emr property "JobFlowInstancesConfigProperty" for instantiating task instances. Please see below example and hope this helps.

``

eg:
emr_clst = CfnCluster(self,"my_emr_cluster",
            name="my-test-cluster",
            applications=[CfnCluster.ApplicationProperty(name="Hive")],
            log_uri=f"s3://mybucket/emr/",
            release_label=<<emr_version>>,
            visible_to_all_users=True,
            service_role=<<service_role>>
            job_flow_role=<<job_flow_role>>,
            instances=CfnCluster.JobFlowInstancesConfigProperty(
                termination_protected=False,
                master_instance_group=CfnCluster.InstanceGroupConfigProperty(
                    instance_count=1, 
                    instance_type="c5.9xlarge"
                ),
                core_instance_group=CfnCluster.InstanceGroupConfigProperty(
                    instance_count=5, 
                    instance_type="c5.9xlarge"
                ),
                ec2_subnet_id=<<subnet_id>>
            ),
            configurations=[
                CfnCluster.ConfigurationProperty(
                    ...
                    ...
                    ...
                )
            ]
        )

``

Please replace the items in << >> with the respecting values and refer the documentation for more details- https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-emr.CfnCluster.JobFlowInstancesConfigProperty.html.

AWS
respondido há 2 anos
  • Thank you for your answer. I also reached to this point, but I am not really sure on what goes inside CfnCluster.ConfigurationProperty?

0

These are the configuration properties that you want to add it to the EMR cluster. A configuration consists of a classification, properties, and optional nested configurations. A classification refers to an application-specific configuration file. Properties are the settings you want to change in that file.

Please refer the link for more details on ConfigurationProperty - https://docs.aws.amazon.com/cdk/api/v1/docs/@aws-cdk_aws-emr.CfnCluster.ConfigurationProperty.html

AWS
respondido há 2 anos
  • Thank you for pointing it out, but could you please provide with a short example on what the classification would be, what to put in the properties? Because, I did not find it in the document. May be you could point me towards it too.

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas