How to introduce distributed mode in maps using aws CDK

0

I have been trying to get a mitigation for the error that pops up in the state machine's execution

The execution reached the maximum number of history events (25000).

I am trying to add a DISTRIBUTED mode for my maps that are calling the s3 api (listObjects). I want to add this functionality at the CDK code in typescript which can be used in deployment of CFN stack and we have a form of tracking and giving relevant permissions for the state machine. example code :

        const listLimitDirectories = new CallAwsService(this, 'listLimitDirectories', {
            service: 's3',
            action: 'listObjectsV2',
            parameters: {
                Bucket: bucketName,
                Prefix: 'limits/',
                Delimiter: '/',
            },
            iamResources:[bucketArn, bucketArn + '/*'],
            iamAction: 's3:getObject'
        });

        const iterateLimitDirectories = new Map(this, "iterateLimitDirectories", {
            maxConcurrency: 24,
            itemsPath: "$.CommonPrefixes",
            resultPath: "$.Contents"
        })

It would be great to get some hands on resources that give some ideas on how to introduce this DISTRIBUTED mode in the code level

已提问 9 个月前475 查看次数
1 回答
0

Hi, look at this Github ticket: https://github.com/aws/aws-cdk/issues/23216

You have some code samples that you are looking for toward the bottom of the ticket

Also this one may be of interest for you: https://github.com/alessandromr/aws-sfn-map-demo with joint article at https://alessandromarinoac.com/unleashing-the-power-of-large-scale-parallel-data-processing-on-aws

Best,

Didier

profile pictureAWS
专家
已回答 9 个月前
profile picture
专家
已审核 9 个月前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则

相关内容