How to introduce distributed mode in maps using aws CDK

Question

I have been trying to get a mitigation for the error that pops up in the state machine's execution 
```
The execution reached the maximum number of history events (25000).
```

I am trying to add a DISTRIBUTED mode for my maps that are calling the s3 api (listObjects).
I want to add this functionality at the CDK code in typescript which can be used in deployment of CFN stack and we have a form of tracking and giving relevant permissions for the state machine.
example code : 
```
        const listLimitDirectories = new CallAwsService(this, 'listLimitDirectories', {
            service: 's3',
            action: 'listObjectsV2',
            parameters: {
                Bucket: bucketName,
                Prefix: 'limits/',
                Delimiter: '/',
            },
            iamResources:[bucketArn, bucketArn + '/*'],
            iamAction: 's3:getObject'
        });

const iterateLimitDirectories = new Map(this, "iterateLimitDirectories", {
            maxConcurrency: 24,
            itemsPath: "$.CommonPrefixes",
            resultPath: "$.Contents"
        })
```

It would be great to get some hands on resources that give some ideas on how to introduce this DISTRIBUTED mode in the code level

Answer

Hi, look at this Github ticket: https://github.com/aws/aws-cdk/issues/23216

You have some code samples that you are looking for toward the bottom of the ticket

Also this one may be of interest for you: https://github.com/alessandromr/aws-sfn-map-demo with joint article at https://alessandromarinoac.com/unleashing-the-power-of-large-scale-parallel-data-processing-on-aws

Best,

Didier

How to introduce distributed mode in maps using aws CDK

相關內容