How to introduce distributed mode in maps using aws CDK

0

I have been trying to get a mitigation for the error that pops up in the state machine's execution

The execution reached the maximum number of history events (25000).

I am trying to add a DISTRIBUTED mode for my maps that are calling the s3 api (listObjects). I want to add this functionality at the CDK code in typescript which can be used in deployment of CFN stack and we have a form of tracking and giving relevant permissions for the state machine. example code :

        const listLimitDirectories = new CallAwsService(this, 'listLimitDirectories', {
            service: 's3',
            action: 'listObjectsV2',
            parameters: {
                Bucket: bucketName,
                Prefix: 'limits/',
                Delimiter: '/',
            },
            iamResources:[bucketArn, bucketArn + '/*'],
            iamAction: 's3:getObject'
        });

        const iterateLimitDirectories = new Map(this, "iterateLimitDirectories", {
            maxConcurrency: 24,
            itemsPath: "$.CommonPrefixes",
            resultPath: "$.Contents"
        })

It would be great to get some hands on resources that give some ideas on how to introduce this DISTRIBUTED mode in the code level

gefragt vor 9 Monaten475 Aufrufe
1 Antwort
0

Hi, look at this Github ticket: https://github.com/aws/aws-cdk/issues/23216

You have some code samples that you are looking for toward the bottom of the ticket

Also this one may be of interest for you: https://github.com/alessandromr/aws-sfn-map-demo with joint article at https://alessandromarinoac.com/unleashing-the-power-of-large-scale-parallel-data-processing-on-aws

Best,

Didier

profile pictureAWS
EXPERTE
beantwortet vor 9 Monaten
profile picture
EXPERTE
überprüft vor 9 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen