- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
Using AWS Step Functions is the way to go when you're trying to setup polling without making it too complex.
The workflow using SQS isn't unheard of. I have personally used it too. But this is a problem that Step Functions can solve in a better way.
Not sure if it is applicable, but, did you try the integration of Amazon AppFlow and Marketo? If it meets your requirements you can have a fully automated solution. https://docs.aws.amazon.com/appflow/latest/userguide/marketo.html
Hi The reason I did not choose Appflow was since it does not support Assets. Also I need something that can enque jobs first and then pull data, since running a fill job will cause the 500MB exceeded error.
My understanding is that if we can queue the 2 jobs first, then it does not matter If I exceed the limit, I can still download the files. Thanks
Contenuto pertinente
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 3 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
Thank you, I was looking at the same example. However I am trying to do it without the Batch Job. Ty
Also what I understand is you are suggesting to implement the complete flow without using the SQS, so move the #2 lambda in my case as part of step function, and put a default timer of 10 mins, and the proceed with the polling checks?
That is only an example. You can also use a Lambda function as the first Element of the execution. With Step Functions you can orchestrate your workflow, and you can interact with over 200 services including Lambda. https://docs.aws.amazon.com/step-functions/latest/dg/connect-supported-services.html
So i also strongly recommend to use Step Functions for this use case.