- Mais recentes
- Mais votos
- Mais comentários
Using AWS Step Functions is the way to go when you're trying to setup polling without making it too complex.
The workflow using SQS isn't unheard of. I have personally used it too. But this is a problem that Step Functions can solve in a better way.
Not sure if it is applicable, but, did you try the integration of Amazon AppFlow and Marketo? If it meets your requirements you can have a fully automated solution. https://docs.aws.amazon.com/appflow/latest/userguide/marketo.html
Hi The reason I did not choose Appflow was since it does not support Assets. Also I need something that can enque jobs first and then pull data, since running a fill job will cause the 500MB exceeded error.
My understanding is that if we can queue the 2 jobs first, then it does not matter If I exceed the limit, I can still download the files. Thanks
Conteúdo relevante
- AWS OFICIALAtualizada há um ano
- AWS OFICIALAtualizada há um ano
Thank you, I was looking at the same example. However I am trying to do it without the Batch Job. Ty
Also what I understand is you are suggesting to implement the complete flow without using the SQS, so move the #2 lambda in my case as part of step function, and put a default timer of 10 mins, and the proceed with the polling checks?
That is only an example. You can also use a Lambda function as the first Element of the execution. With Step Functions you can orchestrate your workflow, and you can interact with over 200 services including Lambda. https://docs.aws.amazon.com/step-functions/latest/dg/connect-supported-services.html
So i also strongly recommend to use Step Functions for this use case.