Lambda to query from RDS and store to S3 bucket

0

Hi,

I want to understand the best way to query RDS using a lambda function to fetch data and write to another AWS account S3 bucket. Do we need to create view? how long lambda can be executed? What should be batching strategy? Can it write in parquet format?

Thanks in advance.

1 回答
0

Hi mr cloud.

I want to understand the best way to query RDS using a lambda function to fetch data and write to another AWS account S3 bucket.

For Fetching data I d recommend RDS Proxy so it handles automatically connection pool and more: https://github.com/aws-samples/amazon-rds-proxy-video-demo

Do we need to create view?

If you mean database view, no, no need.

how long lambda can be executed?

Max lambda execution is 15 minutes. Id though strongly recommend to keep it much shorter for that. If the workload is a long running one, lambda May not be best fit.

What should be batching strategy?

If you mean in regards database, Rds proxy will alleviate that otherwise sdk actions are available.

Can it write in parquet format?

Yes. Use python or node JS libraries/packages to perform that.

Hope it helps, and if does, I d appreciate to get accepted as answer so that community can benefit for clarity when asking similar questions, cheers!

profile picture
专家
已回答 1 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则