Lambda to query from RDS and store to S3 bucket

0

Hi,

I want to understand the best way to query RDS using a lambda function to fetch data and write to another AWS account S3 bucket. Do we need to create view? how long lambda can be executed? What should be batching strategy? Can it write in parquet format?

Thanks in advance.

1 個回答
0

Hi mr cloud.

I want to understand the best way to query RDS using a lambda function to fetch data and write to another AWS account S3 bucket.

For Fetching data I d recommend RDS Proxy so it handles automatically connection pool and more: https://github.com/aws-samples/amazon-rds-proxy-video-demo

Do we need to create view?

If you mean database view, no, no need.

how long lambda can be executed?

Max lambda execution is 15 minutes. Id though strongly recommend to keep it much shorter for that. If the workload is a long running one, lambda May not be best fit.

What should be batching strategy?

If you mean in regards database, Rds proxy will alleviate that otherwise sdk actions are available.

Can it write in parquet format?

Yes. Use python or node JS libraries/packages to perform that.

Hope it helps, and if does, I d appreciate to get accepted as answer so that community can benefit for clarity when asking similar questions, cheers!

profile picture
專家
已回答 1 年前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南