Lambda to query from RDS and store to S3 bucket

0

Hi,

I want to understand the best way to query RDS using a lambda function to fetch data and write to another AWS account S3 bucket. Do we need to create view? how long lambda can be executed? What should be batching strategy? Can it write in parquet format?

Thanks in advance.

1 Risposta
0

Hi mr cloud.

I want to understand the best way to query RDS using a lambda function to fetch data and write to another AWS account S3 bucket.

For Fetching data I d recommend RDS Proxy so it handles automatically connection pool and more: https://github.com/aws-samples/amazon-rds-proxy-video-demo

Do we need to create view?

If you mean database view, no, no need.

how long lambda can be executed?

Max lambda execution is 15 minutes. Id though strongly recommend to keep it much shorter for that. If the workload is a long running one, lambda May not be best fit.

What should be batching strategy?

If you mean in regards database, Rds proxy will alleviate that otherwise sdk actions are available.

Can it write in parquet format?

Yes. Use python or node JS libraries/packages to perform that.

Hope it helps, and if does, I d appreciate to get accepted as answer so that community can benefit for clarity when asking similar questions, cheers!

profile picture
ESPERTO
con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande