Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Scheduler to fetch data from On-prem db

0

We have a requirement to sync the data from the on-prem database to AWS RDS (PotgreSQL) at specific intervals (unlike one-time data migration). Assume there is an Interconnect/VPN already established between AWS and On-prem network. The expected data volume is likely 1000 rows only, so I do not see the necessity to build ETL with AWS Glue. Given that, what are the possible solution options to fetch the data?
Can AWS Batch/a pg_cron job be considered here to execute a set of select and update SQLs? Alternatively, how do we trigger the AWS Lambda at certain intervals if AWS Lambda is a solution option for this requirement?

Appreciate your input.

Topics
ServerlessAnalyticsComputeMigration & ModernizationDatabase
Tags
AWS DataSyncAWS Data ExchangePostgreSQLAWS GlueAWS Lambda
Language
English
asked 3 years ago432 views
1 Answer
0

Based on the volume, Lambda might be a very good option. The best way to invoke it on a schedule is to use EventBridge Scheduler.

AWS
EXPERT
answered 3 years ago
  • cloudarch
    3 years ago

    Thanks Uri. But, though EventBridge is a serverless service, do you think an EventBridge event bus is required for this simple daily job scenario? Can't we manage an AWS Batch/a pg_cron job (within PostgreSQL db) to trigger a Lambda at a specific interval?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content