Skip to content

Feature Request: Amazon RDS PostgreSQL 18 support for zero-ETL integration to Amazon Redshift

0

Hi team, I see that Amazon Aurora PostgreSQL 18.3+ already natively supports zero-ETL pipelines to Amazon Redshift. However, our current architecture relies on standard Amazon RDS, and we are planning an upgrade to PostgreSQL 18. Are there any timelines or roadmaps for when standard RDS PostgreSQL 18 will receive zero-ETL support to Redshift? Thank you!

2 Answers
0

The technical alignment between operational online transaction processing environments and real-time analytical data warehouses requires a predictable infrastructure deployment lifecycle. Amazon Web Services routinely protects its operational predictability by withholding public feature roadmaps or definitive release dates for unreleased managed engine versions, such as standard Amazon Relational Database Service for PostgreSQL version 18. Because feature availability for community engine updates follows a sequential engineering validation process, managed capabilities like zero-ETL data replication typically lag behind the initial engine version launch while underlying change data capture mechanisms are verified for scale. To bypass this roadmap dependency and successfully execute an architecture strategy targeting PostgreSQL 18, you must implement an operational solution that establishes analytical synchronization without waiting for a native managed integration pathway.

The first technical approach to achieving immediate zero-ETL replication capabilities while upgrading your data layer involves migrating your standard database workload over to an Amazon Aurora PostgreSQL-Compatible Edition cluster. Amazon Aurora serves as the primary development platform for native data integrations within the cloud ecosystem, meaning major version upgrades such as version 18 natively receive zero-ETL capabilities significantly faster than standard relational database service offerings. To execute this step, you utilize standard snapshot restoration or configure an external logical replication link from your existing database to a new Amazon Aurora cluster running version 18. Once the database is migrated to the Aurora framework, you can immediately utilize the native, built-in cloud orchestration pipeline to establish near real-time data replication directly into your Amazon Redshift data warehouse without managing individual schema mapping or custom extraction scripts.

The second technical approach enables your team to remain entirely on the standard Amazon Relational Database Service infrastructure while manually constructing an equivalent, highly resilient change data capture pipeline using cloud-native data migration components. To simulate the behavior of a zero-ETL integration on a standard database instance running version 18, you must first access your custom database parameter group to enable native logical replication parameters. Specifically, you alter the database parameter settings to turn on logical replication, increase the maximum number of write-ahead log senders, and adjust replication slots to allocate a stable buffer for outgoing data changes. This technical configuration converts your transactional engine into a streaming publisher, allowing downstream consumption tools to capture every individual insertion, update, and deletion block as soon as a transaction commits.

The third technical phase requires configuring an intermediate managed streaming service to bridge the gap between your standard database publication slots and your target analytics data store. You deploy an AWS Database Migration Service task configured for ongoing replication to capture the newly available logical replication stream from your database. The migration service connects directly to your database engine, monitors the transactional write-ahead logs, and continuously transforms those log entries into a format optimized for analytical ingestion. By targeting your Amazon Redshift cluster as the destination endpoint for this migration task, you establish a automated, continuous data pipeline that maintains sub-minute synchronization between your production transactions and your business intelligence reporting dashboards, effectively delivering the core operational benefits of a zero-ETL integration manually.

The final technical step involves ensuring your target analytical warehouse is structured to ingest the incoming change data capture records smoothly and handle schema drift without dropping connections. You must confirm that your target Amazon Redshift cluster is deployed using modern architectural node types, such as high-performance storage-optimized nodes, which support continuous data injection without bottlenecking your concurrent read queries. Additionally, you enable case-sensitive identifiers within your warehouse cluster configuration to ensure incoming column and table names map perfectly to their source relational layouts. By establishing this explicit schema and connection alignment, you insulate your analytics platform from operational changes, allowing your engineering team to proceed with your targeted major engine version upgrades on your exact deployment schedule without relying on future feature roadmap announcements.

Amazon RDS zero-ETL integrations: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/zero-etl.html

answered 14 days ago

0

Currently, Amazon RDS for PostgreSQL zero-ETL integrations with Amazon Redshift support PostgreSQL versions 15.7+, 16.3+, and 17.1+. PostgreSQL 18 is not yet supported for RDS zero-ETL integrations to Redshift.

While I don't have access to specific timelines or roadmaps for when PostgreSQL 18 support will be added to RDS zero-ETL integrations, AWS typically extends support to newer database versions after they've been thoroughly tested and validated. Given that earlier major versions (15, 16, and 17) are already supported, it's reasonable to expect that PostgreSQL 18 support may be added in the future, but I cannot provide a specific timeline.

For the most current information about feature availability and timelines, I recommend checking the AWS What's New announcements.
Sources
Amazon RDS zero-ETL integrations - Amazon Relational Database Service

answered 14 days ago

EXPERT

revised 14 days ago

EXPERT

revised 14 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.