Skip to content

General Question - test_decoding / pgoutput / multiple replication slots

0

Hi,

Some general question of some sort. I can't find any topic close to what I am afer.

Aurora PostgreSQL 13 Multi-AZ Plugin test_decoding, pgoutput

Presumably, we can have multiple replication slots. Is it advisable or safe to do so though? For example, can we have test_decoding_slot1, test_decoding_slot2, pgoutput_slot1 and then have several connectors to each of the slot processing different tables or will this cause a problem later on with LSN getting moved forward and consume by one of the connectors.

asked a year ago628 views
1 Answer
0

Yes, it is possible to have multiple replication slots in Aurora PostgreSQL 13, including a combination of slots using different logical decoding plugins like test_decoding and pgoutput. However, there are some important considerations to keep in mind:

  1. Resource usage: Each replication slot consumes resources on the database server. Multiple slots can increase the load on your system, especially if they are actively used.

  2. WAL retention: Replication slots prevent the write-ahead log (WAL) from being removed until all slots have processed the data. If one slot falls behind, it can cause WAL buildup, potentially leading to increased storage usage and performance issues.

  3. LSN management: Each replication slot tracks its own Log Sequence Number (LSN). The database will retain WAL segments until all slots have consumed them. This means that if one connector falls significantly behind others, it can impact the overall replication process and storage usage.

  4. Performance impact: Using multiple slots and connectors can increase the load on your database, potentially affecting its performance. This is especially true if you're replicating a large number of tables or processing high volumes of changes.

  5. Complexity: Managing multiple slots and connectors adds complexity to your setup, which can make troubleshooting and maintenance more challenging.

While it's technically possible to set up multiple slots as you described (test_decoding_slot1, test_decoding_slot2, pgoutput_slot1), it's generally advisable to carefully consider your use case and requirements before implementing such a configuration. If you need to process different tables with different connectors, you might want to explore other options, such as:

  1. Using a single slot with a connector that can filter and route changes to different destinations based on the table.
  2. Implementing a custom solution that can read from a single slot and distribute the changes to multiple consumers.

If you do decide to use multiple slots, it's crucial to monitor them closely for replication lag, WAL retention, and overall system performance. Regularly check the status of your replication slots and ensure that all connectors are keeping up with the changes to avoid potential issues with LSN advancement and WAL buildup.

Sources
Cross-account Amazon Aurora PostgreSQL and Amazon RDS for PostgreSQL migration with reduced downtime using AWS DMS | AWS Database Blog
Monitoring the write-through cache and logical slots for Aurora PostgreSQL logical replication - Amazon Aurora

answered a year ago
AWS
SUPPORT ENGINEER
reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.