Zero ETL integration stops working after a few days
I have configured zero ETL integration between Aurora MySQL database and Redshift serverless, the integration is showing as active.
This worked for 3 days then stopped. I recreated the zero ETL from scratch then it stopped working again after 3 days. This is the error I'm getting "ERROR: Zero-ETL database is not available for querying right now. See SVV_INTEGRATION for more information":
The SVV_INTEGRATION table is not showing any error:
All tables are showing as synced without errors.
The Aurora database is running and I verified it's working by connecting with mysql workbench and running a few queries.
How can I find out why zero ETL stops working after few days every time? Is there any way to fix it other than recreating integration everytime?
- Newest
- Most votes
- Most comments
There are a lot of things that can go wrong when dealing with Zero ETL on Aurora.
- Check the Aurora cluster parameters such as
max_allowed_packetwhich need to be set sufficiently. - Does Aurora have primary keys defined?
- Make sure the IAM roles have the necessary permissions for the integration to access both Aurora and Redshift.
- Avoid operations on Aurora tables that require resynchronization like modifying multiple columns at once. These can cause the integration to stop until resync is complete.
- Delete and recreate the integration.
https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/zero-etl.troubleshooting.html
Relevant content
- Accepted Answerasked 4 months ago
- asked 7 months ago
- asked 4 months ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 2 years ago
Can you clarify how to determine sufficient value for max_allowed_packet? The etl documentation does not mention this parameter as required anywhere. How would any of the primary keys, iam roles etc cause it to stop working after 3 days every time after recreation, wouldn't it just not work from the beginning?