- Newest
- Most votes
- Most comments
Yes, auto-copy jobs in Amazon Redshift are designed to handle maintenance windows gracefully. When your Redshift cluster is undergoing maintenance or is paused, the auto-copy jobs will not run during that time. However, the system will continue to monitor the S3 path for new files.
Once the maintenance is complete and the cluster is back online, the auto-copy functionality will resume and process any files that were added to the S3 bucket during the maintenance window. This behavior is part of the design of the auto-copy feature, which continuously monitors the specified S3 path and automatically copies new files as they arrive.
It's worth noting that if you're using a provisioned Redshift cluster, COPY jobs don't run when the cluster is paused. For Redshift Serverless, as long as your workgroup is active after the maintenance, the auto-copy jobs will continue processing.
To verify that files are being properly loaded after maintenance, you can query system tables like SYS_COPY_JOB_DETAIL to view the status of your S3 files and check if they have been loaded, are pending ingestion, or if there are any errors. Additionally, STL_LOAD_COMMITS, STL_LOAD_ERRORS, and STL_LOADERROR_DETAIL can help you verify data loading and identify any issues that might occur.
Sources
Redshift auto copy job is not working | AWS re:Post
COPY JOB - Amazon Redshift
Redshift serverless: Autocopy jobs not displayed in web UI | AWS re:Post
Relevant content
asked a year ago
asked a year ago
