Recent slowness with Glue Crawlers

0

Are there any known, recently (~07/18/2023) introduced performance issues with Glue crawlers?

We have recently observed excessive slowness with Glue crawlers that had been running for months without issue. The crawlers in question are all crawling S3 data sources. We are seeing this across multiple crawlers in both our pre-production and production environments. Our pre-production volume is quite low, so not a data volume issue and in at least one case, the crawler is configured to use S3 Event notifications (so not a full re-crawl every time).

For context, crawlers that typically took 6-10 minutes to complete are now routinely taking 50 minutes or longer, up to around 2 hours, though nothing has changed with our configuration or data. We've found nothing unusual in the Crawler logs.

First observed instances of slowness occurred on July 18th. At first the issues seemed intermittent. Now it is consistent for the affected crawlers.

TimD
posta 9 mesi fa28 visualizzazioni
Nessuna risposta

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande