Recent slowness with Glue Crawlers

0

Are there any known, recently (~07/18/2023) introduced performance issues with Glue crawlers?

We have recently observed excessive slowness with Glue crawlers that had been running for months without issue. The crawlers in question are all crawling S3 data sources. We are seeing this across multiple crawlers in both our pre-production and production environments. Our pre-production volume is quite low, so not a data volume issue and in at least one case, the crawler is configured to use S3 Event notifications (so not a full re-crawl every time).

For context, crawlers that typically took 6-10 minutes to complete are now routinely taking 50 minutes or longer, up to around 2 hours, though nothing has changed with our configuration or data. We've found nothing unusual in the Crawler logs.

First observed instances of slowness occurred on July 18th. At first the issues seemed intermittent. Now it is consistent for the affected crawlers.

TimD
gefragt vor 9 Monaten28 Aufrufe
Keine Antworten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen