Recent slowness with Glue Crawlers

0

Are there any known, recently (~07/18/2023) introduced performance issues with Glue crawlers?

We have recently observed excessive slowness with Glue crawlers that had been running for months without issue. The crawlers in question are all crawling S3 data sources. We are seeing this across multiple crawlers in both our pre-production and production environments. Our pre-production volume is quite low, so not a data volume issue and in at least one case, the crawler is configured to use S3 Event notifications (so not a full re-crawl every time).

For context, crawlers that typically took 6-10 minutes to complete are now routinely taking 50 minutes or longer, up to around 2 hours, though nothing has changed with our configuration or data. We've found nothing unusual in the Crawler logs.

First observed instances of slowness occurred on July 18th. At first the issues seemed intermittent. Now it is consistent for the affected crawlers.

TimD
demandé il y a 9 mois28 vues
Aucune réponse

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions