Recent slowness with Glue Crawlers

0

Are there any known, recently (~07/18/2023) introduced performance issues with Glue crawlers?

We have recently observed excessive slowness with Glue crawlers that had been running for months without issue. The crawlers in question are all crawling S3 data sources. We are seeing this across multiple crawlers in both our pre-production and production environments. Our pre-production volume is quite low, so not a data volume issue and in at least one case, the crawler is configured to use S3 Event notifications (so not a full re-crawl every time).

For context, crawlers that typically took 6-10 minutes to complete are now routinely taking 50 minutes or longer, up to around 2 hours, though nothing has changed with our configuration or data. We've found nothing unusual in the Crawler logs.

First observed instances of slowness occurred on July 18th. At first the issues seemed intermittent. Now it is consistent for the affected crawlers.

TimD
preguntada hace 9 meses28 visualizaciones
No hay respuestas

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas