Recent slowness with Glue Crawlers

0

Are there any known, recently (~07/18/2023) introduced performance issues with Glue crawlers?

We have recently observed excessive slowness with Glue crawlers that had been running for months without issue. The crawlers in question are all crawling S3 data sources. We are seeing this across multiple crawlers in both our pre-production and production environments. Our pre-production volume is quite low, so not a data volume issue and in at least one case, the crawler is configured to use S3 Event notifications (so not a full re-crawl every time).

For context, crawlers that typically took 6-10 minutes to complete are now routinely taking 50 minutes or longer, up to around 2 hours, though nothing has changed with our configuration or data. We've found nothing unusual in the Crawler logs.

First observed instances of slowness occurred on July 18th. At first the issues seemed intermittent. Now it is consistent for the affected crawlers.

TimD
feita há 9 meses28 visualizações
Sem respostas

Você não está conectado. Fazer login para postar uma resposta.

Uma boa resposta responde claramente à pergunta, dá feedback construtivo e incentiva o crescimento profissional de quem perguntou.

Diretrizes para responder a perguntas