ConnectionRefusedError: [Errno 111] Connection refused

0

While running a glue job, when writing parquet files encountered the following error Error : ConnectionRefusedError: [Errno 111] Connection refused I tried running it by increasing the number of workers, but it was no help. Could someone help me with the same Thanks

  • Find the stacktrace in the log to try to figure out what is trying to connect to what

質問済み 1年前1448ビュー
1回答
0

Hi,

I understand that you are facing the “ConnectionRefusedError” while running your glue job. This can happen because of multiple reasons.

  1. If you are using any glue connection in your job then please ensure that the “test connection” is successful.

  2. Please check the CloudWatch Logs to see if there is any specific error other than the “ConnectionRefusedError”.

  3. If you are observing the "Max number of executor failures reached" error or any errors that shows that this could be a memory issue then please make sure you are using the “G.2X” worker type as this is the largest worker type available for Glue ETL jobs. If you still face the memory issues even after this then you would need to look further into optimising the glue script. Please refer to the following documentation for more details on the same -

    https://aws.amazon.com/blogs/big-data/optimize-memory-management-in-aws-glue/

  4. If the issue still persist, then please open a support case with AWS Glue providing the connection details along with the Job Run Id and Region - https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-case

Thank you.

AWS
サポートエンジニア
回答済み 1年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ