We recently experienced an issue where our glue connection ran out of IP addresses inside the subnet that it was created in. Error message: ``` AWS Glue Error: The specified subnet does not have enough free addresses to satisfy the request ``` This was due to too many workers/dpus being assigned to the jobs in a single subnet. I have two questions: 1. Since glue appears to take a **list** of connections, will it "roll-over" when one subnet is maxxed out and use the next connection in the list? (at least this is what it shows in Terraform module--a list argument) 2. How many IP addresses does a G.2x worker use? Since its 2 dpu, does it use 2 IPs? Or is it variable?

1. No, all the cluster will be in a single subnet from a single connection. 2. One per worker, doesn't matter if the worker is 1DPU or 8

Glue connections running out of IPs

We recently experienced an issue where our glue connection ran out of IP addresses inside the subnet that it was created in.

Error message:

AWS Glue Error: The specified subnet does not have enough free addresses to satisfy the request

This was due to too many workers/dpus being assigned to the jobs in a single subnet.

I have two questions:

Since glue appears to take a list of connections, will it "roll-over" when one subnet is maxxed out and use the next connection in the list? (at least this is what it shows in Terraform module--a list argument)
How many IP addresses does a G.2x worker use? Since its 2 dpu, does it use 2 IPs? Or is it variable?

トピック

タグ

言語

English

質問済み 1年前499ビュー

2回答