Questions tagged with Elastic Load Balancing

Content language: English

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Intermittent health check timeouts causing ECS to kill tasks

We have an ECS service running our API. Normally this service runs with ~12 tasks. The service is configured with an HTTP health check that returns a 200 if certain conditions are met - usually this returns within ~200ms. We have a scaling policy that starts new tasks based on the average CPU of our tasks. Recently we have seen that ECS is terminating a large chunk of tasks at a time (often ~50% of the tasks) and then our service drops requests as we don't have the capacity to handle the inbound requests. I have noticed, at least on the most recent occurrence, that we had a spike in traffic of about 40% of our current traffic around the time that ECS terminated a bunch of tasks, however, the capacity should be there in our API to handle this without any issue. This issue has happened ~5 times in the past week or so but is very intermittent and doesn't seem to affect the entire service - only certain tasks. I have checked all of our monitoring and logging and I can't see anything as to why the health check would be failing. The application logs for the tasks are completely normal. All I have to go on are the following messages in the ECS event log: ``` service my-service (port 8000) is unhealthy in target-group my-target-group due to (reason Request timed out). ``` Is there any further troubleshooting I can do to understand what is causing this? Also, if the issue is somehow triggered by an increase in load, is there a way we can prevent the ECS service from immediately terminating the tasks (which inevitably compounds the issue).
0
answers
0
votes
39
views
mefs
asked 3 months ago

[Launch Announcement] New Gateway Load Balancer enhancement provides options to define a flow using 2-, 3- or 5- tuple

Amazon Web Services (AWS) Gateway Load Balancer (GWLB) is a new member of Elastic Load Balancing (ELB) product suite to help you easily deploy, scale, and manage your third-party virtual appliances. GWLB now supports configurable flow stickiness, enabling you to configure the hashing used to maintain stickiness of flows to a specific target appliance. You can modify the target group of your GWLB to maintain stickiness of flows using 3-tuple (source IP, destination IP, transport protocol) or 2-tuple (source IP, destination IP) in addition to the default method of 5-tuple. The configuration applies to all traffic using the target group. The configurable flow stickiness is intended for customers who need to support applications, such as IDS/IPS that identify flows using 3- or 2-tuple, or applications, such as FTP, Microsoft RDP, Windows RPC, and SSL VPN that use separate streams or dynamic port numbers but require to map all traffic from the same client to the same target, using GWLB. The configuration applies to all traffic using the same target group and requires you to drain existing flows to avoid disruption. Configurable flow stickiness doesn’t work if you are using GWLB with transit gateway (TGW) and TGW Appliance Mode is enabled. Please visit Gateway Load Balancer Documentation (https://docs.aws.amazon.com/elasticloadbalancing/latest/gateway/target-groups.html#flow-stickiness) to learn more.
0
answers
3
votes
394
views
asked 3 months ago

Static domain for API Gateway connected to ELB and ECS with Apollo Client

## Prerequisites Currently we have deployed API Gateway that is connected to ECS instance with Elastic Load Balancer. API Gateway is functioning correctly and we can invoke the instance with the generated invoke URL: Note: ECS instance is running Apollo Client for Graph QL access, so we access the client using ``` {invoke_url}/graphql ``` and this works as expected. ![Enter image description here](/media/postImages/original/IM6u1TLNmrSF-BQphWxNyMsQ) *** ## Setting up the custom domain: We follow the instructions provided in https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-custom-domains.html **Step-by-step** 1. We have already purchased a domain which we use for our services + SSL certificate. 2. We want to use subdomain.domain.com to direct to the API Gateway 3. We go to the API Gateway and setup a custom domain: ![Enter image description here](/media/postImages/original/IMXlJTri-USeOXsQ0AQYrLfA) 4. Next, we create an A Record alias that points to the generated API Gateway domain name that was created in the step 3. ![Enter image description here](/media/postImages/original/IMbHYRx4ujS_qiwsFELGr9Zw) 5. We setup the API Mapping to point to the existing API Gateway API: ![Enter image description here](/media/postImages/original/IM7WYW6pkQS5yca00hs-KlEg) All done? This is where we get stuck. According to the documentation, this should be enough but what we receive from the **API Gateway domain generated in step 3** ![Enter image description here](/media/postImages/original/IMJOBwVz4GRwCp36D0bixd9A) **subdomain.domain.com which was set in Route53 to point to the API Gateway** ![Enter image description here](/media/postImages/original/IMxhO1TxqLRNuQsvytlH5ZEQ) *** # The question In order to achieve the desired result of being able to access the API Gateway using custom domain: What did we do wrong and what to do to get it working? Can provide extra information if needed.
0
answers
0
votes
25
views
asked 3 months ago