balancing web traffic on internal ec2 instance

To allow the Application Load Balancer (ALB) to perform health checks on EC2 instances using their private IP addresses, you need to ensure several configurations are in place:

1. VPC Configuration: Ensure that the ALB and EC2 instances are in the same VPC and that the VPC has the necessary routing configured.

2. Subnet Configuration: Place the ALB and EC2 instances in the same subnets or in subnets that can communicate with each other within the VPC.

3. Security Groups:

   • The security group for the EC2 instances should allow inbound traffic from the ALB on the health check port (typically HTTP port 80 or HTTPS port 443).
   • The security group for the ALB should allow outbound traffic to the EC2 instances on the health check port.

4. Network ACLs: Ensure that the Network ACLs associated with the subnets allow the necessary traffic between the ALB and the EC2 instances.

5. EC2 Instance Configuration:

   • Disable auto-assign public IP for the EC2 instances.
   • Ensure that the web server (Flask app in this case) is configured to listen on the private IP address.

6. Load Balancer Configuration:

   • Configure the target group to use IP addresses as targets instead of instance IDs.
   • Specify the private IP addresses of the EC2 instances when registering targets.

7. Health Check Configuration:

   • In the target group settings, configure the health check to use the appropriate path and port that your Flask app is serving.

8. EC2 Instance Metadata: The Flask app's use of the instance metadata service (169.254.169.254) should work fine as it's a link-local address and doesn't require public IP addressing.

If you've configured all these correctly and the health checks are still failing, double-check that your Flask application is running and accessible on the private IP address of the EC2 instances. You may need to modify the Flask app to bind to the private IP or 0.0.0.0 instead of localhost.

Remember, the ALB doesn't need public IP addresses on the EC2 instances to perform health checks or route traffic. It can communicate with the instances using their private IP addresses within the VPC.
Sources
Use Elastic Load Balancing to distribute incoming application traffic in your Auto Scaling group - Amazon EC2 Auto Scaling
Add a Load Balancer - AWS OpsWorks
Registered instances for your Classic Load Balancer - Elastic Load Balancing
Amazon EC2 instance IP addressing - Amazon Elastic Compute Cloud

solved it was a code issue, in one of the templates they sourced the css and js framework as part of the answer, by removing the public address they weren't no more available because I did not attach a nat gateway to the vpc.. I suppose that the flask api went into timeout to not be able to compose the answer, testing a simple answer it works fine with the private address (just with the instance in the target group, no need to balance directly ip addresses ).. moreover the deployment of the cloudinit script didn't succeed for the same reason and initially the check failed rightfully.. I had to setup a connect endpoint in order to get what's going on and install manually the app (enabling temporarily the public address)

here's the test

@application.route("/hello")
def test():
    return 'Hello, World!' 

here's the answer :D

FlaskApp]# FLASK_APP=application.py /usr/local/bin/flask run --host=0.0.0.0 --port=80
 * Serving Flask app 'application.py'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:80
 * Running on http://*.*.*:*:80
Press CTRL+C to quit
*.*.*.* - - [25/Dec/2024 23:00:05] "GET /hello HTTP/1.1" 200 -

I spent almost 10 cents of spot instances to figure it out :( I need to learn to debug the flask apps better