Elastic beanstalk event status issue: Ok to Severe, Severe to shutdown

0

I have a webapp running on Elastic beanstalk. (Platform - Tomcat 8.5 with Corretto 11 running on 64bit Amazon Linux 2/4.3.7)

At beginning it's works fine. But after a few days, I started to get some error event notifications like below:

May 28, 2023 17:04:13 (UTC+8)	INFO	Environment health has transitioned from Severe to Ok.
May 28, 2023 17:03:13 (UTC+8)	WARN	Environment health has transitioned from Ok to Severe. 100.0 % of the requests are erroring with HTTP 4xx.
May 28, 2023 19:41:28 (UTC+8)	INFO	Environment health has transitioned from Severe to Ok.
May 28, 2023 19:39:28 (UTC+8)	WARN	Environment health has transitioned from Ok to Severe. 100.0 % of the requests are erroring with HTTP 4xx.
...

Some times, the server health could recover from a ‘Severe’ to ‘Ok’, but sometimes it cannot recover to 'Ok' and turns to shutdown.

I checked the server backend logs(/var/log/nginx/access.log), I belived that my webapp has been attacked.

The attacker send lots of bad request during a period of time, to make my web server fail to respond properly. The logs as below:

128.199.16.76 - - [28/May/2023:17:02:22 +0000] "GET /db/webdb/index.php?lang=en HTTP/1.1" 404 777 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:22 +0000] "GET /mysqlmanager/index.php?lang=en HTTP/1.1" 404 777 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:23 +0000] "GET /administrator/db/index.php?lang=en HTTP/1.1" 404 785 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:23 +0000] "GET /sql/websql/index.php?lang=en HTTP/1.1" 404 779 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:23 +0000] "GET /admin/web/index.php?lang=en HTTP/1.1" 404 778 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:23 +0000] "GET /database/index.php?lang=en HTTP/1.1" 404 773 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:24 +0000] "GET /phppma/index.php?lang=en HTTP/1.1" 404 771 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:24 +0000] "GET /phpMyAdmin2/index.php?lang=en HTTP/1.1" 404 776 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:24 +0000] "GET /administrator/pma/index.php?lang=en HTTP/1.1" 404 786 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:24 +0000] "GET /php-my-admin/index.php?lang=en HTTP/1.1" 404 777 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:25 +0000] "GET /phpmyadmin2022/index.php?lang=en HTTP/1.1" 404 779 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:25 +0000] "GET /db/phpmyadmin4/index.php?lang=en HTTP/1.1" 404 783 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:25 +0000] "GET /mysql/pma/index.php?lang=en HTTP/1.1" 404 778 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"
128.199.16.76 - - [28/May/2023:17:02:25 +0000] "GET /index.php?lang=en HTTP/1.1" 404 760 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36" "-"

...

I think at that moment my server is still alive but the EC2 heath check found that in one minute all the request were responded as 404, so AWS set my server into 'Severe'.

What can I do on ElasticBeanstalk to make my webApp not go fail?

May I change EC2 heath check rule ? Or dose AWS support any service to protect the webApp like firewall?

asked 10 months ago229 views
1 Answer
0

How about deploying AWS WAF to protect your web applications?
AWS WAF can be configured on ALB or CloudFront and can be used to prevent attacks on web applications.
Also, AWS WAF can be configured with rate-based rules, so it is possible to have it deal with attacks such as DDoS.
https://docs.aws.amazon.com/waf/latest/developerguide/getting-started.html

If you want to specialize in DDoS countermeasures, you can also consider deploying AWS Shield Advanced as a countermeasure.
https://docs.aws.amazon.com/waf/latest/developerguide/ddos-advanced-summary.html

It would be better to start with AWS WAF, which can be easily configured.

profile picture
EXPERT
answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions