AWS Abuse Report

Question

Hi All

I have recived report from aws regarding breached/brute attack from an  ip and they have blocked it 
Now i need to identify and remove the service which caused downtime

AWS Report :
If you're unaware of this activity, it's possible that your environment has been compromised by an external attacker, or a vulnerability is allowing your machine to be used in a way that it was not intended.
"payload_class": "exploit:gen/cve_2019_9082",
"payload_data":

issue faced :
i am getting unknow RAM consumption of 2.8 gb out of 4 gb ram 
free -h
              total        used        free      shared  buff/cache   available
Mem:           3.8G        2.8G        173M        1.5M        821M        767M
Swap:            0B          0B          0B

all servers are stopped but still facing the unknown ram

note apparmor is running and found some logs in journal 
journal log : 
Apr 20 05:47:17 ip-172-31-4-115 kernel: Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
Apr 20 05:47:17 ip-172-31-4-115 kernel: Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
Apr 20 05:47:17 ip-172-31-4-115 kernel: Spectre V1 : Mitigation: usercopy/swapgs barriers and __user pointer sanitization
Apr 20 05:47:17 ip-172-31-4-115 kernel: Spectre V2 : Mitigation: Retpolines
Apr 20 05:47:17 ip-172-31-4-115 kernel: Spectre V2 : Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch
Apr 20 05:47:17 ip-172-31-4-115 kernel: Spectre V2 : Spectre v2 / SpectreRSB : Filling RSB on VMEXIT
Apr 20 05:47:17 ip-172-31-4-115 kernel: RETBleed: WARNING: Spectre v2 mitigation leaves CPU vulnerable to RETBleed attacks, data leaksApr 20 05:47:17 ip-172-31-4-115 kernel: RETBleed: Vulnerable
Apr 20 05:47:17 ip-172-31-4-115 kernel: Speculative Store Bypass: Vulnerable
Apr 20 05:47:17 ip-172-31-4-115 kernel: MDS: Vulnerable: Clear CPU buffers attempted, no microcode
Apr 20 05:47:17 ip-172-31-4-115 kernel: MMIO Stale Data: Vulnerable: Clear CPU buffers attempted, no microcode
Apr 20 05:47:17 ip-172-31-4-115 kernel: Freeing SMP alternatives memory: 40K

Answer

1. **Identify Affected Instances**:
Begin by identifying which instances or services are exhibiting unusual behavior, particularly those that relate to the high RAM usage and the security incident reported by AWS.
2. **Review Security Groups and Network ACLs**:
Check your security groups and network ACLs for any rules that might allow unauthorized access. Tighten these configurations to limit access only to trusted IPs and essential ports.
3. **Audit Logs**:
Examine logs from your EC2 instances, VPC flow logs, and any application logs that might give insights into unauthorized access or unusual activities. Look for patterns or IP addresses that might be linked to the exploit.
CloudTrail logs can be particularly helpful for understanding API calls that have been made in your account, including those that started or stopped services.
4. **Memory and Process Analysis**:
Since you're observing high RAM usage despite stopping your services, it’s important to directly inspect the processes running on your system.
Connect to your instance using SSH and use tools like `top`, `htop`, or `ps aux` to check which processes are consuming excessive memory and are not recognized or expected.
5. **Check for Malware**:
The specific exploit mentioned (CVE-2019-9082) relates to vulnerabilities that could potentially allow remote code execution or other malicious activities.
Run a thorough scan using an updated anti-virus or a security tool like AWS Inspector to check for malware or other intrusions.
6. **Patch and Update**:
Ensure that all software on your instances is up-to-date, particularly focusing on patches for known vulnerabilities (like CVE-2019-9082).
Update your operating system, applications, and any dependencies to their latest secure versions.
7. **Isolate Affected Instances**:
If possible, isolate affected instances to prevent potential spread or further exploitation. This could involve network isolation or temporarily taking the instance offline.
8. **Forensic Analysis**:
If you have snapshots or backups, consider mounting them in an isolated environment for a detailed analysis. This might help in understanding what was compromised without risking further damage to your production environment.
9. **Recovery and Rebuild**:
For compromised instances, it might be safer to rebuild them rather than trying to clean them up. This can help ensure that no traces of the attacker remain.
Restore from known good backups if necessary and apply strict security measures before reconnecting them to the network.
10. **Notify AWS Support**:
Inform AWS of the steps you've taken and any findings. This can be important both for compliance and for potential assistance from AWS.
11. **Enhance Monitoring and Security Posture**:
Going forward, enhance your monitoring capabilities using AWS CloudWatch and other security tools.
Regularly review your security policies and practices to prevent similar issues.

AWS Abuse Report

Relevant content