- Newest
- Most votes
- Most comments
In addition to the re:Post Agent answer ... while the general network checks are correct, the error unable to acquire credentials on AL2023 often points to two specific issues:
IMDSv2 Hop Limit: AL2023 requires IMDSv2 by default. If your instance has a complex network stack or runs containers, the default "Hop Limit" of 1 might prevent the SSM Agent from reaching the metadata service to get its IAM credentials.
- Fix: Increase the Metadata response hop limit to 2 in the Instance Metadata Options.
DNS Resolution (Private VPCs): The error send request failed usually means the instance cannot resolve the SSM endpoint URL.
- Fix: Ensure "Enable DNS Support" and "Enable DNS Hostnames" are both set to True in your VPC settings, especially if you are using Interface VPC Endpoints.
Default Host Management: The log "Skipping default host management fallback" confirms that the agent is trying to connect but the entire network path to AWS is blocked. Double-check that your VPC Endpoints have "Private DNS names" enabled, otherwise, the agent will try (and fail) to reach the public internet.
PS: Since you mentioned that a reboot didn't help, have you checked if the instance can resolve any external domain names at all using the EC2 Serial Console?
The error messages you're seeing indicate that the SSM Agent on your EC2 instance is unable to establish a connection with the Systems Manager service. The "RequestError: send request failed" suggests network connectivity issues preventing the agent from communicating with Systems Manager endpoints.
Here are the key areas to investigate:
Network Connectivity Issues: The SSM Agent needs to connect to Systems Manager endpoints to function properly. Check for firewall restrictions, routing problems, or lack of internet connectivity that could be blocking the connection. If your instance is in a private subnet, ensure you have the proper VPC endpoints configured.
VPC Endpoint Configuration (for private subnets): If your instance is in a private subnet without internet access, you must configure VPC endpoints for Systems Manager. You need three specific endpoints:
- com.amazonaws.[region].ssm
- com.amazonaws.[region].ssmmessages
- com.amazonaws.[region].ec2messages
Ensure these endpoints have:
- Private DNS enabled
- Proper subnet associations in the same subnets as your instances
- Security groups that allow inbound HTTPS traffic (port 443) from your instances
- Your instance's security group allows outbound traffic on port 443
IAM Instance Profile: Verify that your EC2 instance has an IAM instance profile attached with the necessary permissions for Systems Manager. The error about "unable to acquire credentials" suggests the instance may be having trouble accessing its IAM role credentials or the role lacks required permissions.
Security Group Configuration: Confirm that your instance's security group allows outbound HTTPS traffic on port 443 to reach the Systems Manager endpoints, and that the VPC endpoint security groups allow inbound traffic from your instance.
Check the SSM Agent logs for more detailed error messages that can help pinpoint the exact connectivity issue.
Sources
Troubleshooting Session Manager - AWS Systems Manager
Resolve VPC endpoint Systems Manager connectivity | AWS re:Post
Relevant content
- asked a year ago
- asked 3 years ago
- asked 2 years ago
