Unable to use Session Manager on EC2 instances in a private subnet with SSM VPC endpoint

0

I am setting up an environment to mimic what customer wants to achieve. I have EC2 instances in a private subnet in a VPC. In order to use Session Manager on them, I created VPC endpoint to allow SSM communication. Those EC2 instances has instance profile with an IAM role granting managed policy " AmazonSSMManagedInstanceCore".

All the instances are showing up properly in Systems Manager. However, when I tried to start a session using Session Manager, when I select the instance, it shows the following error message:

The version of SSM Agent on the instance supports Session Manager, but the instance is not configured for use with AWS Systems Manager. Verify that the IAM instance profile attached to the instance includes the required permissions.

To compare and troubleshoot, I launched EC2 instances in a public subnet, using the same IAM role, they all working well with session manager. The ssm-agent version on those EC2 instances are 2.3.662.0 and 2.3.372.0, all supported for Session Manager. The only difference between working and non-working instances are the working ones are running from public subnet, while the non-working ones are running from private subnet with SSM VPC endpoint.

What could be wrong? Thanks

posta 5 anni fa6797 visualizzazioni
2 Risposte
1
Risposta accettata

Make sure that you have specified all VPC endpoint for SSM:

  • com.amazonaws.region.ssm: The endpoint for the Systems Manager service.
  • com.amazonaws.region.ec2messages: Systems Manager uses this endpoint to make calls from SSM Agent to the Systems Manager service.
  • com.amazonaws.region.ec2: If you're using Systems Manager to create VSS-enabled snapshots, you need to ensure that you have an endpoint to the EC2 service. Without the EC2 endpoint defined, a call to enumerate attached EBS volumes fails, which causes the Systems Manager command to fail. - com.amazonaws.region.ssmmessages: This endpoint is required only if you are connecting to your instances through a secure data channel using Session Manager. For more information, see AWS Systems Manager Session Manager.

Source: https://docs.aws.amazon.com/systems-manager/latest/userguide/setup-create-vpc.html#sysman-setting-up-vpc-create

con risposta 5 anni fa
profile picture
ESPERTO
verificato 2 mesi fa
profile picture
ESPERTO
verificato 4 mesi fa
  • Also, I'm still confused if a VPC endpoint is just like a wormhole between the VPC and AWS Services, which will avoid packets to and from the instance to travel over the Internet?

  • The documentation referenced is not clear enough. I still don't know which type of endpoint I need, in the 1st page of the creation wizard, among: AWS Services, EC2 Instance Connect Endpint, PrivateLink, and possibly others. Also, you'll note the black magic that consists in inverting the Service Name into a namespace to be "verified" with some types, not others. The comment above uses the namespace notation, which, in particular, is valid for PrivateLink type, but not only.

0

I followed all docos available under the sun: all possible SG to protect instance and/or VPC endpoint. It only worked once (Connect button was available, and I could open a session onto instance). Then I followed the advice to restrict the Source CIDR of VPC endpoint Inbound SG to priv subnet, (instead of entire VPC), and it failed with error: "SSM Agent is offline". When I rolled back SG to entire VPC, it never worked again...

The only way I could make it work is by adding a NAT Gwy. I anyway like NAT Gwy to keep my EC2 up to date in terms of patching level.

Conclusion : Total fiasco, and 6 hours wasted. NAT Gwy fixed it and allows decent security level of instance.

con risposta 3 mesi fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande