Skip to content

Amazon SSM agent service not starting

1

Hi, I have a few Windows instances that are having issues with te Amazon SSM Agent service.
The event log is showing the following:
The Amazon SSM Agent service terminated with service-specific error Incorrect function

error log is also showing:
ERROR [HandleAwsError @ awserr.go.48] [instanceID=my_instance_id] [MessageProcessor] error when calling AWS APIs. error details - GetMessages Error: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2017-08-12 22:50:05 ERROR [HandleAwsError @ awserr.go.48] [instanceID=my_instance_id] [HealthCheck] error when calling AWS APIs. error details - NoCredentialProviders: no valid providers in chain. Deprecated.

Please help, thanks!

asked 8 years ago11.8K views
7 Answers
0

Hello,

The SSM agent error indicates that it is not able to find the credential to connect and authenticate with SSM APIs endpoints.

On your instance make sure that you have an IAM Role attached or the Credential file (for AWS CLI) configured properly with necessary IAM policy. You may take a look at our below public document about configuring SSM agent:
http://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-agent.html

Regards,
Santosh K

answered 8 years ago
0

Hi.

I have the same issue but the event viewer show " incorrect Function" thats the only log i have.

regards!

answered 8 years ago
0

also the error log shows...

ERROR error occurred when starting core manager: Failed to fetch region. Data from vault is empty. Get http://169.254.169.254/latest/dynamic/instance-identity/document: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

Please any suggestion?

answered 8 years ago
0

Hi everyone,

My problemm was that i can not fetch metadata.
and the solution was EC2 Launch.

Here the solution:
http://docs.aws.amazon.com/es_es/AWSEC2/latest/WindowsGuide/ec2launch.html

regards!

answered 8 years ago
0

This forum is old, but I wanted to provide the reason for my failure in case anyone stumbles on this problem. I was also getting the metadata failure. I noticed I couldn't get to http://169.254.169.254/latest/meta-data/instance-id in a browser either. I had NMAP installed which installs a loopback network device. This network device was set with IP 169.254.x.x with a subnet of 255.255.0.0. So, all requests to 169.254.169.254 were being sent to that loopback device and not to AWS. Disabling the loopback adapter fixed it.

answered 6 years ago
0

I had the same issue on my hyper-v instance. To get the amazon ssm agent working again I first disabled my hyper-v switch for testing. The SSM agent started right away. Next I enabled my hyper-v switch and ran the commands here for adding the metadata routes - https://repost.aws/knowledge-center/waiting-for-metadata

Prior to doing anything I ran "Route print" from a command prompt. The metadata routes were there under persistent routes, but they were not in the "active routes" table. After adding the routes via the above commands my ssm agent service was able to start successfully

AWS
answered 3 years ago
0

Hi,

I know by this time you have fixed the issue. But the error you have indicated shows the instance is unable to fetch the metadata or the IAM role is assigned is invalid.

Systems Manager Prerequisites:

As you aware, In order for a instance to properly report to systems manager, The instance should satisfy the below prerequisites

  1. SSM agent running
  2. Proper IAM role attached
  3. Configure your instances to allow HTTPS (port 443) outbound traffic on the security group
  4. Verify the instance can reach the Instance Metadata Service
  5. Check the instance can access the systems manager endpoints using IGW/NAT/VGW or VPC endpoints

Please login to the instance using RDP and perform the below test to identify the issue

SSM initial troubleshooting steps:

Verify connectivity to Systems Manager endpoints on port 443

Login to the instance -> Start menu -> Powershell -> run the below commands Note: change the region to your region (us-east-1)

Test-NetConnection ssm.us-east-1.amazonaws.com -port 443 Test-NetConnection ssmmessages.us-east-1.amazonaws.com -port 443

Verify metadata routes on the instance

Open browser and type http://169.254.169.254/latest/meta-data/

or

Open Powershell

Test-NetConnection 169.254.169.254 -port 80

Verify connectivity to the instance metadata service

Login to the instance -> Start menu -> Powershell -> run the below command

route print

(Note: The output should point your Instance Gateway)

Adding Metadata Route

In case the route are not correct for metadata. please execute the below script to make the route correct:

The below one works for EC2launch V1

Start->PowerShell->Run as Administrator

Import-Module “C:\ProgramData\Amazon\EC2-Windows\Launch\Module\Ec2Launch.psd1”

Add-Routes

Restart the SSM agent or Reboot instance and verify the instance able to reach SSM service

Verify the time is synced with NTP server (Amazon Time Sync Service)

Start-> cmd-> Run as Administrator

w32tm /query /configuration

check the following value under the TimeProviders "NtpServer: 169.254.169.123,0x9 (Local)"

if it is not set with the value, then execute the below command

w32tm /config /manualpeerlist:169.254.169.123 /syncfromflags:manual /update

Next, Restart the SSM agent and verified the instance under fleet manager.

Complete Diagnostics:

Start->PowerShell->Run as Administrator

C:\Program Files\Amazon\SSM

.\ssm-cli.exe get-diagnostics --output table

SSM Logs:

If the issue persists after validating the above steps then, Please investigate the below logs:

%PROGRAMDATA%\Amazon\SSM\Logs\amazon-ssm-agent.log

%PROGRAMDATA%\Amazon\SSM\Logs\errors.log

Please note that the above folder is hidden by default, so you will need to either enable Hidden Items to be viewed or enter the path manually.

Please find the prerequisites and troubleshooting document links for your reference [1] - [5]

References:

[1] Configure SSM Agent to use a proxy for Windows Server instances - https://docs.aws.amazon.com/systems-manager/latest/userguide/configure-proxy-ssm-agent-windows.html

[2] Systems Manager prerequisites - https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-prereqs.html

[3] Troubleshooting SSM - https://repost.aws/knowledge-center/systems-manager-ec2-instance-not-appear

[4] Troubleshooting managed node availability using <code class="code">ssm-cli</code> - https://docs.aws.amazon.com/systems-manager/latest/userguide/ssm-cli.html

[5] Troubleshooting using logs - https://repost.aws/knowledge-center/ssm-agent-logs

AWS
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.