How do I troubleshoot issues when I configure SSM Agent to use a proxy for managing my Amazon EC2 instance?

Short description

You can use Systems Manager to manage your Amazon EC2 instances. However, if the instances are behind a proxy, then you must configure SSM Agent to communicate with the Amazon endpoints through your proxy. If SSM Agent isn’t correctly configured, you receive an error message similar to the following:

Post https://ssm.RegionID.amazonaws.com/: proxyconnect tcp: dial tcp xxx.xxx.xxx.xxx:yyyy: i/o timeout

Resolution

To troubleshoot proxy issues, follow these steps:

Verify that the EC2 instance can access your proxy server

EC2 Windows instances

You can use the following Windows PowerShell command to verify connectivity to your proxy. In the following example, replace hostname with your proxy hostname, and replace port with your proxy port.

> Test-NetConnection hostname -port port

Expected output:

ComputerName     : hostname
RemoteAddress    : xxx.xxx.xxx.xxx
RemotePort       : port
InterfaceAlias   : Ethernet
SourceAddress    : YYY.YYY.YYY.YYY
TcpTestSucceeded : True
PS C:\Windows\system32>

EC2 Linux instances

You can use the following telnet or netcat commands to verify connectivity to your proxy. In the following examples, replace hostname with your proxy hostname, and replace port with your proxy port.

Telnet

$ telnet hostname port

Expected output:

Trying xxx.xxx.xxx.xxx...
Connected to hostname.
Escape character is '^]'.

Netcat

$ nc -vz hostname port'

Expected output:

Ncat: Version 7.50 ( https://nmap.org/ncat )
Ncat: Connected to xxx.xxx.xxx.xxx:YYYY.
Ncat: 0 bytes sent, 0 bytes received in 0.01 seconds.

Note: Netcat doesn't come preinstalled on Amazon EC2 instances. To manually install Netcat, see Ncat on the Nmap website.

Verify that your EC2 instance can reach the Systems Manager endpoint through the proxy server

Run the following command to confirm that your instance can reach the Systems Manager endpoint through your proxy server. Receiving an http error code 4xx indicates that you can reach the endpoint through your proxy server.

Note: In the following command examples, replace RegionID with your AWS Region, replace hostname with your proxy hostname, and replace port with your proxy port.

EC2 Windows instances

> (invoke-webrequest https://ssm.RegionID.amazonaws.com -DisableKeepAlive -UseBasicParsing -Method head -Proxy http://hostname:port )

Expected output:

invoke-webrequest : The remote server returned an error: (404) Not Found.

EC2 Linux instances

$  curl -k --proxy http://hostname:port -m 5 -s -o /dev/null -w "%{http_code}" https://ssm.RegionID.amazonaws.com

Expected output:

404

Verify that SSM Agent is configured to use your proxy information

EC2 Windows instances

You must review the SSM Agent log to verify that the proxy settings are applied as shown in the following example. Enter the following command:

> type C:\ProgramData\Amazon\SSM\Logs\amazon-ssm-agent.log | findstr -i "proxy"

Expected output:

2021-03-18 19:06:16 INFO Getting IE proxy configuration for current user: The operation completed successfully.
2021-03-18 19:06:16 INFO Getting WinHTTP proxy default configuration: The operation completed successfully.
2021-03-18 19:06:16 INFO Proxy environment variables:
2021-03-18 19:06:16 INFO http_proxy: hostname:port
2021-03-18 19:06:16 INFO https_proxy:
2021-03-18 19:06:16 INFO no_proxy: 169.254.169.254

For more information, see Configure SSM Agent to use a proxy for Windows Server instances.

EC2 Linux instances

First, run the following command to verify whether SSM Agent currently uses the required proxy variables by checking the process environment variable:

sudo cat /proc/$(pidof amazon-ssm-agent)/environ | xargs -0 -L1 -I{} echo {}

Expected output:

[root@host123~]# sudo cat /proc/$(pidof amazon-ssm-agent)/environ | xargs -0 -L1 -I{} echo {}
LANG=en_US.UTF-8
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
http_proxy=http://Hostname:Port
https_proxy=http://Hostname:Port
no_proxy=169.254.169.254

Then, if the output does not include the proxy variables, follow the guidance below for your instance type to verify and update the relevant files.

EC2 Linux instances based on an upstart such as Ubuntu 14.04

Verify that the amazon-ssm-agent.override file contains the required configurations. Enter the following command and confirm that your output matches the example expected output. For more information, see Configure SSM Agent to use a proxy (upstart).

Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.

$ cat  /etc/init/amazon-ssm-agent.override

Expected output for an HTTP proxy server:

env http_proxy=http://hostname:port
env https_proxy=http://hostname:port
env no_proxy=169.254.169.254

Expected output for an HTTPS proxy server:

env http_proxy=http://hostname:port
env https_proxy=https://hostname:port
env no_proxy=169.254.169.254

EC2 Linux instances based on Ubuntu 16.04 or later with SSM Agent installed using a snap

Enter the following command and confirm that the proxy information matches the example expected output. For more information, see Configure SSM Agent to use a proxy (systemd).

Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.

$ cat /etc/systemd/system/snap.amazon-ssm-agent.amazon-ssm-agent.service.d/override.conf

Expected output for an HTTP proxy server:

[Service]
Environment="http_proxy=http://hostname:port"
Environment="https_proxy=http://hostname:port"
Environment="no_proxy=169.254.169.254"

Expected output for an HTTPS proxy server:

[Service]
Environment="http_proxy=http://hostname:port"
Environment="https_proxy=https://hostname:port"
Environment="no_proxy=169.254.169.254"

EC2 Linux instances based on Amazon Linux 2

Enter the following command and confirm that the proxy information matches the example expected output. For more information, see Configure SSM Agent to use a proxy (systemd).

Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.

$ cat /etc/systemd/system/amazon-ssm-agent.service.d/override.conf

Expected output for an HTTP proxy server:

[Service]
Environment="http_proxy=http://hostname:port"
Environment="https_proxy=http://hostname:port"
Environment="no_proxy=169.254.169.254"

Expected output for an HTTPS proxy server:

[Service]
Environment="http_proxy=http://hostname:port"
Environment="https_proxy=https://hostname:port"
Environment="no_proxy=169.254.169.254"

EC2 Linux instances based on other operating systems

Enter the following command and confirm that the proxy information matches the example expected output. For more information, see Configure SSM Agent to use a proxy (systemd).

Important: If you update the amazon-ssm-agent.override file, then be sure to restart SSM Agent after the file is edited.

$ cat /etc/systemd/system/amazon-ssm-agent.service.d/amazon-ssm-agent.override

Expected output for an HTTP proxy server:

[Service]
Environment="http_proxy=http://hostname:port"
Environment="https_proxy=http://hostname:port"
Environment="no_proxy=169.254.169.254"

Expected output for an HTTPS proxy server:

[Service]
Environment="http_proxy=http://hostname:port"
Environment="https_proxy=https://hostname:port"
Environment="no_proxy=169.254.169.254"

---

Related information

Working with SSM Agent

Troubleshooting SSM Agent