How do I troubleshoot EC2 agent-based and CIS scan issues in Amazon Inspector?

Short description

Amazon Inspector EC2 agent-based scans and CIS scans require connectivity to various AWS service endpoints. For private EC2 instances that don't have internet access through an internet gateway or NAT gateway, you must configure virtual private cloud (VPC) endpoints to provide this connectivity. Most scan failures occur when the Amazon Simple Storage Service (Amazon S3) VPC endpoint is missing or when you misconfigured the endpoint policy.

If you see one of the following statuses when you run EC2 agent-based scans or CIS scans, then complete the following steps in this article:

• "Internal error"
• "No inventory"
• "Pending initial scan"
• "Stale inventory"
• "Unmanaged EC2 instance"
• "Actively monitoring with partial errors"
• "CANCELLED"
• "TIMED_OUT"

Resolution

Check the instance and IAM role configuration

Verify that Amazon Inspector supports the instance's operating system and that AWS Systems Manager manages the instance. For CIS scans, the instance's AWS Identity and Access Management (IAM) role must also have the AmazonInspector2ManagedCisPolicy policy attached.

To verify that Systems Manager manages your instance, complete the following steps:

1. Open the Systems Manager console.
2. In the navigation pane, choose Fleet Manager, a capability of AWS Systems Manager.
3. Choose Managed nodes, and then check that the instance appears in the list.
   Note: If instances appear under Managed nodes, then this confirms that Systems Manager manages the instance.

If your instance appears in the list, then check that the SSM Agent version is up-to-date in Fleet Manager.

If the instance doesn't appear in the list, then attach the AmazonSSMManagedInstanceCore policy to the instance's IAM role. For instructions, see Attach the Systems Manager instance profile to an instance (console). After you attach the IAM policy, follow the instructions in the Check VPC endpoints and policies section.

Note: Amazon Inspector requires the AmazonInspector2ManagedCisPolicy policy for only CIS scans. Also attach this policy to the instance's IAM role when you run CIS scans.

To manage instances with Systems Manager, configure Systems Manager to use a VPC interface endpoint on AWS PrivateLink.

Check VPC endpoints and policies

Complete the following steps:

1. Open the Amazon Virtual Private Cloud (Amazon VPC) console.

2. In the navigation pane, choose Endpoints.

3. Confirm that you configured the following endpoints in the instance's VPC:
   com.amazonaws.REGION.ssm
   com.amazonaws.REGION.ssmmessages
   com.amazonaws.REGION.ec2messages
   com.amazonaws.REGION.s3
   com.amazonaws.REGION.inspector2
   Note: Replace REGION with your AWS Region. Amazon Inspector requires com.amazonaws.region.inspector2 only for CIS scans.

4. On the Endpoints page, choose Policy.

5. Confirm that the Amazon S3 endpoint policy allows s3:GetObject on the following S3 buckets:
   arn:aws:s3:::aws-ssm-document-attachments-REGION/*
   arn:aws:s3:::inspector2-oval-prod-REGION/*
   arn:aws:s3:::cis-datasets-prod-CisBucketId/*
   Example S3 endpoint policy:

   {
           "Sid": "AllowSSMS3BucketGetObject",
           "Effect": "Allow",
           "Principal": "*",
           "Action": "s3:GetObject",
           "Resource": [
               "arn:aws:s3:::aws-ssm-document-attachments-REGION/*",
               "arn:aws:s3:::inspector2-oval-prod-REGION/*",
               "arn:aws:s3:::cis-datasets-prod-CisBucketId/*"
           ]
       }

   Note: Replace REGION with your Region and CisBucketId with the CIS bucket ID for your Region. Amazon Inspector requires the arn:aws:s3:::inspector2-oval-prod-REGION/* bucket only for Windows instances. Amazon Inspector requires the arn:aws:s3:::cis-datasets-prod-CisBucketId/* bucket for only CIS scans.

Check the network configuration

Confirm that the security group that's attached to the instance and the network access control list (network ACL) doesn't block outbound HTTPS traffic on port 443.

If you use a gateway endpoint for S3, then specify the route to the S3 VPC endpoint in the route table.

If you use an interface endpoint for S3 with Amazon-provided DNS, then verify that the Private DNS names enabled attribute is set to Yes in the VPC endpoint settings. To confirm this, select the endpoint on the Endpoints page.

If you use custom name resolution methods, then confirm that the DNS or proxy settings return the private IP address for the VPC endpoint. Confirm that the IP address returned by the curl or Test-NetConnection command matches the private IP address of the VPC endpoint.

Check for other scan blockers

Take the following actions:

Confirm that the Amazon Inspector Systems Manager (SSM) plugin is installed at the following path on the instance:

For Linux, use /opt/aws/inspector/bin/inspectorssmplugin.

For Windows, use C:\Program Files\Amazon\Inspector\inspectorssmplugin.exe.

Note: The Amazon Inspector SSM plugin installs when the instance runs. If you stop the instance frequently, then the plugin might not install. Start the instance and let it run for a period of time to resolve this issue.

Check that the instance doesn't have the InspectorEc2Exclusion tag. This tag excludes the instance from Amazon Inspector EC2 scans. For more information, see Excluding instances from Amazon Inspector scans.

Check that the service control policy (SCP) doesn't restrict the Systems Manager actions that's required for EC2 scans. These actions are allowed in the AmazonSSMManagedInstanceCore policy.

Gather required information for additional support

Note: If you receive errors when you run AWS Command Line Interface (AWS CLI) commands, then see Troubleshooting errors for the AWS CLI. Also, make sure that you're using the most recent AWS CLI version.

If you can't resolve the issue, then contact AWS Support and provide the following information in your support case:

• Instance ID

• SSM Agent log and Amazon Inspector log files For Linux, the log files are in the /var/log/amazon/ssm and /var/log/amazon/inspector directories. For Windows, the log files are in the %ProgramData%\Amazon\SSM\Logs\ and %ProgramData%\Amazon\Inspector\ directories.

• For CIS scans, run the following AWS CLI commands to provide information about the scan details and scan configuration.
  To get the date and status of the CIS scan, run the following list-cis-scans command:

  aws inspector2 list-cis-scans

  To get the target details and confirm if the tag matches, run the following list-cis-scan-configurations command:

  aws inspector2 list-cis-scan-configurations

• Run the following Test-NetConnection and ssm-cli commands to provide the connectivity test results for each endpoint.
  Note: Replace REGION with your Region. In the following connectivity test commands, replace ENDPOINT_DNS_NAME with a DNS name from the following list. The inspector2.REGION.amazonaws.com DNS name is required only for CIS scans.
  ssm.REGION.amazonaws.com
  ssmmessages.REGION.amazonaws.com
  ec2messages.REGION.amazonaws.com
  s3.REGION.amazonaws.com
  inspector2.REGION.amazonaws.com
  To get the connectivity test results on Linux, run the following curl command:

  curl -v https://ENDPOINT_DNS_NAME

  To collect SSM Agent diagnostics on Linux, run the following ssm-cli command:

  ssm-cli get-diagnostics --output table

  To get the connectivity test results on Windows, run the following Test-NetConnection command:

  Test-NetConnection ENDPOINT_DNS_NAME -port 443

  To collect SSM Agent diagnostics on Windows, run the following ssm-cli command:

  & "C:\Program Files\Amazon\SSM\ssm-cli.exe" get-diagnostics --output table

Related information

Why doesn't Amazon Inspector scan my Amazon EC2 instances?

How do I create Amazon VPC endpoints so that I can use Systems Manager to manage private Amazon EC2 instances without internet access?

Scanning Amazon EC2 instances with Amazon Inspector

Center for Internet Security (CIS) scans for Amazon EC2 instance operating systems