Why does my Amazon EC2 instance appear as non-compliant in the Compliance dashboard?

Short description

The following factors affect the compliance status of a managed EC2 instance in the Compliance dashboard:

• The status of patches in Patch Manager, a capability of AWS Systems Manager.
• The status of associations in State Manager, a capability of AWS Systems Manager.
• Issues with the AWS-GatherSoftwareInventory document.

Resolution

To determine the compliance status of an instance, view the compliance report, and then check the Compliance type for each non-compliant resource. The Patch compliance type means that there are patching issues, and Association means that there are issues with the State Manager associations.

Troubleshoot non-compliance based on patching

Note: Make sure that your servers can reach the repositories with the patches to download the updates. Patch Manager doesn't provide the patches. Instead, Patch Manager orchestrates patches with a mechanism for each operating system (OS) to install updates on an instance. For example, Patch Manager uses Windows Update to install patches on instances that run Windows.

To identify the missing patches, complete the following steps:

1. View the compliance report.
2. For Group dashboard results based on, select Compliance type.
3. Under Compliance resources summary, for Patch, choose the value for Non-compliant resources.
4. Under Compliance rule, check the value for ID to identify the missing patch.

Then, complete the following steps to troubleshoot issues with the missing patches.

Resolve issues where the patch document didn't run on the instance

If the patching operation didn't install missing updates on non-compliant resources, then you must install the patch.

To install the patch, take one of the following actions:

• Run Patch now so that Patch Manager scans and installs the missing patches.
• Use Run Command, capability of AWS Systems Manager, to immediately run the AWS-RunPatchBaseline document.
• Use Maintenance Window, a capability of AWS Systems Manager, to schedule a maintenance window for your instance. Then, configure the maintenance window to run the AWS-RunPatchBaseline document with Run Command.

Note: By default, AWS-RunPatchBaseline approves patches 7 days after they're released. You can create a custom patch baseline for Windows, macOS, or Linux.

Check whether the patch document installed all the approved patches

Complete the following steps to identify why the patch document didn't install specific approved packages:

1. Use SSH or Session Manager, a capability of AWS Systems Manager, to connect to your instance.
2. Check the following AWS Systems Manager Agent (SSM Agent) logs based on your OS.
   Linux or macOS instances:
   /var/log/amazon/ssm/amazon-ssm-agent.log
   /var/lib/amazon/ssm/InstanceID/document/orchestration/CommandID
   Windows instances:
   %PROGRAMDATA%\Amazon\SSM\Logs\amazon-ssm-agent.log
   %PROGRAMDATA%\Amazon\PatchBaselineOperations
   Note: For Windows instances, look for a log file that's named Install-PatchBaselineOperation-date.
3. Troubleshoot errors that you find in the logs.

Check whether you must reboot the instance

Note: To check the state of patches, use Fleet Manager, a capability of AWS Systems Manager.

To finalize patch installation, you must reboot your instance. If you set the RebootOption parameter in the AWS-RunPatchBaseline document to NoReboot, then the instance doesn't automatically reboot after patch installation.

To troubleshoot this issue, complete the following steps to manually reboot the instance:

1. Open the Systems Manager console.
2. Choose Fleet Manager, and then select your instance.
3. Under General, choose Patches.
4. In the search bar, enter State = InstalledPendingReboot to identify rejected patches.
   Note: Instances with the InstalledPendingReboot status are non-compliant until you reboot and scan the instance.
5. Reboot the instance.
6. Scan the instance.
7. Verify that the instance appears as Compliant in the compliance report.

Check for rejected patches on the instance

If the instance has a patch installed that's on the list of rejected patches, then the instance is listed as non-compliant. This issue typically occurs because you or the patch document installed the patch before Systems Manager added the patch to the rejected list.

To identify and resolve this issue, complete the following steps:

1. Open the Systems Manager console.
2. Choose Fleet Manager, and then select your instance.
3. Under General, choose Patches.
4. In the search bar, enter State = InstallRejected to identify rejected patches.
5. Use SSH or Session Manager to connect to your instance.
6. Remove rejected patches from your OS.

Check whether the patch document failed

If Systems Manager initiated a patch document on an instance but the document didn't run successfully, then the instance remains non-compliant. To identify issues with State Manager or with the Patch Now operation, check the association history. To identify issues with Run Command, open the Systems Manager console, and then choose Run Command to check the Run Command history.

To troubleshoot issues, see Errors when running AWS-RunPatchBaseline on Linux or Errors when running AWS-RunPatchBaseline on Windows Server.

Check whether the instance is non-compliant because of a new feature

The instance might appear non-compliant even after the scan shows no missing updates. This issue typically occurs when you installed a patch on the instance that's not approved by the patch baseline yet. New features might have the AVAILABLE_SECURITY UPDATE status but still not appear as compliant.

If a patch isn't approved in the baseline, then you receive the following output after the scan:

"Scan found the following available security updates not approved by the baseline:KB5075040"

To resolve this issue, complete the following steps:

1. Open the Systems Manager console.
2. Choose Patch Manager, and then choose Patch baseline.
3. Select your patch baseline, and then choose Edit.
4. Under Patch baseline details, for Available security updates compliance status, select Compliant.
5. Choose Save.
6. Rerun the instance patch scan.

Troubleshoot non-compliance based on the status of State Manager associations

When you create a State Manager association, Systems Manager defines a configuration state for the instance. If State Manager runs a patching operation that fails, or the server is missing patches, then the association status is non-compliant.

To resolve this issue, complete the following steps:

1. View the compliance report.
2. For Group dashboard results based on, enter Compliance type.
3. Under Compliance resources summary, for Association, choose the value for Non-compliant resources.
4. Under Compliance rule, check the value for ID to identify the non-compliant association ID.
   Note: Instances might appear as non-compliant even if State Manager ran the association. This typically occurs when the association runs patching operations and the instance is missing patches. If the association is a patching association, then make sure that you ran the patching documents to install the latest patch updates. To check the association type, choose the association ID. After you install the updates, Patch Manager updates the association status to Compliant.
5. Check the association history for the non-compliant association to understand why it failed.
6. Troubleshoot issues for the non-compliant association.

Troubleshoot AWS-GatherSoftwareInventory document issues

If your instance is non-compliant because the AWS-GatherSoftwareInventory document didn't run, then the association becomes stuck in the Pending or Failed state. To resolve this issue, troubleshoot issues with Inventory, a capability of AWS Systems Manager.

Run the patching documents to install the latest patch updates

To update the patch compliance status of an instance, run one of the following patching documents:

• AWS-RunPatchBaseline
• AWS-RunPatchBaselineAssociation
• AWS-RunPatchBaselineWithHooks

You can configure patching with Patch Manager, or with Quick Setup, a capability of AWS Systems Manager.