How do I manage and view the Systems Manager patch and association compliance data for all my accounts using QuickSight?

8 minute read
0

I want to use Amazon QuickSight to manage and view compliance data for AWS Systems Manager.

Resolution

With Amazon QuickSight, you can query, analyze, and visualize Systems Manager Inventory data. You can also publish interactive dashboards. You can use Amazon QuickSight with Amazon Athena table dataset to create dashboards and widgets for displaying compliance information.

Check the prerequisites

Set up Systems Manager Inventory and resource data sync in your account. This setup allows the following:

  • Systems Manager gathers the inventory information.
  • Resource data sync synchronizes the inventory to an Amazon Simple Storage Service (Amazon S3) bucket.

You can create this setup for multi-account multi-Region use cases and synchronize the data to a central S3 bucket. Amazon Athena integration uses resource data sync to view the inventory data from all managed nodes in the inventory data Detailed view page. For more information, see Querying inventory data from multiple Regions and accounts.

After setting up Systems Manager Inventory, resource data sync, and Athena access configuration, you can proceed to set up your QuickSight account.

Set up a QuickSight account

If you don't have an Amazon QuickSight account, log in to your AWS Management Console with the AWS Identity and Access Management (IAM) user or role that has appropriate QuickSight permissions. Go to Amazon QuickSight to create a new account.

  1. Choose Enterprise or Enterprise + Q.
    -or-
    Scroll down, and then choose Sign up for Standard Edition.
  2. Choose the appropriate IAM identity.
  3. Under Quicksight access to AWS services, select Amazon Athena and Amazon S3.
  4. For Select Amazon S3 buckets, select the target S3 bucket where inventory data is stored. Select Write permission for Athena Workgroup against the selected S3 bucket to give write permission for Athena Workgroup.
  5. Choose Finish.

If you have an existing QuickSight account, do the following in your QuickSight profile:

  1. Choose the user profile, and then choose Manage QuickSight.
  2. Choose Security & permissions.
  3. Choose Add or remove under QuickSight access to AWS services.
  4. To allow Athena and S3 permissions, follow steps 2 and 3 from the previous section.

Create a dataset in QuickSight

You can create datasets in QuickSight using Athena tables as the source. An AWS Glue crawler crawls the inventory data in the S3 bucket and updates the tables in the AWS Glue Data Catalog. These tables are then made available in Athena by the AWS Glue crawler. Each inventory metadata has a corresponding Athena table that's created by AWS Glue. To create the dataset and analyze the data, use the aws_compliancesummary and aws_complianceitem tables:

  1. On the QuickSight start page, choose Datasets from the navigation pane, and then choose New dataset.
  2. Under Create a Dataset, select Athena as the data source.
  3. Enter the data source name.
  4. Choose Create data source.
  5. From the dropdown list of databases, select the S3 bucket with inventory data.
    The database name is in the format S3_bucket_name-<region>-database.
  6. Select aws_compliancesummary from the list of tables, and then choose Select.
  7. Select Directly query your data.
  8. Choose Edit/Preview data.
  9. Choose Save and publish.

Use the preceding instructions to create another dataset for the aws_complianceitem table.

Analyze the dataset

You can use QuickSight Analyses to visualize and analyze data. To use the aws_compliancesummary and aws_complianceitem datasets for data analysis, do the following:

  1. On the Amazon QuickSight start page, choose New Analysis.
  2. On the Datasets page, choose the aws_compliancesummary dataset, and then choose USE IN ANALYSIS.
  3. To add multiple datasets in the same analysis, choose Edit (pencil icon) next to Dataset.
  4. In the pop-up page that appears, choose Add dataset, and then select aws_complianceitem from the list. Choose Select.
    From the dataset dropdown list, you can view these two datasets for Analyses.

Note: You can also add multiple other datasets to the same analysis to create the visuals.

Add visuals

Note: The provided steps for adding visuals are examples. You can create these graphs according to your use case and requirements.

You can add a visual to your Amazon QuickSight analysis based on the QuickSight dataset. The dataset includes the tables from Athena that has the Systems Manager inventory data with compliance information.

Visuals for aws_compliancesummary dataset

You can add a visual for the aws_compliancesummary dataset by following the instructions in Adding a visual.

You can also add filters to filter the data based on compliance type, such as patch compliance and association compliance:

  1. Choose Filter from the left navigation pane.
  2. Choose ADD FILTER, and then select Compliance type.
  3. From the list of values, select Patch to include only patch compliance.
  4. Select all applicable visuals in the Applied to- dropdown.
  5. Choose Apply.

To view the count of resources based on patch compliance status, do the following:

  1. In Visual types, select Donut chart.
  2. From the Fields list, select Status to add it to Group/Color dimension.
  3. Drag and drop Resourceid under Value.
  4. To count distinct values, choose the arrow that's next to resourceid.
  5. Choose Aggregate: Count, and then choose Count distinct.
  6. Select the graph. Then, choose Format Visual icon (pencil icon).
  7. Under Data labels, select Show metric.
    You can see the actual values and percentages in the graph.

To view the compliant instances by Region, do the following:

  1. Choose the preceding visual, and then choose the three dots on the chart.
  2. Choose Duplicate visual.
  3. Under Field wells, in the Group/Color dimension dropdown, select region.
  4. Choose Filter, choose ADD FILTER, select Status, and then select COMPLIANT.
  5. Choose Apply.
    You can see the graph for compliant instances in each Region.

To view noncompliant instances by Region, do the following:

  1. In the preceding visual, choose the three dots on the chart.
  2. Choose Duplicate visual.
  3. Choose Filter. Choose the Status filter, and then select NON_COMPLIANT.
  4. Choose Apply.
    You can see the graphs for noncompliant instances in each Region.

To view the account information for all accounts in a multi-account setup, use the preceding visual. Under Fields wells, in the Group/Color dimension, select accountid. You can see the graph that's based on account IDs.

Visuals for aws_complianceitem dataset

You can add a visual for the aws_complianceitem dataset by following the instructions in Adding a visual.

You can also add filters to filter the data based on compliance type, such as patch compliance and association compliance. To do so, use corresponding instructions from the preceding section.

To view the list of missing patches by instances, do the following:

  1. In Visual types, select Pivot table.
  2. Add Region, resourceid, patchstate, id, and title under Rows and id under Values.
  3. To count distinct values, choose the arrow that's next to id.
  4. Choose Aggregate: Count, and then choose Count distinct.
  5. Choose Filter. Choose ADD FILTER, choose Patchstate, and then select Missing.
  6. Choose Apply.

To view the list of instances by compliance status, do the following:

  1. Select the aws_complianceitem dataset.
  2. Choose ADD, and then choose Add visual.
  3. In Visual types, select Pivot table.
  4. Add Region, resourceid, patchstate, id, and title under Rows and id under Values.
  5. To count distinct values, choose the arrow that's next to id.
  6. Choose Aggregate: Count, and then choose Count distinct.

To get information on all accounts, in the preceding visual, add accountid as the first field under Rows. This filters the pivot table based on account ID.

Publish a dashboard

You can publish all the visuals that are created as a dashboard and share it with other users.

  1. After adding all the visuals, choose Themes, and then select the appropriate theme.
  2. Choose Share on the top right of the page.
  3. Select Publish Dashboard.
  4. Enter a name for the dashboard, and then choose Publish Dashboard.

Considerations

  • AWS Glue crawler crawls the inventory data in the central S3 bucket twice daily by default. Therefore, data is updated based on this schedule. You can modify the frequency based on the requirement by editing the AWS Glue crawler schedule.
  • You can create a joined dataset in QuickSight to join multiple Athena tables to create a merged dataset. The use case for this scenario is to create a joined dataset with aws_compliancesummary and aws_instanceinformation tables to visualize the data based on Platform (Linux/Windows). The Platform information is captured only in the aws_instanceinformation table. Also, you can use this information to filter out data on terminated instances. This data is saved for 30 days in Systems Manager Inventory. For more information, see Joining data.
AWS OFFICIAL
AWS OFFICIALUpdated 10 months ago