Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Why is the Glue database dropdown empty when trying to create a dashboard for the Security Lake data source for OpenSearch Service?

0

I have Security Lake setup and I have an opensearch data source for security lake configured. On the screen for the data source, there's an option to add a new dashboard (for VPC flow logs, WAF logs, or CloudTrail logs). You can enter the name, description, select the Glue database, set the refresh interval, and then whether to add it to a new or existing OpenSearch UI workspace.

However when I go to this screen, the Glue database dropdown is empty.

In the Chrome devtools network tab, I see a bunch of POST requests to the Glue API to get the list of databases and I see the response is 200 OK and contains valid JSON data with all my Glue databases except for my security lake Glue database.

NOTE: My security lake is in another account and I have Lake Formation permissions set up so that I can see them from the Glue console. But maybe OpenSearch can't see them even from the "Add Dashboard" console UI?

I've scoured the docs but I can't find anything about this and as far as I can tell (following the docs) I've got all the IAM and LakeFormation permissions set up properly.

Not sure if this is related or a separate issue, but just in case it's related: When I go to OpenSearch UI and go to a workspace and go to Discover and then select data source and then select Security Lake, then my data connection for my security lake, I get a 400 BadRequest error with the error message "Error while decoding fas credentials KMS key arn must be a non-null string."

The URL is: (my UI URL)/api/directquery/listTables?dataSourceName=MySourceName

I saw in the docs that if your security lake S3 bucket uses a Customer-managed KMS key then you need to tweak the policy but my security lake bucket doesn't use a customer-managed KMS key as far as I can tell. The S3 bucket is in a different account from my opensearch account (both in the same AWS Organization).

Topics
Analytics
Tags
AWS Lake FormationAWS GlueAmazon OpenSearch Service
Language
English
asked a year ago190 views
1 Answer
0

The issue you're experiencing with the empty Glue database dropdown when creating a dashboard for Security Lake in OpenSearch Service is likely related to insufficient permissions or incomplete cross-account setup.

Since your Security Lake is in another account, there are several potential causes:

  1. IAM Permissions: The IAM role used by OpenSearch Service might be missing the necessary permissions to access the Glue databases in the Security Lake account. The role needs permissions like glue:GetDatabase, glue:GetDatabases, glue:GetTable, glue:GetTables, and other Glue-related actions.

  2. Lake Formation Resource Link: When accessing Security Lake data from an account other than the Security Lake delegated admin account, you need to create a resource link to the shared Lake Formation database. This resource link points your account to the shared database. If this resource link is missing or improperly configured, the Glue database won't appear in the dropdown.

  3. Lake Formation Permissions: Even though you can see the databases in the Glue console, OpenSearch Service might be using a different role that doesn't have the proper Lake Formation permissions.

To resolve this issue:

  1. Ensure the IAM role used by OpenSearch has the necessary Glue and Lake Formation permissions, including glue:GetDatabase, glue:GetDatabases, and lakeformation:GetDataAccess.

  2. Verify that you've created a resource link to the shared Lake Formation database from the Security Lake delegated admin account to your account.

  3. Check that the Lake Formation permissions are properly set up for the role that OpenSearch Service is using.

  4. Consider creating a unique database with the CreateDatabase API to store resource link tables, as recommended for Security Lake subscribers.

If you've confirmed all these settings are correct, you might need to check if the cross-account data sharing settings in Lake Formation are updated to version 3, which is required for proper query access to Security Lake data across accounts.
Sources
Troubleshooting AWS Glue Data Quality errors - AWS Glue
Creating an Amazon Security Lake data source integration in OpenSearch Service - Amazon OpenSearch Service
Introducing Amazon OpenSearch Service and Amazon Security Lake integration to simplify security analytics | AWS News Blog
Security OU - Security Tooling account - AWS Prescriptive Guidance

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content