By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Crawling DynamoDB table from another account with AWS Glue Crawler

0

I'm trying to figure out how to crawl a DynamoDB in another account with AWS Glue. Right now it seems like Glue can only crawl DynamoDB's in the current user's account. I've tried setting up several IAM roles and policies, and seems like they all don't work. Can anyone confirm if Glue Crawler supports accessing external DynamoDB tables?

Topics
Security, Identity, & ComplianceAnalyticsDatabase
Tags
AWS Identity and Access ManagementAWS GlueAmazon DynamoDB
Language
English
rePost-User-6209720
asked a year ago711 views
1 Answer
0

AWS Glue Crawlers do support cross-account crawling. Do you know if your analytics components are managed by AWS Lake Formation? If so, and assuming you've set up the proper cross-account permissions it could be that the permissions are missing Lake Formation. Check to see if the Glue crawlers have the right permissions within the Lake Formation permission management as well.

profile pictureAWS
EXPERT
pechung
answered a year ago
  • rePost-User-6209720
    a year ago

    Thanks for replying! After looking into AWS Lake Formation, I had two questions.

    1. Seems like all the examples are about S3, and Lake Formation is optimized for S3. Does approach applies to DynamoDB?
    2. As mentioned in https://aws.amazon.com/blogs/big-data/aws-glue-crawlers-support-cross-account-crawling-to-support-data-mesh-architecture/, it seems like for Account B to crawl data from Account A's data stores, the crawler needs to be created and run in Account A, and the result can be shared with Account B. Is there a way to build a centralized service in which the crawler is created within Account?
  • Gonzalo Herreros EXPERT
    a year ago

    The idea is the same for both s3 and DDB, one account provides the tables (maybe by crawling) and then they are shared. With s3 you could bypass that but it's not ideal.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content