Skip to content

AWS LakeFormation Cross Account RAM Share

0

Hi AWS, in our organization we monthly generate a report which we need to share with a Partner AWS account. The report is in parquet format stored inside one of the Glue Tables under the Glue Database which is the part of AWS LakeFormation Data Catalog. As a result we figured out that we need to share the data from the Customer AWS account to the external Partner AWS account using RAM Share. The location for the Glue Table is an S3 bucket.

I have two questions on this:

  1. What all are the steps I need to follow as part of RAM Share (manually for the first time).
  2. As a long term solution I need to automate this. What should be the right way to implement it. Will a standalone CloudFormation template enough or I need to set up a new GitHub/Bitbucket repo for the same. We are using Bitbucket as a version control tool.
1 Answer
0

To share your monthly report stored in a Glue Table with a partner AWS account using Lake Formation and AWS Resource Access Manager (RAM), you'll need to follow these steps:

  1. Steps for manual RAM Share:

a) Sign in to the Lake Formation console as a data lake administrator. b) Use the Lake Formation console to grant Lake Formation permissions on the specific Glue database and table to the partner AWS account. c) Lake Formation will use AWS RAM to facilitate the permission granting between accounts. d) The partner account will receive an AWS RAM share invitation, which they need to accept. e) Once accepted, the data lake administrator in the partner account must create a resource link to the shared database and/or table. f) The partner account administrator then needs to grant necessary Lake Formation permissions (like 'Describe' on the resource link and 'Grant on target' on the original shared resource) to the relevant IAM principals in their account. g) Finally, the partner account can set up Lake Formation permissions for their users on the shared database or table.

  1. For automation and long-term solution:

Automating this process would indeed be beneficial for a recurring monthly task. A standalone CloudFormation template could be sufficient for this purpose, but considering the complexity and potential need for version control, setting up a dedicated repository in Bitbucket would be a more robust solution. Here's a suggested approach:

a) Create a new Bitbucket repository for this automation project. b) Develop a CloudFormation template that defines the necessary resources and permissions for the RAM share. This should include the Lake Formation permissions, AWS RAM resource share, and any necessary IAM roles or policies. c) Create a script (e.g., using AWS CLI or SDK) to automate the process of updating the shared data and triggering the CloudFormation stack update. d) Set up a CI/CD pipeline in your preferred tool (e.g., Bitbucket Pipelines) to run this script on a monthly schedule. e) Store any sensitive information (like account IDs) in a secure parameter store (e.g., AWS Systems Manager Parameter Store) and reference them in your scripts. f) Include documentation in the repository explaining the setup and process.

This approach allows for version control of your infrastructure-as-code, easier collaboration, and a more maintainable long-term solution. It also provides the flexibility to make changes or expand the sharing process in the future if needed.
Sources
Cross-account data sharing in Lake Formation - AWS Lake Formation
Cross-account data sharing best practices and considerations - AWS Lake Formation
Sharing a Lake Formation resource using hybrid access mode - AWS Lake Formation

answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.