Skip to content

Is there a way to migrate an Atlan data access management solution to SageMaker Catalog?

0

I want to start using SageMaker more to consolidate my data engineering stack. Currently, my team uses Atlan for data access management and we'd like to migrate over to SageMaker Catalog. Is there an easy way to do this?

2 Answers
0

While there isn't specific documentation about migrating directly from Atlan to Amazon SageMaker Catalog, there is information about integrating SageMaker Catalog with Collibra, another data governance platform. This suggests that similar integration approaches might be possible with Atlan.

Amazon SageMaker Catalog is built on Amazon DataZone and allows users to centrally manage data assets, catalog data, create metadata using generative AI capabilities, and enforce access policies with fine-grained access controls. It's accessible through Amazon SageMaker Unified Studio, which brings together various AWS data, analytics, AI, and ML services in a single interface.

For migration, you might consider an approach similar to the Collibra integration, which includes:

  1. Bidirectional metadata synchronization to preserve glossary terms, descriptions, and structure
  2. Access subscription workflow integration to manage data access requests

To implement a migration strategy, you would likely need to:

  • Set up Amazon SageMaker Catalog in your AWS environment
  • Export your metadata and access policies from Atlan
  • Import this information into SageMaker Catalog
  • Configure access controls and permissions in the SageMaker environment

I would recommend reaching out to AWS Support or consulting services for specific guidance on migrating from Atlan, as they may have developed patterns or tools to facilitate this type of migration.
Sources
Unifying metadata governance across Amazon SageMaker and Collibra | AWS Big Data Blog
Amazon SageMaker and when to use Amazon SageMaker vs Amazon DataZone - Amazon DataZone
Amazon SageMaker Data and AI Governance - AWS
The next generation of Amazon SageMaker: The center for all your data, analytics, and AI | AWS Big Data Blog

answered a year ago

0

SageMaker Catalog supports data sources that use the Apache Iceberg table standard.

Atlan's documentation doesn't clearly indicate support for Apache Iceberg, rather Atlan can be configured to work with Apache Iceberg provided the integration is through Apache Polaris, an incubating Apache project that provides an API layer over Apache Iceberg.

The Atlan documentation describes at a high-level how to integrate Atlan with Apache Iceberg for the purposes of using Atlan as the centralized control plane. If the goal is to use Amazon SageMaker, it may make sense to consider a more thorough migration approach using Amazon Glue, and AWS Database Migration Service.

AWS

answered a year ago

EXPERT

reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.