By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Multidata architecture help

0

How can I setup the datapipeline to help us setup a way to query 30-60 like databases.

Currently we are using a connector to replicate data from all our locations to RDS MySQL db. We are wanting to use Quicksights to query Sets of data from all the RDS Sources brought into the RDS MySQL db.

Would love some guidance on how we can accomplish this. I appreciate all of you brilliant people out there

Topics
Migration & ModernizationAnalyticsDatabase
Tags
AWS Database Migration ServiceAmazon Relational Database ServiceAmazon QuickSight
Language
English
Gentry-Develops
asked 2 years ago300 views
1 Answer
0

Currently there may not be a way to accomplish this with DataPipeline. Instead, I would recommend using Glue Crawlers to crawl all the tables in RDS database - after which the table definitions will be stored in Glue Data Catalog. Then, you can access these tables from Athena, where you can perform queries and also implement some transformations using Glue DataBrew and then visualize them/create BI reports in QuickSight. See documentation references -

  1. https://docs.aws.amazon.com/glue/latest/dg/populate-data-catalog.html
  2. https://docs.aws.amazon.com/athena/latest/ug/glue-athena.html
  3. https://aws.amazon.com/blogs/big-data/data-preparation-using-an-amazon-rds-for-mysql-database-with-aws-glue-databrew/
AWS
Meghana_S
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content