By using AWS re:Post, you agree to the Terms of Use

Multidata architecture help

0

How can I setup the datapipeline to help us setup a way to query 30-60 like databases.

Currently we are using a connector to replicate data from all our locations to RDS MySQL db. We are wanting to use Quicksights to query Sets of data from all the RDS Sources brought into the RDS MySQL db.

Would love some guidance on how we can accomplish this. I appreciate all of you brilliant people out there

1 Answers
0

Currently there may not be a way to accomplish this with DataPipeline. Instead, I would recommend using Glue Crawlers to crawl all the tables in RDS database - after which the table definitions will be stored in Glue Data Catalog. Then, you can access these tables from Athena, where you can perform queries and also implement some transformations using Glue DataBrew and then visualize them/create BI reports in QuickSight. See documentation references -

  1. https://docs.aws.amazon.com/glue/latest/dg/populate-data-catalog.html
  2. https://docs.aws.amazon.com/athena/latest/ug/glue-athena.html
  3. https://aws.amazon.com/blogs/big-data/data-preparation-using-an-amazon-rds-for-mysql-database-with-aws-glue-databrew/
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions