Sagemaker and Data on Databases

Question

A customer has a question about data sources

> “most of our data is stored in SQL databases, while the SageMaker docs
> say that I have to put it all in S3. It’s not obvious what the best
> way to do this is. I can think for example of splitting my analysis
> code in two; one pre-processing step to go from SQL queries to tabular
> data, and e.g. store that as Parquet files. For high-dimensional
> tensor data it’s even less obvious.”

Can someone comment on that?

Accepted Answer

We have an example notebook for interacting from Redshift data from a SageMaker managed notebook, which I believe is suitable for an Exploratory Data Analysis (EDA) use-case: https://github.com/awslabs/amazon-sagemaker-examples/blob/master/advanced_functionality/working_with_redshift_data/working_with_redshift_data.ipynb

For production purposes, the customer should consider separating the job of first extracting data from relational databases to S3 (to build out a data lake), and then using that for downstream processing/machine learning (including SageMaker, EMR, Athena, Spectrum, etc.). Customers can build extraction pipelines from popular relational databases using AWS Glue, EMR, or their preferred ETL engines like those on the AWS Marketplace.

Answer

I'd recommend using SageMaker Data Wrangler to connects the dots of different SageMaker services. https://docs.aws.amazon.com/sagemaker/latest/dg/data-wrangler-import.html

Answer

for custom models, you could use Import data into Canvas from database dircetly :Connect to data in Amazon S3, Amazon Athena, or Amazon RDS
For Amazon RDS, if you have the AmazonSageMakerCanvasFullAccess policy attached to your user’s role, then you’ll be able to import data from your Amazon RDS databases into Canvas.

https://docs.aws.amazon.com/sagemaker/latest/dg/canvas-importing-data.html

Sagemaker and Data on Databases

Relevant content