How to convert SQL Results into a Pandas Dataframe in SageMaker Unified Studio?

In Amazon SageMaker Studio, you have a couple of straightforward methods to convert SQL query results into a pandas DataFrame:

1. Using the JupyterLab SQL extension: When you run your SQL query, look for the query-result dropdown in the JupyterLab SQL extension. From this dropdown, simply select the "Pandas dataframe" option. This will automatically save your query results as a pandas DataFrame.

2. Using the %%sm_sql magic command: If you're working in a notebook, you can use the %%sm_sql magic command with a specific parameter to output your query results as a DataFrame. Here's how:

   • Add the following parameter to your %%sm_sql command: --output '{"format": "DATAFRAME", "dataframe_name": "your_dataframe_name"}'

   For example, your command might look like this:

   %%sm_sql --output '{"format": "DATAFRAME", "dataframe_name": "my_data"}'
   SELECT * FROM your_table
   

   This will execute your SQL query and save the results in a pandas DataFrame named "my_data".

Both of these methods allow you to seamlessly transition from SQL queries to pandas DataFrames, enabling you to continue your data analysis or visualization tasks in Python. After converting your query results to a DataFrame, you can manipulate and analyze the data using standard pandas operations.
Sources
Save SQL query results in a pandas DataFrame - Amazon SageMaker
Explore data with ease: Use SQL and Text-to-SQL in Amazon SageMaker Studio JupyterLab notebooks | AWS Machine Learning Blog