Skip to content

Athena MySQL Connector adding unexpected partition field

0

Hi, I have set up an Athena MySQL connector for federated queries to RDS. For some reason an extra column not present in my RDS table is added called "partition_name" all values set to "*". I have no partitions set up in my table in RDS. I'm I missing something can anyone help me understand why this column is present in Athena.

Thank you in advance for the help,

Kind regards,

Domenico

1 Answer
0
Accepted Answer

The unexpected "partition_name" column you're seeing in your Athena query results when using the MySQL connector is actually a normal behavior of the connector. This synthetic column is created by Athena to help with query optimization and data retrieval, even if you don't have any partitions set up in your RDS MySQL table.

The Athena MySQL connector uses this synthetic partition column to determine how to generate splits for the connector. It's important to note that this column is not actually modifying your table definition in RDS; it's only used internally by Athena to improve query performance.

The "partition_name" column with all values set to "*" is essentially a placeholder that allows Athena to use its internal partitioning mechanism even when the source MySQL table isn't partitioned. This approach helps Athena to potentially parallelize queries and improve overall performance when working with federated data sources.

You don't need to take any action regarding this column, as it's part of how the Athena MySQL connector operates. It doesn't affect your actual data or table structure in RDS. If you need to work with your data without this additional column, you can simply exclude it from your SELECT statements in your Athena queries.
Sources
Amazon Athena MySQL connector - Amazon Athena

answered a year ago
EXPERT
reviewed a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.