1 Answer
- Newest
- Most votes
- Most comments
1
The index is just speeding up the partition lookup, it won't affect the data.
The partition columns do affect how the data is structured on s3 and how is inserted.
Normally you define the partition columns/index and the tool writing into the table will register the partitions after inserting the data, if the partitions are not registered with specific values and paths then the data is not available for querying.
Relevant content
- asked 2 years ago
- asked 2 years ago
- AWS OFFICIALUpdated 3 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
Thanks a lot,
After deleting the partition and index, I can see the data in the table again.
If I used as a tool to write into the table, Glue ETL, how should I register the partition and values/paths in the above configuration (should I use a specific component or should I be able to do this from the AWS Glue Data Catalog component) ?
When you said speeding the lookup, does this means also to decrease the scanned volume of data (instead of scanning data, to scan the index) ?
Thank you, Mihai