1 Respuesta
- Más nuevo
- Más votos
- Más comentarios
0
new partitions will be visible for SELECTS after the metadata about them is available which will happen either after
MSCK REPAIR TABLE
or (more lightweight and therefore preferred)
ALTER TABLE ... ADD PARTITION
You can however add those partitions "in advance" even before any data is added for those, and in this case the data will be available to SELECT queries as soon as some of the files are added to those partitions by INSERT INTO SELECT
https://docs.aws.amazon.com/athena/latest/ug/msck-repair-table.html
https://docs.aws.amazon.com/athena/latest/ug/alter-table-add-partition.html
respondido hace 2 años
Contenido relevante
- OFICIAL DE AWSActualizada hace 2 años
- OFICIAL DE AWSActualizada hace 4 meses
- OFICIAL DE AWSActualizada hace un año
If I don't ADD PARTITION in advance, and don't call MSCK REPAIR, is it still the case that "the data will be available to SELECT queries as soon as some of the files are added to those partitions by INSERT INTO SELECT"? That would essentially be the same as saying that in this scenario, the existence of a partition does not guarantee the corresponding INSERT INTO has finished writing the partition.