Athena tables gets missing on its own

Question

I have to create athena table using spark sql, like below:

spark.sql(f"""
CREATE TABLE dbname.tablename
using PARQUET
options(path 's3_full_path')
""")

The table is getting created, but after around 10 mins it get missing

Ravish · Answer

I would suggest to create the table using EXTERNAL keyword as per the suggested standard-

Except when creating Iceberg tables, always use the EXTERNAL keyword. If you use CREATE TABLE without the EXTERNAL keyword for non-Iceberg tables, Athena issues an error. When you drop a table in Athena, only the table metadata is removed; the data remains in Amazon S3.

Ref- https://docs.aws.amazon.com/athena/latest/ug/creating-tables.html

https://docs.aws.amazon.com/athena/latest/ug/parquet-serde.html

Athena tables gets missing on its own

Add your answer

Relevant content