Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Glue crawler updating tables from s3 which doesn't exists.

0

I have a weird situation, where I have setup a glue crawler which consumes SQS(s3 event triggered). SO, technically it triggers whenever a message is sent to the queue with a table level of 3 s3://s3_path/folder1/folder2/table. My problem is that, i have deleted some of the table from the s3 path, glue catalog and athena. But with crawler run, the old table gets created which techincally doesn't exists anymore in the s3. I am not sure how these tables are getting created.

Topics
AnalyticsApplication IntegrationStorage
Tags
AWS GlueAmazon S3 Access GrantsAmazon AthenaAmazon Simple Queue Service
Language
English

asked 2 years ago115 views

1 Answer
0

check that your sqs queue is not re sending old messages that leads to old tables recreation , or check the tables in glue catalog and they point to which s3 path exactly , can be that you hae duplicated files in s3

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content