AWS Glue crawler
AWS Glue crawler is not pulling all tables names from s3 bucket. Files are present in S3 bucket, if there any setting we have to change for crawler so it pull all the tables from s3 bucket. Is there is know issue for glue crawler to pull data from s3 bucket.
Hi,
how the Crawler is extracting the metadata from the files in the buckets depends on a few factors that you can review in this knowledge base article.
For example if the files are in subfolders/prefixes and are very similar they could be detected as partitions. More information can be found here.
To have more precise feedback for your specific situation could you please provide an example of the structure in S3 and the tables that are and are not created by the crawler?
Thank you, and hope this helps
Relevant questions
403 Access denied error from S3 in Glue
Accepted Answerasked 5 years agoAWS Glue crawler
asked a month agoAWS Glue crawler creating multiple tables
asked 5 months agoGlue Crawler getting 403 from S3 because "ciphertext refers to a CMK that doesn't exist." (using SSE-S3, not KMS)
Accepted Answerasked 3 months agoError Running Glue Crawler
Accepted Answerasked 3 years agoAWS Glue crawler exclude patterns not working
Accepted Answerasked 5 months agoAWS Crawler to directly read Delta lake files from S3
asked 5 days agoCSV crawler tables name
asked 5 months agoRunning glue crawler on encrypted S3 objects present in different account
asked 5 months agoDelete partitions in Glue Data Catalog using crawler not working.
asked 2 months ago