Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Single CRAWLER Table for S3 Path

0

Hi Team,

I want to create only 2 crawler tables (Folder_x & Folder_y) for below path irrespective of the schemas

  1. S3> Folder_x> Folder_x1>Folder_x2 > file1.txt
  2. S3> Folder_x> Folder_x1>Folder_x2 >file2.txt
  3. S3> Folder_y> Folder_y1>Folder_y2 > file3.txt
  4. S3> Folder_y> Folder_y1>Folder_y2 >files4.txt

Could you please help me on this

Thank you in advance

Topics
Analytics
Tags
AWS Glue
Language
English
asked 2 years ago288 views
1 Answer
0

You can follow the steps outlined at Tutorial: Adding an AWS Glue crawler and choose the S3 path as S3> Folder_x for the first one and for the second crawler choose path S3> Folder_y.

This will allow you to run or re-run each crawler independently as your data changes.

AWS
EXPERT
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content