Unable to run CloudFormation template for 'crawl delta lake tables using AWS crawler' ?

0

Enter image description here

Not able to create crawler using cloud formation template facing this error "Invalid Delta Target with empty delta tables". Though I have data in the tables available and also, I am able to create the crawler using aws console having same data path successfully and is viewed in athena.

AWSTemplateFormatVersion: "2010-09-09"
Description: Delta lake tables using aws glue crawler

Resources:
  Crawl:
    Type: AWS::Glue::Crawler
    Properties:
      Name: crawler_name
      DatabaseName: database_name
      Role: role_arn
      Targets:
        DeltaTargets:
          - DeltaTables: 
            - S3_path_for_delta_table_folder (having deltalogs as child inside)
          - WriteManifest: True  
  • Are you sure the path is correct and available at the time of creation?, maybe check the crawler logs for more info

gefragt vor 6 Monaten150 Aufrufe
1 Antwort
1
Akzeptierte Antwort

Hello,

I tried deploying your template and faced the same error. Then after checking the syntax for the DeltaTargets Property, I noticed there's an extra '-' (hyphen) in the template, which isn't needed. Below is the corrected template.

Resources:
  Crawl:
    Type: AWS::Glue::Crawler
    Properties:
      Name: crawler_name
      DatabaseName: database_name
      Role: role_arn
      Targets:
        DeltaTargets:
          - DeltaTables: 
              - S3_path_for_delta_table_folder (having deltalogs as child inside)
            WriteManifest: True  

DeltaTables and WriteManifest both are part of one DeltaTarget[1]. Hence we need to add '-' only on DeltaTables and not on WriteManifest. I tried the above template and it worked. Hope this helps.

References: [1]https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-crawler-deltatarget.html

AWS
SUPPORT-TECHNIKER
beantwortet vor 5 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen