Athena Iceberg in CDK

0

Usually I have no problems creating Athena tables in CDK v2 using new glue.Table However now I'm stuck trying to create Iceberg table using the same approach. Documentation was not really helpful, of far. I do not really want to create it using CREATE TABLE SQL statement for a various reasons.

It there any approach?

profile picture
Smotrov
asked 5 months ago348 views
2 Answers
0

I haven't tried with glue.Table, but with glue.CfnTable it worked for me setting tableType='ICEBERG and parameters={ "classification": "glueparquet", "lakeformation.aso.status": "true" }

The lakeformation.aso.status parameter enables automated compaction.

profile pictureAWS
EXPERT
Tasio
answered 5 months ago
  • Thank you, I try this approach but with no luck. A simple select from such table is errored: HIVE_UNSUPPORTED_FORMAT: Unable to create input format

      const dataIceberg = new mainGlue.CfnTable(scope, tableName, {
        databaseName: database.databaseName,
        catalogId: database.catalogId,
        tableInput: {
          name: tableName,
          tableType: 'ICEBERG',
          parameters: {
            classification: 'glueparquet',
            'lakeformation.aso.status': 'true',
          },
          // De
          storageDescriptor: {
            location: `s3://${bucket.bucketName}/${s3Prefix}`,
            columns: [
              {
                name: 'loadId',
                type: 'string',
              },
              {
                name: 'od_group',
                type: 'string',
              },
              {
                name: 'data',
                type: 'string',
              },
            ],
          },
        },
      });
    
0

Hello.

CfnTable has an optional parameter openTableFormatInput. There is a small bit of reference here: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_glue.CfnTable.html#opentableformatinput.

Here is a shortened version of CDK code I used as example, maybe it helps you.

this.table = new glue.CfnTable(this, "Table", {
        databaseName: this.databaseName,
        catalogId: this.account,
        tableInput: {
            name: tableName.toLowerCase(),
            description: description,
            storageDescriptor: {
                columns: columns,
                location: `s3://${this.bucketName}/${tableName.toLowerCase()}`,
                schemaReference: this.schemaReference,
                sortColumns: this.sortColumns,
                storedAsSubDirectories: false,
            },
            tableType: 'EXTERNAL_TABLE'
        },
        openTableFormatInput: {
            icebergInput: {
                metadataOperation: "CREATE",
                version: "2",
            }
        }
});
answered 5 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions