スキップしてコンテンツを表示

AWS Glue Data Quality with Iceberg

0

Hallo, I would like to add some data quality rules to my existing tables which are in Iceberg format. (Created via Glue). After setting up the DQ rule got the following error message:


Error encountered
InvalidInputException (status: 400): Invalid table definition: Parameter "table_type" in table [REDACTED] is set to "iceberg" but storage location doesn't conform to the format "s3://<warehouse-dir>/<glue database>/<glue table id>". To specify your warehouse directory, pass it in the request through AdditionalOptions={WarehouseDirectory=<your-warehouse-dir>}

We had to use a custom location of the warehouse directory. But how can I specify the warehouse directory? I cannot see any place where can I paste that info in the Data Quality module. Thanks in advance!

質問済み 2年前956ビュー
1回答
0

I don't think you can specify that property but you shouldn't need either. Check the table location property, as long as it points to a directory (but without a trailing /). It expects something like this:
s3://mybucket/myWH/mydb.db/table1
s3://mybucket/somePath/mydb.db/table1
(In practice it will consider everything up to the db folder as the Warehouse dir)

AWS
エキスパート
回答済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

関連するコンテンツ