AWS Glue API get-partitions can't seem to cope with the partition column name "key" in the `expression` filter

Question

I created a AWS Glue table with the following columns:
- key : string (partition column)
- x : int

The following command works as expected:

```
aws glue get-partitions --database-name glue --table-name repro100 --region us-east-2
```

On the other hand, the command:

```
aws glue get-partitions --database-name glue --table-name repro100 --region us-east-2  --expression="key=part1"
```

returns the following exception message:

```
An error occurred (InvalidInputException) when calling the GetPartitions operation:
```

Doing the same call via Java AWS API I could retrieve the following error code:

```
HIVE_METASTORE_ERROR  code 16777216
```

When using a different partition column name (e.g. : `foobar` ) the expression filter can be handled successfully.
I assume that the problem is linked to the usage of the column name `key` in the expression filter.
Any hints on how to escape the column name in the filter ?

Out of curiosity, what exactly means `HIVE_METASTORE_ERROR  code 16777216` ?

Answer

Hello,

By using "key" as one of the column names in AWS Glue table, get-partitions API call returns error as stated

An error occurred (InvalidInputException) when calling the GetPartitions operation:

I would recommend you to please change the column "key" name as "key1" or any other name in Glue data catalog which is not in the reserved words as a workaround.

Regarding the Hive error code 16777216, it usually happens when the Hive metastore does not return any data for a partition. Here in your case since "key" is not being recognized as a partition, the above error code is returned.

AWS Glue API get-partitions can't seem to cope with the partition column name "key" in the `expression` filter

관련 콘텐츠