Bug: Firehose to Athena Iceberg Table does not support Databases with dashes in name

Question

I'm trying out the new preview of Firehose writing to Athena Iceberg tables.

I'm publishing to Firehose with the following body, which works fine:
```
{
  "ADF_Record": {
    "user_id": "the-user-id"
  },
  "ADF_Metadata": {
    "OTF_Metadata": {
      "DestinationTableName": "foo_bar",
      "DestinationDatabaseName": "foo_baz",
      "Operation": "INSERT"
    }
  }
}
```

However, if I change the database name to `foo-baz`:
```
{
  "ADF_Record": {
    "user_id": "the-user-id"
  },
  "ADF_Metadata": {
    "OTF_Metadata": {
      "DestinationTableName": "foo_bar",
      "DestinationDatabaseName": "foo-baz",
      "Operation": "INSERT"
    }
  }
}
```
nothing is written, and I see an error of `"An internal error occurred when attempting to deliver data. Delivery will be retried; if the error persists, it will be reported to AWS for resolution."`

Is there a workaround for this, or is it just a bug?

Chanhyeok Seo · Answer

Hello,

[The AWS documentation](https://docs.aws.amazon.com/athena/latest/ug/tables-databases-columns-names.html) for table, database, and column names does not explicitly mention that dashes should not be used:

> Acceptable characters for database names, table names, and column names in AWS Glue must be a UTF-8 string. The string must not be less than 1 or more than 255 bytes long.

However, I found the following information in [the AWS documentation](https://docs.aws.amazon.com/athena/latest/ug/vpc-flow-logs.html#create-vpc-logs-table) about Creating and querying tables for custom VPC flow logs in Athena:

> In Athena, the only acceptable characters for database names, table names, and column names are lowercase letters, numbers, and the underscore character.

I hope this information is helpful.

Bug: Firehose to Athena Iceberg Table does not support Databases with dashes in name

Add your answer

Relevant content