- Più recenti
- Maggior numero di voti
- Maggior numero di commenti
I have been struggling with this one myself. I achieved solving my problem by crawling the DynamoDB table with AWS Glue and let AWS Glue infer the schema.
-
In AWS Glue, create a new database with its
Location URI
orLocation
set to the valuedynamo-db-flag
- use this exact value; otherwise, Athena won't see the future table. -
In AWS Glue, create a crawler:
- Select DynamoDB as the data source.
- For the table name, give the same table name as your dynamodb table.
- For the target database, choose the database you just created.
- Run the crawler.
- In AWS Glue, after the Crawler ran successfully, verify the table's schema, which the crawler just populated. Edit the table in Glue to add two additional table properties (Table > Edit Table Property > Table Properties):
sourceTable
is the exact name of your dynamodb table, matching case. E.g. if my dynamodb table is namedDbUsersTableXDXDXDXDX-SZSZSZSZSZSZSZ
,sourceTable
property's value has to beDbUsersTableXDXDXDXDX-SZSZSZSZSZSZSZ
.columnMapping
defines column name mappings. In AWS Glue catalog, column names should be in lowercase. So, this entry is necessary. E.g. if your table has 3 column:id
,email
,createdAt
, the value ofcolumnMapping
isid=id,email=email,createdat=createdAt
, (Glue column name) in the left side of each mapping (=
).
- After saving the changes, go to Amazon Athena, for the DataSource
DynamoDB
(from the dynamodb connector), select the new database you created in (1), the table should be here.
I'm worried about this solution. I have used the existing database (dynamoDBconnection.default) as the data source for QuickSight, which has an ETL built on those tables. I don't know if I can change that data source easily.
Have you tried: https://docs.aws.amazon.com/quicksight/latest/user/troubleshoot-athena-column-not-found.html
This would be a slightly different issue. Right now, queries in Athena don't show the column, so they are not expected in Quicksight which is using Athena as a data source
Contenuto pertinente
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
- AWS UFFICIALEAggiornata 2 anni fa
Same issue is occurring for me now. The initial scan of the table works on the install, but the table definitions are not updated with newly added columns.