Athena DynamoDB table queries fail when there are null booleans

0

When we attempt to query a table and include boolean fields that may be null in the select clause, we receive an exception. (Note, when the records with a null are filtered out with where table.boolColumn is not null, the query succeeds.)

In Athena, I get:

GENERIC_USER_ERROR: Encountered an exception[java.lang.NullPointerException] from your LambdaFunction[arn:aws:lambda:<redacted>:function:dynamodb2] executed in context[S3SpillLocation{bucket='<redacted>:', key='athena-spill/f91514ab-130d-4464-9c5f-cfef12dd2e91/e4763105-fb52-4106-b619-006b6d7f521b', directory=true}] with message[java.lang.NullPointerException]
This query ran against the "default" database, unless qualified by the query. Please post the error message on our forum  or contact customer support  with Query Id: f91514ab-130d-4464-9c5f-cfef12dd2e91

In our CloudWatch log, I see this exception:

java.lang.NullPointerException: java.lang.NullPointerException
java.lang.NullPointerException
	at com.amazonaws.athena.connectors.dynamodb.util.DDBTypeUtils.lambda$makeExtractor$4(DDBTypeUtils.java:433)
	at com.amazonaws.athena.connector.lambda.data.writers.fieldwriters.BitFieldWriter.write(BitFieldWriter.java:76)
	at com.amazonaws.athena.connector.lambda.data.writers.GeneratedRowWriter.writeRow(GeneratedRowWriter.java:116)
	at com.amazonaws.athena.connectors.dynamodb.DynamoDBRecordHandler.lambda$readWithConstraint$0(DynamoDBRecordHandler.java:207)
	at com.amazonaws.athena.connector.lambda.data.S3BlockSpiller.writeRows(S3BlockSpiller.java:183)
	at com.amazonaws.athena.connectors.dynamodb.DynamoDBRecordHandler.readWithConstraint(DynamoDBRecordHandler.java:207)
	at com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doReadRecords(RecordHandler.java:192)
	at com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doHandleRequest(RecordHandler.java:158)
	at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:138)
	at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:103)

java.lang.NullPointerException: java.lang.NullPointerException java.lang.NullPointerException at com.amazonaws.athena.connectors.dynamodb.util.DDBTypeUtils.lambda$makeExtractor$4(DDBTypeUtils.java:433) at com.amazonaws.athena.connector.lambda.data.writers.fieldwriters.BitFieldWriter.write(BitFieldWriter.java:76) at com.amazonaws.athena.connector.lambda.data.writers.GeneratedRowWriter.writeRow(GeneratedRowWriter.java:116) at com.amazonaws.athena.connectors.dynamodb.DynamoDBRecordHandler.lambda$readWithConstraint$0(DynamoDBRecordHandler.java:207) at com.amazonaws.athena.connector.lambda.data.S3BlockSpiller.writeRows(S3BlockSpiller.java:183) at com.amazonaws.athena.connectors.dynamodb.DynamoDBRecordHandler.readWithConstraint(DynamoDBRecordHandler.java:207) at com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doReadRecords(RecordHandler.java:192) at com.amazonaws.athena.connector.lambda.handlers.RecordHandler.doHandleRequest(RecordHandler.java:158) at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:138) at com.amazonaws.athena.connector.lambda.handlers.CompositeHandler.handleRequest(CompositeHandler.java:103)
asked a year ago682 views
1 Answer
0

Hello,

I understand that you are facing the following error while trying to query a table and include boolean fields that are null in the select clause:

GENERIC_USER_ERROR: Encountered an exception[java.lang.NullPointerException] from your LambdaFunction[arn:aws:lambda:<redacted>:function:dynamodb2] executed in context[S3SpillLocation{bucket='<redacted>:', key='athena-spill/f91514ab-130d-4464-9c5f-cfef12dd2e91/e4763105-fb52-4106-b619-006b6d7f521b', directory=true}] with message[java.lang.NullPointerException]
This query ran against the "default" database, unless qualified by the query. Please post the error message on our forum  or contact customer support  with Query Id: f91514ab-130d-4464-9c5f-cfef12dd2e91

According to the documentation: https://usefulangle.com/post/332/dynamodb-attribute-types Null is recognized as a datatype in DynamoDB and same as Boolean. Which means that the reason why you are receiving this error is because under the same column you have two different datatype, Null and Boolean and you cannot have two datatypes in one column, either you choose a Boolean datatype or Null datatype.

AWS
SUPPORT ENGINEER
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions