Method for converting all values of a field to string when querying with Ion SerDe?

0

I have some event logs stored as Ion data in S3, and I am trying to query it using Amazon Athena's Ion Hive SerDe. One of the datafields that is in most of our events is typically an int, but in a few events the value for the datafield is a string. I have tried specifying the datafield as a STRING when creating the external table, which works most of the time, but sometimes returns a HIVE_BAD_DATA error that says the SerDe cannot convert from IonIntLite to IonText. Since the Ion SerDe requires homogenous data, is there an official way in Athena to make all values conform to string when querying? The datafield is an ID so we often just need to check equality to an ID number. Or, is there a way to just ignore any entries that have the datafield not as the type we specify (i.e. specify the ID as BIGINT, and have Athena ignore any events with a string ID)?

wfchang
demandé il y a 2 ans106 vues
Aucune réponse

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions