getJDBCSink does not like Postgres timestamp fields

0

Hi All,
My DynamicFrame:

{"id": "fakeid1", "createdDate": "2020-11-28 20:20:44", "lastModifiedDate": "2029-10-26 17:23:40", "firstName": "yedUsFwdkelQbxeTeQOvaScfqIOOmaa", "middleName": "RYtGKbgicZaHCBRQDSx", "lastName": "JxkyvRnL", "secondLastName": "secondlastname", "prefix": "VLhpfQGTMDYpsBZxvfBoeygjb", "suffix": "UMaAIKKIkknjWEXJUfPxxQHeWKEJ", "role": "dpHYZGhtgdntugzvvKAXLhM", "title": "LlN", "department": "gNfZBdyFGRajVfJNonEnOinZj", "dateOfBirth": "2020-10-30", "sex": "UfzQhdgLLfDTDGspDb", "gender": "QvBQYuxiXXVytGCxzVllpgTJKhRQq", "pronoun": "qszYL", "kyc": true, "userStatus": "fakeuserstatus", "employeeNumber": "eOMtThyhVNLWUZNRcBaQKxI", "citizenId": "YdvDhtAsLghPXAgtbprXPZkhnfLTBSX"}   

My write command (which works for other tables):

glueContext.getJDBCSink(catalogConnection = catalogueConnection, options = JsonOptions(employeeAccountConnOptions), redshiftTmpDir = tempStorage).writeDynamicFrame(employeeAccountsDynF)  

Now, even though I can manually insert that string into a postgres db, as a timestamp field, glue will not. Glue says

org.postgresql.util.PSQLException : ERROR: column "createdDate" is of type timestamp without time zone but expression is of type character varying  

This typically means that the string I'm passing cannot be interpreted as a timestamp. That's why I tested it manually.

Any ideas? Please and thanks.
M

demandé il y a 3 ans349 vues
1 réponse
0

Found it. The spark column type was string. Cast it and it works.

répondu il y a 3 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions