getJDBCSink does not like Postgres timestamp fields

0

Hi All,
My DynamicFrame:

{"id": "fakeid1", "createdDate": "2020-11-28 20:20:44", "lastModifiedDate": "2029-10-26 17:23:40", "firstName": "yedUsFwdkelQbxeTeQOvaScfqIOOmaa", "middleName": "RYtGKbgicZaHCBRQDSx", "lastName": "JxkyvRnL", "secondLastName": "secondlastname", "prefix": "VLhpfQGTMDYpsBZxvfBoeygjb", "suffix": "UMaAIKKIkknjWEXJUfPxxQHeWKEJ", "role": "dpHYZGhtgdntugzvvKAXLhM", "title": "LlN", "department": "gNfZBdyFGRajVfJNonEnOinZj", "dateOfBirth": "2020-10-30", "sex": "UfzQhdgLLfDTDGspDb", "gender": "QvBQYuxiXXVytGCxzVllpgTJKhRQq", "pronoun": "qszYL", "kyc": true, "userStatus": "fakeuserstatus", "employeeNumber": "eOMtThyhVNLWUZNRcBaQKxI", "citizenId": "YdvDhtAsLghPXAgtbprXPZkhnfLTBSX"}   

My write command (which works for other tables):

glueContext.getJDBCSink(catalogConnection = catalogueConnection, options = JsonOptions(employeeAccountConnOptions), redshiftTmpDir = tempStorage).writeDynamicFrame(employeeAccountsDynF)  

Now, even though I can manually insert that string into a postgres db, as a timestamp field, glue will not. Glue says

org.postgresql.util.PSQLException : ERROR: column "createdDate" is of type timestamp without time zone but expression is of type character varying  

This typically means that the string I'm passing cannot be interpreted as a timestamp. That's why I tested it manually.

Any ideas? Please and thanks.
M

gefragt vor 3 Jahren370 Aufrufe
1 Antwort
0

Found it. The spark column type was string. Cast it and it works.

beantwortet vor 3 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen