getJDBCSink does not like Postgres timestamp fields

0

Hi All,
My DynamicFrame:

{"id": "fakeid1", "createdDate": "2020-11-28 20:20:44", "lastModifiedDate": "2029-10-26 17:23:40", "firstName": "yedUsFwdkelQbxeTeQOvaScfqIOOmaa", "middleName": "RYtGKbgicZaHCBRQDSx", "lastName": "JxkyvRnL", "secondLastName": "secondlastname", "prefix": "VLhpfQGTMDYpsBZxvfBoeygjb", "suffix": "UMaAIKKIkknjWEXJUfPxxQHeWKEJ", "role": "dpHYZGhtgdntugzvvKAXLhM", "title": "LlN", "department": "gNfZBdyFGRajVfJNonEnOinZj", "dateOfBirth": "2020-10-30", "sex": "UfzQhdgLLfDTDGspDb", "gender": "QvBQYuxiXXVytGCxzVllpgTJKhRQq", "pronoun": "qszYL", "kyc": true, "userStatus": "fakeuserstatus", "employeeNumber": "eOMtThyhVNLWUZNRcBaQKxI", "citizenId": "YdvDhtAsLghPXAgtbprXPZkhnfLTBSX"}   

My write command (which works for other tables):

glueContext.getJDBCSink(catalogConnection = catalogueConnection, options = JsonOptions(employeeAccountConnOptions), redshiftTmpDir = tempStorage).writeDynamicFrame(employeeAccountsDynF)  

Now, even though I can manually insert that string into a postgres db, as a timestamp field, glue will not. Glue says

org.postgresql.util.PSQLException : ERROR: column "createdDate" is of type timestamp without time zone but expression is of type character varying  

This typically means that the string I'm passing cannot be interpreted as a timestamp. That's why I tested it manually.

Any ideas? Please and thanks.
M

已提问 3 年前366 查看次数
1 回答
0

Found it. The spark column type was string. Cast it and it works.

已回答 3 年前

您未登录。 登录 发布回答。

一个好的回答可以清楚地解答问题和提供建设性反馈,并能促进提问者的职业发展。

回答问题的准则