getJDBCSink does not like Postgres timestamp fields

0

Hi All,
My DynamicFrame:

{"id": "fakeid1", "createdDate": "2020-11-28 20:20:44", "lastModifiedDate": "2029-10-26 17:23:40", "firstName": "yedUsFwdkelQbxeTeQOvaScfqIOOmaa", "middleName": "RYtGKbgicZaHCBRQDSx", "lastName": "JxkyvRnL", "secondLastName": "secondlastname", "prefix": "VLhpfQGTMDYpsBZxvfBoeygjb", "suffix": "UMaAIKKIkknjWEXJUfPxxQHeWKEJ", "role": "dpHYZGhtgdntugzvvKAXLhM", "title": "LlN", "department": "gNfZBdyFGRajVfJNonEnOinZj", "dateOfBirth": "2020-10-30", "sex": "UfzQhdgLLfDTDGspDb", "gender": "QvBQYuxiXXVytGCxzVllpgTJKhRQq", "pronoun": "qszYL", "kyc": true, "userStatus": "fakeuserstatus", "employeeNumber": "eOMtThyhVNLWUZNRcBaQKxI", "citizenId": "YdvDhtAsLghPXAgtbprXPZkhnfLTBSX"}   

My write command (which works for other tables):

glueContext.getJDBCSink(catalogConnection = catalogueConnection, options = JsonOptions(employeeAccountConnOptions), redshiftTmpDir = tempStorage).writeDynamicFrame(employeeAccountsDynF)  

Now, even though I can manually insert that string into a postgres db, as a timestamp field, glue will not. Glue says

org.postgresql.util.PSQLException : ERROR: column "createdDate" is of type timestamp without time zone but expression is of type character varying  

This typically means that the string I'm passing cannot be interpreted as a timestamp. That's why I tested it manually.

Any ideas? Please and thanks.
M

質問済み 3年前361ビュー
1回答
0

Found it. The spark column type was string. Cast it and it works.

回答済み 3年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ