1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
Since you talk about modifying existing ones, I'm going to assume you are talking about JDBC bookmarks and not s3 (in s3 you cannot update rows without updating files).
You can as long as you meet the requirement of having a numeric column monotonically increasing (for instance using a column with the update timestamp/version), so the bookmark can keep track of which rows to read, and in the case of updates you have your own logic to do an upsert (for instance using a temporary table and then doing a DB upsert into the final table).
관련 콘텐츠
AWS 공식업데이트됨 3년 전- AWS 공식업데이트됨 일 년 전
- AWS 공식업데이트됨 2년 전
Thanks. Yes I'm talking about JDBC bookmarks. I have updatedAt columns. Can you give me more info about how they can be used with bookmarks to allow sending updated info to my db table?
You would need something that marks the updated rows (e.g. a trigger that updates the timestamp on the bookmark column) AND on the destination handle updates correctly (if you use something like Hudi that's done automatically, for others it might be more complicated)