1 Answer
- Newest
- Most votes
- Most comments
0
Since you talk about modifying existing ones, I'm going to assume you are talking about JDBC bookmarks and not s3 (in s3 you cannot update rows without updating files).
You can as long as you meet the requirement of having a numeric column monotonically increasing (for instance using a column with the update timestamp/version), so the bookmark can keep track of which rows to read, and in the case of updates you have your own logic to do an upsert (for instance using a temporary table and then doing a DB upsert into the final table).
Relevant content
- Accepted Answerasked 5 years ago
AWS OFFICIALUpdated a year ago- AWS OFFICIALUpdated 7 months ago
- AWS OFFICIALUpdated a year ago
Thanks. Yes I'm talking about JDBC bookmarks. I have updatedAt columns. Can you give me more info about how they can be used with bookmarks to allow sending updated info to my db table?
You would need something that marks the updated rows (e.g. a trigger that updates the timestamp on the bookmark column) AND on the destination handle updates correctly (if you use something like Hudi that's done automatically, for others it might be more complicated)