- Mais recentes
- Mais votos
- Mais comentários
Hello,
For this use case, the bookmark keys are used at data consuming side, and per documented at [1], the create_dynamic_frame.from_catalog just takes the column names for the "jobBookmarkKeys". There is no option to give a default value to a column when its value is null.
However, there is workaround to this.
If your original table is stored in an RDBMS system, then you can add a computed column, which has same value as the original column, and has a default value where the original is null.
Then in your glue job, you can use the computed column as part of bookmark keys.
Hope it helps.
=========
Reference: [1] - https://docs.aws.amazon.com/glue/latest/dg/monitor-continuations.html
Hello, I wanted to know if using this solution is feasible, since a condition for user-defined bookmark keys is that the field be strictly monotonically increasing, and in this case by assigning a default value to null cases then it is not possible. would meet this condition.
Our particular case is that we have a database table with millions of records that did not have the updated_at field, the idea is to incorporate it and assign the current timestamp to the existing records and assign the current_timestamp to the new records or modifications, however, We are not sure that job bookmark accepts this column because previous records will all have the same timestamp value. I would appreciate your help with this question.
Conteúdo relevante
- AWS OFICIALAtualizada há 2 meses
- AWS OFICIALAtualizada há 2 anos
Where is that origin table stored?