1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
Hi Bigbang
AWS AppFlow has Incremental transfers, which seem to be what you are looking for: see https://docs.aws.amazon.com/appflow/latest/userguide/flow-triggers.html
When you select incremental transfer, Amazon AppFlow transfers only the records that
have been added or changed since the last successful flow run. You can also select a
source timestamp field to specify how Amazon AppFlow identifies new or changed
records. For example, if you have a Created Date timestamp field, choose this to instruct
Amazon AppFlow to transfer only newly-created records (and not changed records) since the
last successful flow run. The first schedule-triggered flow will pull 30 days of past records
at the time of the first flow run.
Same page details the Offset option
Optionally, you can add a time offset (t) to the time range for the incremental transfer.
The flow run will import records that were created or changed between the previous flow
run and the specified offset prior to the current flow run. This feature can be used to accommodate
any latencies in the source systems in timestamping changes to records. By choosing a
sufficiently large offset, you can avoid missing records that changed in the source application close
to the run time of the scheduled flow.
If a schedule-triggered flow runs at time instances T0, T1, T2, and so on, then records that are new
or have changed between T0 minus t and T1 minus t will be imported from the source at T1, and
those that have changed between T1 minus t and T2 minus t will be imported from the source at T2.
So, by combining both, you should be able to achieve what you want.
Best, Didier
Contenus pertinents
- demandé il y a un an
- demandé il y a 8 mois
- demandé il y a 3 mois
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 2 ans
- AWS OFFICIELA mis à jour il y a 3 ans