I'm trying to achieve data change capture using AWS Glue and don't want to use DMS. I'm trying to transfer data between two Oracle RDS instances which are in different AWS Account. Here I am trying to follow Delta approach (i.e., I am to transfer only updated data...)
From source, we are going to fetch records for which updated data of record is greater than last job run date and place all those records in Delta table and from Delta table, we wanted to manually upsert the records to target table based on primary key.
We are using cursors to do upsert operation and trying to import oracledb for Connection. I have added below dependencies in Python Library section in AWS Glue Job details:
<s3 location>/oracledb.zip,<s3 location>/ oracledb-2.0.1.dist-info.zip,s<s3 location>/ pycparser-2.21.dist-info.zip,s<s3 location>/ pycparser.zip,<s3 location>/cryptography.zip,<s3 location>/cryptography-41.0.7.dist-info.zip,s<s3 location>/cffi.zip,<s3 location>/ cffi-1.16.0.dist-info.zip
When I am trying to run the Glue job facing error like below:
Error:
ImportError: cannot import name 'base_impl' from partially initialized module 'oracledb' (most likely due to a circular Import)(/tmp/oracledb.zip/oracledb/init. py)
AWS OFFICIALUpdated 2 months ago
Thanks for solution, Do we have any option in glue where we can replicate deleted records as well. (ie, We have two AWS RDS Instances A and B. if Record is deleted in RDS A, I wanted it to be removed from RDS B as well. Does AWS Glue support this functionality.