Set up a DMS Task to copy a table with CLOB column in a S3 bucket

0

Hi, I have created a DMS task to copy a table with a clob column to a s3 bucket, the table gets copied, all the rows, but the clob column is getting truncated. Is there a way to get the entire CLOB column without it getting truncated?

Here is the error I am getting

2023-05-29T23:08:50 [SOURCE_UNLOAD ]W: The value of column 'CLOB_INFO' in table 'XXXXX.XXXX_CLOB' was truncated to length 32768 (oracle_endpoint_utils.c:2832)

asked 10 months ago598 views
3 Answers
0
Accepted Answer

IT seems the FULL LOB Mode is not supported for s3 buckets

Limitations to using Amazon S3 as a target The following limitations apply when using Amazon S3 as a target:

Don’t enable versioning for S3. If you need S3 versioning, use lifecycle policies to actively delete old versions. Otherwise, you might encounter endpoint test connection failures because of an S3 list-object call timeout. To create a lifecycle policy for an S3 bucket, see Managing your storage lifecycle. To delete a version of an S3 object, see Deleting object versions from a versioning-enabled bucket.

A VPC-enabled (gateway VPC) S3 bucket is supported in versions 3.4.7 and higher.

The following data definition language (DDL) commands are supported for change data capture (CDC): Truncate Table, Drop Table, Create Table, Rename Table, Add Column, Drop Column, Rename Column, and Change Column Data Type.

Note A truncate DDL operation removes all files and corresponding table folders from an S3 bucket. You can use task settings to disable that behavior and configure the way DMS handles DDL behavior during change data capture (CDC). For more information, see Task settings for change processing DDL handling.

Full LOB mode is not supported.

Changes to the source table structure during full load are not supported. Changes to data are supported during full load.

Multiple tasks that replicate data from the same source table to the same target S3 endpoint bucket result in those tasks writing to the same file. We recommend that you specify different target endpoints (buckets) if your data source is from the same table.

BatchApply is not supported for an S3 endpoint. Using Batch Apply (for example, the BatchApplyEnabled target metadata task setting) for an S3 target might result in loss of data.

answered 9 months ago
0

Hi, DMS has 2 modes for LOBs: truncated and full. You must select full mode to get all the data of the lob See https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.LOBSupport.html

profile pictureAWS
EXPERT
answered 10 months ago
0

When I go and try to create a task and go to setting , there is no option for full LOB mode.

Is there some place these settings have to be switched on, LOB Column Settings

answered 10 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions