Help! DynamoDB zero ETL integration with opensearch failing with error in cloudwatch

1

How can I resolve the following error and perform data ingestion from dynamodb to OpenSearch?

2024-02-08T12:32:51.077 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Failed to initialize OpenSearch sink with a retryable exception. org.opensearch.client.opensearch._types.OpenSearchException: Request failed: [security_exception] no permissions for [cluster:monitor/state] and User [name=arn:aws:iam::152583569487:role/OpensearchTrustRole, backend_roles=[arn:aws:iam::dhfdh:role/OpensearchTrustRole], requestedTenant=null] at org.opensearch.client.transport.aws.AwsSdk2Transport.parseResponse(AwsSdk2Transport.java:473) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.client.transport.aws.AwsSdk2Transport.executeSync(AwsSdk2Transport.java:392) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.client.transport.aws.AwsSdk2Transport.performRequest(AwsSdk2Transport.java:192) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.client.opensearch.cluster.OpenSearchClusterClient.getSettings(OpenSearchClusterClient.java:289) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkISMEnabled(AbstractIndexManager.java:202) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkAndCreateIndexTemplate(AbstractIndexManager.java:229) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.setupIndex(AbstractIndexManager.java:224) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitializeInternal(OpenSearchSink.java:231) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitialize(OpenSearchSink.java:193) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.model.sink.AbstractSink.initialize(AbstractSink.java:52) ~[data-prepper-api-2.6.1.jar:?] at org.opensearch.dataprepper.pipeline.Pipeline.isReady(Pipeline.java:200) ~[data-prepper-core-2.6.1.jar:?] at org.opensearch.dataprepper.pipeline.Pipeline.lambda$execute$2(Pipeline.java:252) ~[data-prepper-core-2.6.1.jar:?] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.base/java.lang.Thread.run(Thread.java:829) [?:?]

2 Answers
1

You will need to map the arn:aws:iam::152583569487:role/OpensearchTrustRole role as a back end user in opensearch. https://opensearch.org/docs/latest/security/access-control/users-roles/#map-users-to-roles

profile pictureAWS
EXPERT
answered 2 months ago
0

1/ Ensure the DynamoDB table and OpenSearch cluster are in the same AWS region. The ingestion integration currently does not support cross-region operations.

2/ Check for any mapping conflicts between the data types in DynamoDB and OpenSearch. The integration automatically maps types but a mix of data could cause failures. An explicit mapping template may be needed.

AWS
Phani_L
answered 3 months ago
    1. The region is the same for both DynamoDB and OpenSearch.
    2. How can I identify mapping conflicts? I dont see any error mentioning mapping in the logs. Also how can I add explicit mapping template to the ingestion pipeline?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions