Help! DynamoDB zero ETL integration with opensearch failing with error in cloudwatch

1

How can I resolve the following error and perform data ingestion from dynamodb to OpenSearch?

2024-02-08T12:32:51.077 [dynamodb-pipeline-sink-worker-2-thread-1] WARN org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink - Failed to initialize OpenSearch sink with a retryable exception. org.opensearch.client.opensearch._types.OpenSearchException: Request failed: [security_exception] no permissions for [cluster:monitor/state] and User [name=arn:aws:iam::152583569487:role/OpensearchTrustRole, backend_roles=[arn:aws:iam::dhfdh:role/OpensearchTrustRole], requestedTenant=null] at org.opensearch.client.transport.aws.AwsSdk2Transport.parseResponse(AwsSdk2Transport.java:473) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.client.transport.aws.AwsSdk2Transport.executeSync(AwsSdk2Transport.java:392) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.client.transport.aws.AwsSdk2Transport.performRequest(AwsSdk2Transport.java:192) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.client.opensearch.cluster.OpenSearchClusterClient.getSettings(OpenSearchClusterClient.java:289) ~[opensearch-java-2.8.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkISMEnabled(AbstractIndexManager.java:202) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.checkAndCreateIndexTemplate(AbstractIndexManager.java:229) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.index.AbstractIndexManager.setupIndex(AbstractIndexManager.java:224) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitializeInternal(OpenSearchSink.java:231) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.plugins.sink.opensearch.OpenSearchSink.doInitialize(OpenSearchSink.java:193) ~[opensearch-2.6.1.jar:?] at org.opensearch.dataprepper.model.sink.AbstractSink.initialize(AbstractSink.java:52) ~[data-prepper-api-2.6.1.jar:?] at org.opensearch.dataprepper.pipeline.Pipeline.isReady(Pipeline.java:200) ~[data-prepper-core-2.6.1.jar:?] at org.opensearch.dataprepper.pipeline.Pipeline.lambda$execute$2(Pipeline.java:252) ~[data-prepper-core-2.6.1.jar:?] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?] at java.base/java.lang.Thread.run(Thread.java:829) [?:?]

2 Antworten
1

You will need to map the arn:aws:iam::152583569487:role/OpensearchTrustRole role as a back end user in opensearch. https://opensearch.org/docs/latest/security/access-control/users-roles/#map-users-to-roles

profile pictureAWS
EXPERTE
beantwortet vor 3 Monaten
0

1/ Ensure the DynamoDB table and OpenSearch cluster are in the same AWS region. The ingestion integration currently does not support cross-region operations.

2/ Check for any mapping conflicts between the data types in DynamoDB and OpenSearch. The integration automatically maps types but a mix of data could cause failures. An explicit mapping template may be needed.

AWS
Phani_L
beantwortet vor 3 Monaten
    1. The region is the same for both DynamoDB and OpenSearch.
    2. How can I identify mapping conflicts? I dont see any error mentioning mapping in the logs. Also how can I add explicit mapping template to the ingestion pipeline?

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen