Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

How do I set up cross-account streaming from Kinesis Data Firehose to Amazon OpenSearch Service (vpc based access)?

0

I have a Opensearch inside the VPC running in account A.

I want to deliver logs from Firehose in Account B to the Opensearch in Account A.

Is it possible? (i saw the documentation here for publicly accessibly opensearch https://repost.aws/knowledge-center/kinesis-firehose-cross-account-streaming )

When I try to create delivery stream from AWS CLI or terraform I am getting below exception,

An error occurred : InvalidArgumentException: Verify that the IAM role has access to the ElasticSearch domain. The same IAM role, and same input.json works when modified to the Opensearch in Account B. I have Transit gateway connectivity enabled between the AWS accounts and I can connect telnet to the Opensearch in Account A from EC2 instance in Account B.

  • Security group for Opensearch domain, allows the Account B VPC (443) access rule. (I'm not using data transformation feature)
Topics
Analytics
Tags
Amazon Data FirehoseAmazon OpenSearch Service
Language
English
asked 2 years ago767 views
1 Answer
0

Check your OpenSearch resource-based policy. It should allow actions from your Kinesis role

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/ac.html#ac-types-resource

EXPERT
answered 2 years ago
EXPERT
reviewed 2 years ago
  • Mounika
    2 years ago
    • I already included the Resource based policy on Openserach domain in Account-A like this, but same error
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": "es:*",
      "Resource": "<ES Domain ARN in Account-A>/*",
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "<Firehose IAM Role ARN in Account B>"
      },
      "Action": [
        "es:ESHttpPost",
        "es:ESHttpPut"
      ],
      "Resource": [
        "<ES Domain ARN in Account-A>",
        "<ES Domain ARN in Account-A>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "<Firehose IAM Role ARN in Account-B>"
      },
      "Action": "es:ESHttpGet",
      "Resource": [
        "<ES Domain ARN in Account-A>/_all/_settings",
        "<ES Domain ARN in Account-A>/_cluster/stats",
        "<ES Domain ARN in Account-A>/index-name*/_mapping/type-name",
        "<ES Domain ARN in Account-A>/roletest*/_mapping/roletest",
        "<ES Domain ARN in Account-A>/_nodes",
        "<ES Domain ARN in Account-A>/_nodes/stats",
        "<ES Domain ARN in Account-A>/_nodes/*/stats",
        "<ES Domain ARN in Account-A>/_stats",
        "<ES Domain ARN in Account-A>/index-name*/_stats",
        "<ES Domain ARN in Account-A>/roletest*/_stats"
      ]
    }
  ]
}
````

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content