By using AWS re:Post, you agree to the Terms of Use

ECS containers running on private subnet can not reach elastic search

0

I have an ECS cluster running containers on private subnet. My containers are running dotnet core applications and configured to send the logs to elastic search endpoint. They are running on private subnet and also with a NAT gateway.

I have an elasticsearch cluster running also on aws with an ip based access policy. How can I provide access to elastic search to the ecs containers running on private subnets.

asked 3 years ago45 views
5 Answers
0
Accepted Answer

Hi,
for your IP-based access policy, "Allow access to the domain from specific IP(s)", have you added the public elastic IP address that is assigned to your NAT Gateway?
-randy

answered 3 years ago
0

Hi,
To confirm, the ECS cluster is running in a private subnet in one VPC and the Elasticsearch cluster in a another private subnet in a different VPC and you want the ECS cluster to communicate to the Elasticsearch endpoint over private IP address space without going across the public internet?

If yes, do both VPCs have non-overlapping CIDR blocks? If yes, I would recommend reading up on how to setup up AWS VPC Peering. More info available here:
https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html
If this is not what you wanted, let me know.
-randy

answered 3 years ago
0

Thank you Randy for your answer.

So Basically the ES is running not on VPC access but on a public access. And secured via ip policy.

However ECS are running on our VPC on private subnets.

When i describe my es domain I cant see any information about the VPC and subnets and azs.
aws es describe-elasticsearch-domain --domain-name xxxxxx

How do i get the information about the ES domain running with public access? When I setup ES domain I did not find any option to select the VPC so does that mean it runs on Default VPC?

answered 3 years ago
0

Hi,
If your requirements allow you to have Elasticsearch running in a VPC, I would highly recommend that you re-install/or migrate the data from a public endpoint to a VPC so that you don't have to traverse the internet to use Elasticsearch. Note: "When you create a domain, you specify whether it should have a public endpoint or reside within a VPC. Once created, you cannot switch from one to the other. Instead, you must create a new domain and either manually reindex or migrate your data. Snapshots offer a convenient means of migrating data. ."
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-vpc.html#es-migrating-public-to-vpc

When initially creating your Domain, step 12 in the following link shows:
https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg-create-domain.html

For simplicity in this tutorial, we recommend an IP-based access policy. On the Set up access page, in the Network configuration section, choose Public access.

Here is where you can choose the VPC option instead of Public access.

So, let me know if you want help to troubleshoot the current configuration, or if you want to re-install/migrate to the VPC and then see if that works.

-randy

answered 3 years ago
0

Hi,

Yeah I am well aware of the vpc based access elastic search, but the problem is not all of our infrastructure is migrated to AWS completely.

We are in the middle of migration and we still at least need around 6 months to completely in AWS. And since we are using elastic search public access that are running on our on premise application, it is not possible at a moment.

And IF i am correct we can not access vpc based elasticsearch outside the VPC. Let me know if we can access VPC based elastic search somehow outside the VPC as well so our non aws applications still can use & access it.

Anyway, thanks for the information. So think the best would be to wait until if we have everything inside aws environment and we migrate to VPC bases ES cluster.

Thanks again

answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions