Export data from AWS ElasticSearch to AWS RDS instance using Logstash

Question

We have Amazon ElasticSearch Cluster of version 7.10.0. We would like to export some data from ElasticSearch to RDS Table. We are already using Logstash to import some data into ElasticSearch. There is amazon_es plugin which can only be used as output plugin in logstash pipeline but for export, we are not able to find anything.

In our use case, there is some summarize data in Amazon ElasticSearch which we want to insert into RDS table. Let us know how can we do it?

Answer

Hi there, currently logstash and AWS DMS only supports ES as the destination but not source. One closest option to achieve your use case is to use AWS Lambda to read data from Elasticsearch (1) and then insert data to Amazon RDS(2)

(1) https://blogs.tensult.com/2020/01/01/aws-lambda-to-perform-various-tasks-in-elasticsearch/

(2) https://docs.aws.amazon.com/lambda/latest/dg/services-rds-tutorial.html

Export data from AWS ElasticSearch to AWS RDS instance using Logstash

Relevant questions

What features of Open Distro is supported by ElasticSearch?

Quicksight export more than 1 million rows data to multiple csv

Export data from AWS ElasticSearch to AWS RDS instance using Logstash

Problem with deadlock on AWS RDS MySQL

Aws athena- query both s3 and rds

using ingest-geoip or Nginx filebeat/metricbeat module on Amazon ElasticSearch

What is the best practice to load data to redshift with aws glue ?

Logging with CloudWatch vs. ElasticSearch/Kibana

Can we export the mongodb using the data pipeline?

pagination question on appsync to elasticsearch