Can we export the mongodb using the data pipeline?
Howdy. . Customer wants to use the data pipeline to schedule a job, exporting data from mongodb in json format, and then import it to the dynamodb. I checked the developer guide, but it says that the supported data nodes are dynamodb/mysql/redshift/s3.
My guess is that the customer should perform the export in mongodb native way to s3, and then, we can use the data pipeline to import the data to dynamodb.
Just give me a light on it. Thanks, in advance.
You don't need a specific data node to do this. You could schedule an EC2 instance with a shell command activity (http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-shellcommandactivity.html), which would contact the API endpoint of the mongo-cluster to call the export method, download the JSON, and upload it to DynamoDB.
Relevant questions
How to merge aws data pipeline output files into a single file?
asked 3 months agoPossible to save Honeycode data directly to Dynamodb
asked 4 months agoLoading json string data as super from Glue job in to Redshift
asked 6 months agoCan you use AWS Snowball for local data transfers without importing the data to Amazon S3?
Accepted Answerasked 2 years agoData Pipeline error when using RegEx data format
asked 2 months agoWhat is the best practice to load data to redshift with aws glue ?
asked 2 years agoCan we export the mongodb using the data pipeline?
Accepted Answerasked 6 years agoHow to import and export data from Amazon ML to RDS?
Accepted Answerasked 5 years agoMigrate or dump DocumentDB data
asked 3 years agoIs it possible to dynamically change the capacity-upfront for EMR cluster using Data Pipeline?
asked 5 months ago