By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Data sharing between EMR Serverless Jobs

0

Trying to share data between two spark jobs in an EMR serverless application using temp or global temp views without having to write to s3 and then do read. It doesn't seem to work.

What is the recommended approach?

Topics
Analytics
Tags
Amazon EMRAmazon EMR Serverless
Language
English
syd
asked 2 years ago390 views
1 Answer
0

In EMR Serverless you can enable shared applications. When you submit jobs, you can specify IAM role permissions that a EMR Serverless job run can assume when calling other services on your behalf. This includes access to Amazon S3 for any data sources, targets, as well as other AWS resources like Amazon Redshift clusters and DynamoDB tables. To learn more about how to create a role, see https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/getting-started.html#gs-runtime-role

cisdevry
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content