By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Running pyspark jobs on EMR serverless with libraries/dependencies for optimized performance

0

Hey Guys

I want to run my pyspark on EMR Serverless but it has some dependencies/libraries which are needed by the pyspark script to run. Please suggest a optimized approach to import the libraries/dependencies on EMR Serverless. I want to run the jobs with minimum run time possible.

Thanks

Topics
DatabaseAnalytics
Tags
Extract Transform & Load DataAmazon EMR Serverless
Language
English
Jose
asked 9 months ago345 views
1 Answer
0

Take a look at https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html

cisdevry
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content