AWS Builder Center: Learn, Build and Connect with builders in the AWS community

AWS Builder Center is the official home for builders on AWS. Share and read what others are working on, follow people who inspire you, explore training and workshops, and find tools to support what you're building.

Running pyspark jobs on EMR serverless with libraries/dependencies for optimized performance

Hey Guys

I want to run my pyspark on EMR Serverless but it has some dependencies/libraries which are needed by the pyspark script to run. Please suggest a optimized approach to import the libraries/dependencies on EMR Serverless. I want to run the jobs with minimum run time possible.

Thanks

Argomenti: Analisi Database
Tag: Amazon EMR Serverless Estrai, trasforma e carica i dati
Lingua: English

Jose

posta 3 anni fa883 visualizzazioni

1 Risposta

Più recenti
Maggior numero di voti
Maggior numero di commenti

Queste risposte sono utili? Dai un voto positivo alla risposta corretta per aiutare la community a trarre vantaggio dalle tue conoscenze.

Take a look at https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html

cisdevry

con risposta 3 anni fa

Running pyspark jobs on EMR serverless with libraries/dependencies for optimized performance

Aggiungi la tua risposta

Contenuto pertinente