Questions tagged with Amazon EMR Serverless
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi Team,
All examples that I am seeing are passing application configuration overrides as a dictionary. how to pass this value from a S3 path. This configuration will be pushed from airflow
1
answers
0
votes
45
views
asked 4 days agolg...
Hi Team,
We are invoking our EMR-S jobs using an airflow EMR job submit operator. The EMR application is configured with a set of spark default run time configurations. while invoking the job from...
2
answers
0
votes
66
views
asked 9 days agolg...
Let me know if this is something AWS EMR Studio does:
1. in Databricks community edition, and in Google Collab, one can fire up a simple Jupyter notrebook with an automatically started cluster (small...
1
answers
0
votes
144
views
asked 16 days agolg...
While running the serverless job run, I am getting below errror:
"Number of cores specified by 'spark.driver.cores '7' is invalid".
2
answers
0
votes
207
views
asked 20 days agolg...
I am executing pyspark script on EMR serverless and i am using AWS step function for creating EMR application and submitting job in it. I want to know how to add logging option in AWS step function...
1
answers
0
votes
391
views
asked a month agolg...
I have a Serverless EMR appication, I am submitting a spark job via python script. I have packaged all the dependencies an an the script to an s3 bucket. When I execute the job the spark job is...
2
answers
0
votes
302
views
asked a month agolg...
Made an emr application with custom resources - specifically changed executors have more cores and more memory (different amount of memeory from default change for that number of cores, but within the...
0
answers
0
votes
77
views
asked a month agolg...
Hello everyone,
I created a spark_ready.py module that hosts multiple classes that I want to use as a template. I've seen in multiple configurations online that using the "spark.submit.pyFiles" will...
2
answers
0
votes
292
views
asked a month agolg...
question deletedlg...
question deleted question deleted
0
answers
0
votes
105
views
asked a month agolg...
Is there an option to run parallel jobs in emr serverless application natively
Accepted AnswerAmazon EMR Serverless
2
answers
0
votes
327
views
asked a month agolg...
How do I connect EMR Serverless to read and write to a Microsoft SQL Server database hosted on EC2
1
answers
0
votes
185
views
asked a month agolg...
We are running into `No space left on device` errors in EMR Serverless for big jobs, even when setting driver / executor drive size to the maximum 200GB.
I tried to make the S3 shuffle storage...
1
answers
0
votes
145
views
asked 2 months agolg...