Questions tagged with Amazon EMR Serverless
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I am running a Scala spark job on EMR serverless and am trying to pass a postgresql JDBC connector jar through Spark's --driver-class-path config. This is how my spark submit configs look
```
--class...
2
answers
0
votes
46
views
asked 4 days agolg...
I'm running an EMR Serverless Spark job that uses Delta OSS to handle Delta tables. I previously resolved a configuration issue with EMR Serverless and AWS Glue Data Catalog...
1
answers
0
votes
148
views
asked 7 days agolg...
Hi all,
I have shared a Glue table (S3) with another account where I can already query it via Athena.
Now I added LakeFormation permissions for the database and table to the role that I am using...
1
answers
0
votes
95
views
asked 9 days agolg...
I want to use AWS Glue Data Catalog as a metastore. I'm running an EMR Serverless job that inserts and updates data in a Delta Table. I've successfully populated Delta tables on my localhost...
2
answers
0
votes
96
views
asked 9 days agolg...
Hi Team,
All examples that I am seeing are passing application configuration overrides as a dictionary. how to pass this value from a S3 path. This configuration will be pushed from airflow
1
answers
0
votes
183
views
asked 17 days agolg...
Hi Team,
We are invoking our EMR-S jobs using an airflow EMR job submit operator. The EMR application is configured with a set of spark default run time configurations. while invoking the job from...
2
answers
0
votes
200
views
asked 22 days agolg...
Let me know if this is something AWS EMR Studio does:
1. in Databricks community edition, and in Google Collab, one can fire up a simple Jupyter notrebook with an automatically started cluster (small...
1
answers
0
votes
295
views
asked a month agolg...
While running the serverless job run, I am getting below errror:
"Number of cores specified by 'spark.driver.cores '7' is invalid".
2
answers
0
votes
354
views
asked a month agolg...
I am executing pyspark script on EMR serverless and i am using AWS step function for creating EMR application and submitting job in it. I want to know how to add logging option in AWS step function...
1
answers
0
votes
463
views
asked a month agolg...
I have a Serverless EMR appication, I am submitting a spark job via python script. I have packaged all the dependencies an an the script to an s3 bucket. When I execute the job the spark job is...
2
answers
0
votes
456
views
asked a month agolg...
Made an emr application with custom resources - specifically changed executors have more cores and more memory (different amount of memeory from default change for that number of cores, but within the...
0
answers
0
votes
130
views
asked a month agolg...
Hello everyone,
I created a spark_ready.py module that hosts multiple classes that I want to use as a template. I've seen in multiple configurations online that using the "spark.submit.pyFiles" will...
2
answers
0
votes
393
views
asked 2 months agolg...