Questions tagged with Amazon EMR Serverless
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I have created a Spark job with Scala and I am trying to find the way to get the logs into cloudwatch.
So far I have tried to package the job as an uber Jar with a [cloudwatch...
Accepted AnswerAmazon EMR Serverless
1
answers
0
votes
1333
views
asked 2 years agolg...
Hi,
We are planning to make use of EMR Serverless Hive service to move one of our work load from EMR persistent cluster. This particular workload uses the external XML serde to define hive table. In...
1
answers
0
votes
784
views
asked 2 years agolg...
EMR on EC2 provides job flow metadata via `/mnt/var/lib/info/job-flow.json` file on disk. However, this doesn't exist on EMR on EKS. Is there an equivalent metadata provider to get Id of the job run?
1
answers
0
votes
532
views
asked 2 years agolg...
Currently for EMR on EC2 we pass in job configuration using a local file `my.jar --cfg /local/path/config.cfg ....`. Now I'm migrating jobs to both EMR on EKS and EMR Serverless. How can I achieve the...
1
answers
0
votes
475
views
asked 2 years agolg...
I know Step Functions use AWS SDK and it gets updated regularly so my questions:
1. What is the SDK update cycle for Step Functions?
2. Do we know if EMR Serverless can be integrated without using...
1
answers
1
votes
512
views
asked 2 years agolg...
Hello,
How do we pass application.conf or properties file to spark jobs on emr serverless, in standard EMR we copy properties or conf files to master node /home/hadoop/?
1
answers
0
votes
651
views
asked 2 years agolg...
I am using both pyspark and local python kernel (%%local) in a single EMR notebook. I am able to install packages successfully in pyspark kernel using EMR bootstrap but I am unable to install...
2
answers
1
votes
3652
views
asked 2 years agolg...
I am trying to invoke a spark script that runs over EMR serverless from a Step function. Nevertheless, I have not found an out of the box integration. One option is to use a lambda function that...
1
answers
1
votes
1968
views
asked 2 years agolg...
Is it possible to run a JDK11 application on EMR serverless? I tried and got an error:
```
Job execution failed, please check complete logs in configured logging destination. ExitCode: 1. Last few...
Accepted AnswerAmazon EMR Serverless
4
answers
0
votes
2307
views
asked 2 years agolg...
I'm trying to create an isolated Python virtual environment to package Python libraries necessary for a Pyspark job.
I was successful to make it work by simply following these steps...
2
answers
0
votes
968
views
asked 2 years agolg...
Just trying to figure out the time duration to setup the "Time Duration to Setup Server less Aurora V2 Postgre SQL database cluster" with single DB instance in a private subnet "Default settings",...
1
answers
0
votes
368
views
asked 2 years agolg...
Is it possible to check how much instance capacity exists for a given region and AZ?
Use case: trying to determine the best instance types to choose to have the highest likelihood of reservation...
2
answers
0
votes
1191
views
asked 2 years agolg...