Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
We have an EMR cluster with multiple concurrent steps gets executed seamlessly. Not sure what happened certainly, but the step logs, application logs are not published to s3 from yesterday....
Accepted AnswerAmazon EMR
3
answers
0
votes
372
views
asked 5 months agolg...
I am trying to use aws emr-serverless get-dashboard-for-job-run cli command to pull information from emr-serverless but am stumped. This command returns a url and auth token. If I go to the url, it...
0
answers
0
votes
133
views
asked 5 months agolg...
Hi,
after EMR 7.0.0 was released in the previous week, we wanted to start using it.
# Problem
We have shell script EMR steps that are executed during the start of the cluster. These EMR steps never...
2
answers
0
votes
370
views
asked 5 months agolg...
Hi,
One of my dev team members, asking to share the emr spark artifacts s3 location for building a Java application. I referred this doc...
Accepted AnswerAmazon EMR
1
answers
1
votes
185
views
asked 5 months agolg...
We have a airflow setup runs the EMR jobs daily basis. I noticed an odd behavior that when I resubmit job for calculating the adhoc reports, spark application failed with below error, arguments seems...
Accepted AnswerAmazon EMR
1
answers
0
votes
193
views
asked 6 months agolg...
When I try to create a new workspace for an AWS EMR Studio in the AWS Console, I get a blank page and a Javascript error in the console
("Failed to execute 'mark' on 'Performance':...
0
answers
0
votes
114
views
asked 6 months agolg...
I am trying to have glue data catalog as the hive metastore, stood up the EMR(emr-6.15.0) with the following node classification config per AWS, and it always initialize a default glue catalog...
1
answers
0
votes
492
views
asked 6 months agolg...
So I define manually finishing using the RunJobFlow operator (https://docs.aws.amazon.com/emr/latest/APIReference/API_RunJobFlow.html) `"KeepJobFlowAliveWhenNoSteps": True`. However, the cluster...
1
answers
0
votes
167
views
asked 6 months agolg...
I would like to know the log4j configuration to get container logs into more structured format like Json, so I can leverage another automation to parse the files and train some customization to filter...
2
answers
0
votes
433
views
asked 6 months agolg...
Hello,
I have upgraded the EMR from 6.14 to 6.15, and started seeing errors on the existing core node:
`org.apache.hadoop.fs.s3a.auth.NoAwsCredentialsException: IAMInstanceCredentialsProvider:...
1
answers
0
votes
123
views
asked 6 months agolg...
I am trying to connect to my documentDB trhough the spark-mongodb connector, but it looks like DocumentDB does not support Collstats. How disable the collstats command so i can do my transformations...
1
answers
0
votes
422
views
asked 6 months agolg...
How to add additional library i.e. databricks spark xml to a running EMR cluster and access it in Notebook
1
answers
0
votes
235
views
asked 6 months agolg...