Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
If my environment is full of Apache Hudi integrating with EMR and Lake Formation, I found out that Hudi environment is not very friendly to be used by Redshift nor Athena. There are many advanced...
1
answers
0
votes
423
views
asked 5 months agolg...
My customer is using AWS EMR and is storing all the Hive meta data on an external RDS instance, using MySQL 5.7.* And since MySQL 5.7 is running out of its lifecycle, we are pushing them to upgrade...
1
answers
1
votes
242
views
asked 5 months agolg...
Everyday a new emr cluster span up and terminated after completing the step job. Checking the cloudtrail, seems a Data Pipeline created it. I am not sure how to get more details like who created, what...
2
answers
1
votes
232
views
asked 5 months agolg...
I want to save my pyspark dataframe in RecordIO protobuf format. I am using Amazon EMR to run my pyspark scripts, and I want to use AWS SageMaker to train a machine learning model. SageMaker pipe mode...
1
answers
0
votes
178
views
asked 5 months agolg...
Hello,
I have a driver and executor pod template files exist in s3. But when I run the job, it failed wirh FileNotFoundException: s3:/<s3bucket>/podtemplate/driver.yaml (no such file or directory)...
1
answers
2
votes
228
views
asked 5 months agolg...
Hello,
Please share the difference between AWS Glue and AWS EMR and which one we should use and when?
Thanks,
1
answers
0
votes
1154
views
asked 5 months agolg...
In continuation to the ticket - https://repost.aws/questions/QUfxjbaGrXRTSKGy4-rnQ8Uw/how-to-upgrade-python-version-in-emr-since-python-3-7-support-discontinued , the latest version of EMR 6.14.0...
1
answers
0
votes
267
views
asked 6 months agolg...
I am facing issues using inline clustering and compaction in EMR, with the following error..
EMR : 6.13.0
Hudi: 0.13.1
com.esotericsoftware.kryo.KryoException: Unable to find class:...
0
answers
0
votes
93
views
asked 6 months agolg...
We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?
2
answers
0
votes
582
views
asked 6 months agolg...
I am using EMR 6.13.0(EMR on EC2) and it is present in eu-central-1 region. i have my s3 buckets in eu-central-1 and eu-west-1 region, i can be able to access the s3 buckets present in eu-central-1...
1
answers
1
votes
533
views
asked 6 months agolg...
I am trying to create a EMR cluster with version 6.13.0 and spark installed on it with version Spark 3.4.1 But the service role attached gives error that it does not have EC2 permissions.
I tried...
3
answers
0
votes
516
views
asked 6 months agolg...
EMR upgrade with AL2lg...
I have a emr 5.29 cluster running spark and hive application and received notification that AL1 will be unsupported after Dec 23. I am working on applications migration to emr 5.36. Questions,
1. What...
Accepted AnswerAmazon EMR
1
answers
1
votes
212
views
asked 6 months agolg...