Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
If my environment is full of Apache Hudi integrating with EMR and Lake Formation, I found out that Hudi environment is not very friendly to be used by Redshift nor Athena. There are many advanced...
1
answers
0
votes
434
views
asked 5 months agolg...
My customer is using AWS EMR and is storing all the Hive meta data on an external RDS instance, using MySQL 5.7.* And since MySQL 5.7 is running out of its lifecycle, we are pushing them to upgrade...
1
answers
1
votes
259
views
asked 6 months agolg...
Everyday a new emr cluster span up and terminated after completing the step job. Checking the cloudtrail, seems a Data Pipeline created it. I am not sure how to get more details like who created, what...
2
answers
1
votes
244
views
asked 6 months agolg...
I want to save my pyspark dataframe in RecordIO protobuf format. I am using Amazon EMR to run my pyspark scripts, and I want to use AWS SageMaker to train a machine learning model. SageMaker pipe mode...
1
answers
0
votes
184
views
asked 6 months agolg...
Hello,
I have a driver and executor pod template files exist in s3. But when I run the job, it failed wirh FileNotFoundException: s3:/<s3bucket>/podtemplate/driver.yaml (no such file or directory)...
1
answers
2
votes
233
views
asked 6 months agolg...
Hello,
Please share the difference between AWS Glue and AWS EMR and which one we should use and when?
Thanks,
1
answers
0
votes
1212
views
asked 6 months agolg...
In continuation to the ticket - https://repost.aws/questions/QUfxjbaGrXRTSKGy4-rnQ8Uw/how-to-upgrade-python-version-in-emr-since-python-3-7-support-discontinued , the latest version of EMR 6.14.0...
1
answers
0
votes
283
views
asked 6 months agolg...
I am facing issues using inline clustering and compaction in EMR, with the following error..
EMR : 6.13.0
Hudi: 0.13.1
com.esotericsoftware.kryo.KryoException: Unable to find class:...
0
answers
0
votes
96
views
asked 6 months agolg...
We have some parquet format files on AWS s3 and we want to create iceberg table with these files. Can Glue Crawler do this?
2
answers
0
votes
608
views
asked 6 months agolg...
I am using EMR 6.13.0(EMR on EC2) and it is present in eu-central-1 region. i have my s3 buckets in eu-central-1 and eu-west-1 region, i can be able to access the s3 buckets present in eu-central-1...
1
answers
1
votes
553
views
asked 6 months agolg...
I am trying to create a EMR cluster with version 6.13.0 and spark installed on it with version Spark 3.4.1 But the service role attached gives error that it does not have EC2 permissions.
I tried...
3
answers
0
votes
536
views
asked 6 months agolg...
EMR upgrade with AL2lg...
I have a emr 5.29 cluster running spark and hive application and received notification that AL1 will be unsupported after Dec 23. I am working on applications migration to emr 5.36. Questions,
1. What...
Accepted AnswerAmazon EMR
1
answers
1
votes
216
views
asked 6 months agolg...