Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
1. The Issue:
We have a Spark EMR cluster which connects to a remote hive metastore to use our emr hive data warehouse.
When executing Pyspark statement in Zeppelin notebook: sc.sql("create table...
1
answers
0
votes
2689
views
asked 2 years agolg...
What is the AWS recommended way for using Kinesis as a Source with Apache Spark Structured Streaming if running Spark jobs on EMR (not AWS Glue).
There is a [third party open source connector...
0
answers
0
votes
130
views
asked 2 years agolg...
Hi Team,
getting issue while creating EMR CLUSTER. using AWS CLI Script.
Unknown options: --step-concurrency-level,5.
Details:
Spark Application
--release-label emr-5.30.0
Please let me know if...
1
answers
0
votes
398
views
asked 2 years agolg...
I created the tutorial from this...
1
answers
0
votes
563
views
asked 2 years agolg...
EMR 6.7.0 has updated Presto to 0.272, which includes the new [Delta Lake connector](https://docs.delta.io/latest/presto-integration.html) (added on 0.269). However, a new property file needs to be...
2
answers
0
votes
256
views
asked 2 years agolg...
Hi,
I have created a workspace in EMR studio using an endpoint on EMR on EKS as per below (based on the instructions here...
1
answers
1
votes
728
views
asked 2 years agolg...
I've tried to use the pip install librosa command within a jupyter notebook, yet I get a modulenotfounderror. Is there a place where I can use pip install librosa?
Thanks.
1
answers
0
votes
794
views
asked 2 years agolg...
Hi everyone,
I am a bit new to AWS so pardon me on this. I am trying to run Spark on a notebook in EMR. I have created a cluster to achieve this effect. However, when I tried to select Pyspark or...
1
answers
0
votes
660
views
asked 2 years agolg...
EMR on EC2 provides job flow metadata via `/mnt/var/lib/info/job-flow.json` file on disk. However, this doesn't exist on EMR on EKS. Is there an equivalent metadata provider to get Id of the job run?
1
answers
0
votes
545
views
asked 2 years agolg...
Currently for EMR on EC2 we pass in job configuration using a local file `my.jar --cfg /local/path/config.cfg ....`. Now I'm migrating jobs to both EMR on EKS and EMR Serverless. How can I achieve the...
1
answers
0
votes
488
views
asked 2 years agolg...
Hello,
How do we pass application.conf or properties file to spark jobs on emr serverless, in standard EMR we copy properties or conf files to master node /home/hadoop/?
1
answers
0
votes
667
views
asked 2 years agolg...
Hello,
Is it possible to configure EMR service to use the AWS KMS FIPS endpoint (https://ksm-fips.us-gov-east-1.amazonaws.com ) instead of the non-FIPS endpoint...
1
answers
0
votes
275
views