Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hello Everyone,
I am facing difficulties while trying to load data from an Amazon Redshift cluster into a Spark dataframe using the included redshift connector from **EMR release 6.13.0** .
My...
1
answers
0
votes
573
views
asked 8 months agolg...
We want to call an service(ECS based) API at scale around 250K tps using AWS , we are looking for suggestions and want to evaluate can we do the same in EMR or distributed map.
Please let me know if...
1
answers
0
votes
222
views
asked 8 months agolg...
I would like to know the way/step to inspect the running driver/executor pods in emr on eks cluster. Do not find any steps specific to this use case. Please help me out. Thanks in advance.
1
answers
2
votes
294
views
asked 8 months agolg...
D1= spark.createDataframe("a",["b"])
Dd=d1.exceptAll(D1).count()
Running the above in emr.6.8 throwing below error,
Java.lang.IllegalException: couldn't find value #400
Accepted AnswerAmazon EMR
2
answers
0
votes
313
views
asked 8 months agolg...
I am trying to connect with Mysql DB instance (inside RDS) using IAM permissions from EMR(pyspark)
for test purpose i have provided policy 'AmazonRDSFullAccess' to my EMR role, and trying below python...
1
answers
0
votes
243
views
asked 8 months agolg...
HBase table (manual)lg...
I like to create a table in HBase from EMR cluster.
however, after executing HBase shell , It gives the following error.
$ hbase shell
hbase:001:0> create 'test', 'data'
ERROR: KeeperErrorCode =...
1
answers
0
votes
277
views
asked 8 months agolg...
When creating the SC, it keeps failing with fatal error though restarted the kernel multiple times. As I am data analyst, not sure what else to check in terms of resources
Accepted AnswerAmazon EMR
1
answers
0
votes
214
views
asked 8 months agolg...
I have an EMR cluster running for MWAA, it runs perfectly fine. But two days ago it stopped showing any metrics on the Monitoring tab as well as CloudWatch metrics on the console. So it doesn't seem...
1
answers
2
votes
342
views
asked 8 months agolg...
1. **Spun up an EMR instance:**
emr-6.10.0
Spark 3.3.1, HBASE 2.4.15, Hive 3.1.3, JupyterHub 1.5.0, Hadoop 3.3.3, ZooKeeper 3.5.10, Zeppelin 0.10.1, Phoenix 5.1.2, Presto 0.278,
...
1
answers
1
votes
299
views
asked 8 months agolg...
Getting this error when trying to run a simple spark job which reads a json file from s3 and prints the...
3
answers
0
votes
527
views
asked 9 months agolg...
I am using EMR 6.13.0, it is using python 3.7. in my code i have used boto3, the boto3 support for python 3.7 will be discontinued from December-2023.
and as we aware the python 3.7 support stopped...
2
answers
1
votes
4420
views
asked 9 months agolg...
We currently have an EMR cluster with the next configuration.
- Master instanceFleet ( 1 node r5a.xlarge on demand )
- Core instanceFleet ( 32 on demand units from multiple instance types )
- Task...
1
answers
0
votes
213
views
asked 9 months agolg...