Questions tagged with Amazon EMR
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi All,
I am creating an EMR cluster programmatically and calling the start_notebook_execution and attaching that cluster to the notebook. This works fine when I do manually. But programmatically the...
1
answers
0
votes
92
views
asked a year agolg...
Hello,
By default Glue run one executor per worker. I want to run more executors in worker. I have set following spark configuration in Glue params but It didn't work.
`--conf :...
1
answers
0
votes
2041
views
asked a year agolg...
I would like to get data from IceBerg table using AWS Lambda. I was able to create all the code and containers only to discover that AWS Lambda doesn't allow process substitution that spark uses here:...
2
answers
0
votes
1077
views
asked a year agolg...
JupyterHub on Amazon EMR comes with default PySpark kernel. How can I install additional libraries on this kernel (e.g. numpy)? I've tried following instructions on...
2
answers
0
votes
1663
views
asked a year agolg...
Folks:
I am running some code that uses a mix of PySpark (for data manipulation) and Python (for visualization). Very similar to this blog:...
1
answers
0
votes
210
views
asked 2 years agolg...
I am using EMR API, listSteps on boto3 python library. I assigned "1" to Marker item but recevied the error message, "Marker '1' is not valid."
What value is valid at Marker?
API :...
Accepted AnswerAmazon EMR
1
answers
0
votes
246
views
asked 2 years agolg...
HIVE_UNSUPPORTED_FORMAT: Output format org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat with SerDe org.openx.data.jsonserde.JsonSerDe is not supported. If a data manifest file was generated...
0
answers
0
votes
185
views
asked 2 years agolg...
I am working on enabling automatic patching for one of the EMR cluster. I understand that with Amazon EMR release 6.6 and later, when you launch new Amazon EMR clusters with the default Amazon Linux...
1
answers
0
votes
459
views
asked 2 years agolg...
Hi, I need to install Go packages that interact with my Spark script. Is it possible to do such things ?
2
answers
0
votes
1204
views
asked 2 years agolg...
Hi there,
Do you know how to stop a notebook on cell error?
It seems that the normal behavior is to continue on error, but I want to stop execution after the first error or fail,
Maybe a notebook...
1
answers
0
votes
216
views
asked 2 years agolg...
Trying to share data between two spark jobs in an EMR serverless application using temp or global temp views without having to write to s3 and then do read. It doesn't seem to work.
What is the...
1
answers
0
votes
281
views
asked 2 years agolg...
I am trying to read data from 3 node MongoDB cluster(replica set) using PySpark and
native python in AWS EMR. I am facing issues while executing the codes with in AWS EMR cluster as explained below...
2
answers
0
votes
659
views
asked 2 years agolg...