SparkR not working
0
I am trying to control a Spark cluster (using SparkR) from a Sagemaker notebook. I followed these instructions closely: https://aws.amazon.com/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/ and got it to work.
Today when I try to run the SageMaker notebook (using the exact same code as before) I inexplicably get this error:
An error was encountered:
[1] "Error in callJMethod(sparkSession, \"read\"): Invalid jobj 1. If SparkR was restarted, Spark operations need to be re-executed."
Does anyone know why this is? I terminated the SparkR kernel and am still getting this error.
1 réponse
- Le plus récent
- Le plus de votes
- La plupart des commentaires
0
You cannot have multiple SparkContexts in one JVM. The issue is resolved as WON'T FIX. You have to stop the spark session which spawned the sparkcontext (which you have already done).
sparkR.session.stop()
répondu il y a 4 ans
Contenus pertinents
- demandé il y a 7 mois
- demandé il y a un an
- demandé il y a 7 mois
- demandé il y a 2 mois
AWS OFFICIELA mis à jour il y a un an- AWS OFFICIELA mis à jour il y a un an
- AWS OFFICIELA mis à jour il y a un an
