SparkR not working
0
I am trying to control a Spark cluster (using SparkR) from a Sagemaker notebook. I followed these instructions closely: https://aws.amazon.com/blogs/machine-learning/build-amazon-sagemaker-notebooks-backed-by-spark-in-amazon-emr/ and got it to work.
Today when I try to run the SageMaker notebook (using the exact same code as before) I inexplicably get this error:
An error was encountered:
[1] "Error in callJMethod(sparkSession, \"read\"): Invalid jobj 1. If SparkR was restarted, Spark operations need to be re-executed."
Does anyone know why this is? I terminated the SparkR kernel and am still getting this error.
1回答
- 新しい順
- 投票が多い順
- コメントが多い順
0
You cannot have multiple SparkContexts in one JVM. The issue is resolved as WON'T FIX. You have to stop the spark session which spawned the sparkcontext (which you have already done).
sparkR.session.stop()
回答済み 4年前
関連するコンテンツ
AWS公式更新しました 1年前
