- 最新
- 投票最多
- 评论最多
conda_python3 and conda_tensorflow_p36 are local kernels on the SageMaker notebook instance while the Spark kernels execute remotely in the Glue Spark environment.
Hence you are seeing different versions. The Glue Spark environment comes with 1.4.1 version of scipy. So when you use the PySpark (python) or Spark (scala) kernels and you will get the 1.4.1 version of scipy.
If you use the default LifeCycle script that Glue SageMaker notebooks already come with, the connectivity to the Glue Dev endpoint should be in place. Note that the Glue SageMaker notebooks has a tag called 'aws-glue-dev-endpoint' that is used to identify which Glue Dev endpoint that particular notebook instance communicates with.
The Spark kernels cannot be replicated via the python shell. Those kernels relay Spark commands via the Livy service to Spark on the Glue Dev endpoint using a Jupyter module called Sparkmagic.
相关内容
- AWS 官方已更新 2 年前
- AWS 官方已更新 2 年前
- AWS 官方已更新 2 年前