Line magics in Glue Docker container not found

I am interested particularly in %additional_python_modules and I always get this error:

UsageError: Line magic function %additional_python_modules not found.

The same error is thrown when I try other magics:
UsageError: Line magic function %idle_timeout not found.
UsageError: Line magic function %worker_type not found..

I'm using amazon/aws-glue-libs:glue_libs_4.0.0_image_01 image

主题

分析数据库

标签

AWS Glue 提取变换和加载数据

语言

English

siyala

已提问 2 个月前121 查看次数

2 回答

最新
投票最多
评论最多

这些答案有用吗？为正确答案投票，以帮助社区从您的知识中受益。

已接受的回答

The magics are for the Glue Interactive session Kernel when used as part of a Jupyter notebook.
It has nothing to do with the docker container. In the container you can just use pip to install dependencies inside it (but will be gone when you stop the container).

专家

Gonzalo Herreros

已回答 2 个月前

siyala
2 个月前
Thanks, Gonzalo. Understood. However, I'm trying to install my own wheel package, for e.g.:

%additional_python_modules s3://library/simple_salesforce-1.12.1-py2.py3-none-any.whl

but the above error is thrown for the %additional_python_modules or any other for that matter. The same magics work inside of a Jupyter notebook when run in an interactive session
Gonzalo Herreros 专家
2 个月前
Yes, as I said magics only make sense inside of a Jupyter notebook when run in an interactive session

to close this one, I uploaded my package to PyPI and now I can import it into my notebook running in Docker. Thank you!

siyala

已回答 2 个月前

Line magics in Glue Docker container not found

相关内容