By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Glue %connections magic causes error in %additional_python_modules

0

I am running pyspark on glue 3 notebook. The %additional_python_modules works well in the absence of %connections. But if I add %connections the job ignore %additional_python_modules and doesn't install the modules. Nevertheless, %connections also impair the secret manager call. Does anyone could help me?

Topics
AnalyticsDatabase
Tags
AnalyticsAWS GlueExtract Transform & Load Data
Language
English
Rafael Rossi
asked 8 months ago236 views
1 Answer
0

My guess is that when you add the connection, this brings a VPC network configuration, which doesn't allow access to the Pypi repository or Secrets managers. If I'm correct and the connection has a network configuration, check it has connectivity as needed.

profile pictureAWS
EXPERT
Gonzalo Herreros
answered 8 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content