My Glue 4.0 jobs have suddenly stopped working with error message below. As it is related to boto3, I am unable to make any changes to library config. Pls advise.
NB: I noticed that urllib3 released a new version on 20 Sept 2023.
2023-09-27 07:43:09,669 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(77)):
Error from Python:Traceback (most recent call last):
File "/tmp/teset.py", line 9, in <module>
from utils import *
File "/tmp/localPyFiles-bd01bc44-3026-4af7-a56c-3e5c212b0c30/teset.py", line 7, in <module>
import boto3
File "/home/spark/.local/lib/python3.10/site-packages/boto3/init.py", line 17, in <module>
from boto3.session import Session
File "/home/spark/.local/lib/python3.10/site-packages/boto3/session.py", line 17, in <module>
import botocore.session
File "/home/spark/.local/lib/python3.10/site-packages/botocore/session.py", line 26, in <module>
import botocore.client
File "/home/spark/.local/lib/python3.10/site-packages/botocore/client.py", line 15, in <module>
from botocore import waiter, xform_name
File "/home/spark/.local/lib/python3.10/site-packages/botocore/waiter.py", line 18, in <module>
from botocore.docs.docstring import WaiterDocstring
File "/home/spark/.local/lib/python3.10/site-packages/botocore/docs/init.py", line 15, in <module>
from botocore.docs.service import ServiceDocumenter
File "/home/spark/.local/lib/python3.10/site-packages/botocore/docs/service.py", line 14, in <module>
from botocore.docs.client import ClientDocumenter, ClientExceptionsDocumenter
File "/home/spark/.local/lib/python3.10/site-packages/botocore/docs/client.py", line 14, in <module>
from botocore.docs.example import ResponseExampleDocumenter
File "/home/spark/.local/lib/python3.10/site-packages/botocore/docs/example.py", line 13, in <module>
from botocore.docs.shape import ShapeDocumenter
File "/home/spark/.local/lib/python3.10/site-packages/botocore/docs/shape.py", line 19, in <module>
from botocore.utils import is_json_value_header
File "/home/spark/.local/lib/python3.10/site-packages/botocore/utils.py", line 34, in <module>
import botocore.httpsession
File "/home/spark/.local/lib/python3.10/site-packages/botocore/httpsession.py", line 21, in <module>
from urllib3.util.ssl_ import (
ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/home/spark/.local/lib/python3.10/site-packages/urllib3/util/ssl_.py)
AWS OFFICIALUpdated 7 months ago
Cannot reproduce that (and that would be a big issue). Are you adding any python zip files or additional modules?. What region is that?
The region is frankfurt. I am not using python zip files but note that the jobs ran with no issues until Wednesday 27th Sep 2023. Also I get the error in Glue notebook when I start a session
Waiting for session abd-asd-82739739729 to get into ready status... Session abd-asd-82739739729 has been created. ImportError: cannot import name 'DEFAULT_CIPHERS' from 'urllib3.util.ssl_' (/home/spark/.local/lib/python3.10/site-packages/urllib3/util/ssl_.py)
This is the code that raises the error even in notebook cell ++++++++++++++++++++++++++++++++++++++++++++++++++ import boto3 from botocore.exceptions import ClientError
def get_secrets(): google_secret_name = "test" gsheet_secret_name = "test_secrets" region_name = "eu-central-1" session = boto3.session.Session() client = session.client( service_name='secretsmanager', region_name=region_name)
I can run the code correctly on the region, maybe there is something wrong with your Jupyter/IS setup. I guess you are using your own notebook and not a managed one.
Glue jobs runs fine today which means it was from AWS servers. Hopefully, it's a one off as it affects the reliablility of my jobs.