Gule job is stopped while running

Question

Glu job is stopped while runnning.

This process cannot be finished right.

Of cource, I didn't click "Strop job run" bottom.

This problem occurs when the following sentences are added, and there is  a error log in output log.

"""
cond= [col("a.mail_id")==col("b.mail_id"), col("b.mail_sub_id")==2]
castedDf = castedDf.alias("a").join(castedDf.alias("b"),cond,'left')\
            .filter(col('a.mail_sub_id') == 1)\
            .select('a.mail_id',\
            'a.mail_sub_id',...)
"""

Error log
java.lang.ClassNotFoundException: org.apache.logging.log4j.core.lookup.JndiLookup
~~~~

Answer

Typically, the `ClassNotFoundException` is a warning. This is due to a security patch  where vulnerable JNDI lookup classes were removed from the Log4j jars. These warnings can be safely ignored.

You should look in the logs for additional errors, to try to get a better understanding of the logs you can enable [AWS Glue Job Insights](https://docs.aws.amazon.com/glue/latest/dg/monitor-job-insights.html), to try to get some more specific information on the exact error causing the job failure. To identify the exact error messages, searching through the ERROR logs with `GlueExceptionAnalysisListener` typically would lead me to the exact line that errors and the Error message. Please try that and let us know if you are still unable to identify the error

Gule job is stopped while running

相关内容