AWS Glue Job does not log information messages in the output file

Question

Hi,

I was using AWS Glue job with Python shell. The log.info() messages does not log in the output log file instead it logs in the error.log file.

Here is the snippet we use to log messages in the glue job

we define logger  as below code snippet:
    def logger_setup(self):
        self.logger = logging.getLogger(__name__)
        self.logger.setLevel(logging.INFO)
        formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
        console_handler = logging.StreamHandler()
        console_handler.setLevel(logging.INFO)
        console_handler.setFormatter(formatter)
        self.logger.addHandler(console_handler)

However when we used in the main file, 
logger.info(f"first day of month for this job {self.first_day_of_month}")
This message logs in the Error logs instead Output logs. I want to post info messages in the Output logs. Can you tell me how we can achieve this? Is there any documentation available in logging messages in Glue job Python Shell

Answer

Could you please try with the following piece of code:

*self.logger.setLevel(logging.DEBUG)
...
console_handler = logging.StreamHandler(sys.stdout)
console_handler.setLevel(logging.DEBUG)*

Thank you :)

Answer

I tried this with sys.stdout. But this print all the log levels on both output log and error log. I want to print logger.info() into output log and logger.error() in error log. Is there a way to print that?

AWS Glue Job does not log information messages in the output file

Relevanter Inhalt