glue treat empty string as null values
0
Hi Team,
I run an AWS glue job that reads data from a CSV file located on an S3 bucket to my aurora MySQL DB. My job fails because it interprets an empty string from the CSV ("") as a null value then try to insert it in a non-nullable column. is there a specific configuration in glue / pyspark code to prevent the job to treat an empty string as null?
asked 5 months ago334 views
1 Answers
0
Accepted Answer
I added this workaround on my pySpark script and that solve the issue :
data_frame = data_frame.na.fill('', subset=['MY_EMPTY_STRING_COL'])
answered 5 months ago
Relevant questions
Can an S3 import into RDS table be forced to success using an empty file?
asked a month agoSetting ACL in S3 objects written by an AWS Glue Job
Accepted Answerbackslash in CSV with glue
asked 5 months agoglue treat empty string as null values
Accepted Answerasked 5 months agoescape caracter in AWS glue
Accepted Answerasked 6 months agoGlue job s3 file not found exception
asked 5 years agoGlue job fail many workers
asked 3 months agoHow could we have Glue to get data from csv as String?
Accepted Answerasked 2 months agoLoading json string data as super from Glue job in to Redshift
asked 6 months agofail a glue job if the called stored procedure fails
asked 5 months ago