1 Answer
- Newest
- Most votes
- Most comments
0
Hello,
It seems data related issue only. I would suggest before writing the data verify if you are getting correct data without null values and valid format. You can update your script and add following lines to verify the data.
- Comment your write method in code
- Use printSchema and show() method to verify the data before write method might be possible some transformation in between creating the issue(like null and incompatible)
dyf.printSchema()
dyf.toDF().show(5)
Also i would suggest to check the complete logs specially caused by to understand the exeact reason. Also using above method you will be able to find if there is any issue with data.
answered 2 months ago
Relevant content
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated a year ago
More likely that's a partition column and you have an empty value on it. Check the stack trace for hints/causes