I'm testing a relatively simple pyspark script that I first wrote (and tested) EMR. On the EMR script works as intended, but in Glue, the script starts writing output to desired S3 location and stops midway with this error:
An error occurred while calling o285.save. File already exists:s3://bucket/prefix/part-xxxx.json
can you post the full script?