2 Answers
- Newest
- Most votes
- Most comments
1
This is done by Hadoop if the folder does not exist. This _$folder$ is just a placeholder. This is created by mkdir commands. The actual folder is only created when first file is placed. The other jobs where this is not happening might be writing to existing folders. These files should not cause a problem.
answered a year ago
0
This happens because of the S3 path you use during writing.
s3:// vs s3a://
s3:// will make the folder s3a:// will not
They both have their ups and downs and is generally recommended to stick with s3://.
answered a year ago
Relevant content
- asked 5 months ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 7 months ago
- How can I use a Lambda function to automatically start an AWS Glue job when a crawler run completes?AWS OFFICIALUpdated 2 years ago