How do I get the output of an AWS Glue DataBrew job to be a single CSV file?
0
I'm running a test AWS Glue DataBrew job to run a simple Group By and Sum operation on a CSV dataset that contains total costs by product each day. The desired output is a single CSV file with three columns: Day (timestamp), total cost, and product ID.
After I make the changes in the DataBrew project and select Run Job, the job runs successfully, but the output is numerous CSV files. Is there a way for me to specify that I want the output to just be one file, rather than multiple files?
1 Answer
- Newest
- Most votes
- Most comments
1
You have two options:
- Download the processed recipe directly from the project workspace. (Note: This option is limited to the sample data on screen and has a maximum row size of 5000.)
- Write an AWS Glue extract, transform, and load (ETL) job to repartition the data before writing a DynamicFrame to Amazon S3.
answered 6 years ago
Relevant content
asked 4 years ago
asked 3 years ago
asked a year ago
- AWS OFFICIALUpdated 2 years ago
