Issue in Data Backup in CSV from Salesforce to S3 Bucket

I am not familiar with Salesforce, but I believe that if you manually map the data you need in Appflow, it will be in CSV.
https://docs.aws.amazon.com/ja_jp/appflow/latest/userguide/create-flow-console.html

Manually map fields - Use the Amazon AppFlow user interface to specify the field mapping. To map all fields, choose Source field name, Bulk actions, Map all fields directly. Otherwise, select one or more fields from Source field name, Source fields, and then choose Map fields directly.

Also, please check again if the file format is CSV, as I think the default setting was JSON.