DataBrew recipe job fails on file with single record

0

Hi, I have noticed that when running the DataBrew recipe job for data masking, the job is failing when the parquet file has a single record in it. The error is following:

"Exception: Error: Unable to detect MIME type for file"

Denys
asked a year ago495 views
1 Answer
0

This error message indicates that DataBrew is having trouble determining the file format of the input data when the file contains only a single record. This is likely due to the fact that some file formats rely on having a certain number of records present in order to determine the file format.

One potential workaround for this issue would be to add a dummy record to the file so that it contains more than one record. This should allow DataBrew to properly detect the file format and run the recipe job successfully.

Another option would be to change the file format to a format that is more resilient to single-record files, such as CSV or JSON. This would require converting the input data to the new format before running the DataBrew job.

If neither of these options are feasible, you may need to reach out to AWS Support for further assistance in troubleshooting this issue. They may be able to provide more information about the specific cause of the issue and potential solutions.

AWS
answered a year ago
  • I would also pay attention at the Role used for the DataBrew Profiler job. It could be missing the proper permission to read the data. I just had the same error with a 100 lines CSV file and the problem was that I forgot to add a role to the profiler job.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions