DataBrew recipe job fails on file with single record

0

Hi, I have noticed that when running the DataBrew recipe job for data masking, the job is failing when the parquet file has a single record in it. The error is following:

"Exception: Error: Unable to detect MIME type for file"

Denys
preguntada hace un año508 visualizaciones
1 Respuesta
0

This error message indicates that DataBrew is having trouble determining the file format of the input data when the file contains only a single record. This is likely due to the fact that some file formats rely on having a certain number of records present in order to determine the file format.

One potential workaround for this issue would be to add a dummy record to the file so that it contains more than one record. This should allow DataBrew to properly detect the file format and run the recipe job successfully.

Another option would be to change the file format to a format that is more resilient to single-record files, such as CSV or JSON. This would require converting the input data to the new format before running the DataBrew job.

If neither of these options are feasible, you may need to reach out to AWS Support for further assistance in troubleshooting this issue. They may be able to provide more information about the specific cause of the issue and potential solutions.

AWS
respondido hace un año
  • I would also pay attention at the Role used for the DataBrew Profiler job. It could be missing the proper permission to read the data. I just had the same error with a 100 lines CSV file and the problem was that I forgot to add a role to the profiler job.

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas