Al usar AWS re:Post, aceptas las AWS re:Post Términos de uso
AWS re:Postre:Post

Does AWS Glue support fixed-byte-length source data?

0

Hello. Can AWS Glue read source data file like below?

  20220101E00011000AAABBBCCC
  20220101E00021000あいCCC

The second record contains Japanese characters, and is the same byte length with the first record. The numbers of characters vary by record.

Thank you.

  • rePost-User-2300507
    hace 2 años

    If the numbers of characters are the same by record, I can use Grok pattern like "(?<col0>.{6})(?<col1>.{2})..." when making a data catalog using crawler. But in this case, numbers differ, and bytes are the same. Can anyone tell AWS Glue does support/not support data set like this?

Temas
AnálisisBase de datos
Etiquetas
AWS GlueExtraer datos de transformación y carga
Idioma
English
rePost-User-2300507
preguntada hace 2 años413 visualizaciones
1 Respuesta
0
Respuesta aceptada

Unfortunately parsing data by Bytes right now is not supported in glue. I observed your data is unstructured and the only way to parse that data is Grok SerDe or Regex SerDe but they parse data by identifying patterns so they are not feasible. I will recommend you to preprocess your data and then load them in glue.

unstructured data -> preprocessing using some Custom built parser function(csv) -> S3 -> crawl and createDatabase in glue.
AWS
INGENIERO DE SOPORTE
Shubh
respondido hace 2 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas

Contenido relevante