Is it possible to use the native S3 SDK for storing Hive data with S3?

0

I use Glue to store Hive metadata and S3 to store Hive data, so can I just use S3's SDK to do write data without hadoop-fs?

preguntada hace 2 años223 visualizaciones
1 Respuesta
0

To add or change data in your S3 buckets, you should be able to use the SDK for S3.

If you want to use the Hive like capabilities to query your data from S3, you would define the metadata for the S3 data using Athena or Glue APIs to define the table definitions or execute the Glue crawler pointing to the file location so that the tables and partitions are defined for you in the Glue catalog. The data that was manipulated using S3 SDK/APIs would be available for querying in Athena as soon as the schemas are defined in the Glue catalog.

If you want to use EMR to manipulate the data, please check out the EMR File system https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-fs.html

profile pictureAWS
respondido hace un año

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas