Is it possible to use the native S3 SDK for storing Hive data with S3?

0

I use Glue to store Hive metadata and S3 to store Hive data, so can I just use S3's SDK to do write data without hadoop-fs?

1 Risposta
0

To add or change data in your S3 buckets, you should be able to use the SDK for S3.

If you want to use the Hive like capabilities to query your data from S3, you would define the metadata for the S3 data using Athena or Glue APIs to define the table definitions or execute the Glue crawler pointing to the file location so that the tables and partitions are defined for you in the Glue catalog. The data that was manipulated using S3 SDK/APIs would be available for querying in Athena as soon as the schemas are defined in the Glue catalog.

If you want to use EMR to manipulate the data, please check out the EMR File system https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-fs.html

profile pictureAWS
con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande