Is it possible to use the native S3 SDK for storing Hive data with S3?

0

I use Glue to store Hive metadata and S3 to store Hive data, so can I just use S3's SDK to do write data without hadoop-fs?

asked a year ago204 views
1 Answer
0

To add or change data in your S3 buckets, you should be able to use the SDK for S3.

If you want to use the Hive like capabilities to query your data from S3, you would define the metadata for the S3 data using Athena or Glue APIs to define the table definitions or execute the Glue crawler pointing to the file location so that the tables and partitions are defined for you in the Glue catalog. The data that was manipulated using S3 SDK/APIs would be available for querying in Athena as soon as the schemas are defined in the Glue catalog.

If you want to use EMR to manipulate the data, please check out the EMR File system https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-fs.html

profile pictureAWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions