By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

HDFS command to convert to AWS S3

0

We are having many HDFS commands on our ETL scripts. Due to cloud migration we need to convert the HDFS commands to AWS S3 commadns.

We have a command as below:

hdfs dfs -text ${OUTPUT_PATH}/part* | hdfs dfs -cp ${INPUT_PATH}/${FILENAME}

Need help to convert this to S3 command, mainly the first part with '-text'.

Topics
Storage
Tags
S3 Select
Language
English
Abhisek
asked 2 years ago205 views
1 Answer
0

Did you try this command to select the object content? https://docs.aws.amazon.com/cli/latest/reference/s3api/select-object-content.html

jschwar313
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content