Questions tagged with Amazon S3 Glacier

Content language: English

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

Query Athena from s3 database - remove metadata/corrupted data

Hello, I was following along with the tutorials for connecting Tableau to Amazon Athena and got hung up when running the query and returning the expected result. I downloaded the student-db.csv from https://github.com/aws-samples/amazon-athena-tableau-integration and uploaded the csv to a S3 bucket that I created. I can create the database within Athena however when I create a table either with the bulk add or directly from the query editor and preview with a query the data gets corrupted. and includes unexpected characters and unexpected/unnecessary punctuations and sometimes all the data is aggregated into a single column and also contains metadata such as "1 ?20220830_185102_00048_tnqre"0 2 ?hive" 3 Query Plan* 4 Query Plan2?varchar8 @H?P?". Also with my Athena - Tableau connected receiving the same issues when I preview the table that was created with Athena and stored in my bucket. ``` CREATE EXTERNAL TABLE IF NOT EXISTS student( `school` string, `country` string, `gender` string, `age` string, `studytime` int, `failures` int, `preschool` string, `higher` string, `remotestudy` string, `health` string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION 's3://jj2-test-bucket/' TBLPROPERTIES ( 'has_encrypted_data'='false', 'skip.header.line.count'='1', 'transient_lastDdlTime'='1595149168') ``` ``` SELECT * FROM "studentdb"."student" limit 10; ``` ![Enter image description here](/media/postImages/original/IMVPFHEHtDTo2aPRoGcWSBBg)
1
answers
0
votes
65
views
asked 3 months ago

AWS S3 behavior when both transition and expiration is present

I read multiple documents in case when both expiration and transition is present in same rule. [This link](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html) says below 2 somewhat different things: * Example 4 in link: Transition takes precedence over creation of delete markers. * Example 4 in link: If an object has both tags, then Amazon S3 has to decide which rule to follow. In this case, Amazon S3 expires the object 14 days after creation. The object is removed, and therefore the transition action does not apply. * Also [other docs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html) mention that "Glacier has 90 days minimum storage duration" I configured lifecycle policy on a "versioned bucket". I added single rule with expiration (current version) and transition policy (current version): * Transition the latest version of objects to Glacier Flexible Retrieval (formerly Glacier) after 10 days of object creation. * Delete (create Delete Marker since bucket is version enabled) the current version of object after 15 days of object creation. ![Enter image description here](/media/postImages/original/IMg22N_9v3QFSGp5WhJeFBbw) I want to understand AWS S3 behavior for above configuration. Will it 1. "change object's storage class to Glacier after 10 days" and later "delete the object (add delete marker) after 15 days" ? 2. or will it "change object's storage class to Glacier after 10 days" and after 90 days "delete the object (add delete marker)" ? 3. or something else ?
2
answers
0
votes
32
views
asked 3 months ago