Questions tagged with Amazon S3 Glacier
Content language: English
Sort by most recent
Athena Query Internal Error
Hello, I have been using Athena to query json document from s3 bucket and I got below error while executing the query, **[ErrorCode: INTERNAL_ERROR_QUERY_ENGINE] Amazon Athena experienced an internal error while executing this query. Please contact AWS support for further assistance. You will not be charged for this query. We apologize for the inconvenience. This query ran against the "pc_slc_smtt_sustainability_db_prod" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 3e3e22c7-76c9-46fb-9d66-2d464a2b67a3** I do not understand what is the issue causing this?
head_object request on S3 object after restore still shows old storage class
I'm trying to dynamically determine the storage class of an object that was restored from GLACIER to STANDARD. But when i make this request from boto3's head_object i still keep getting the old storage class of the object. I've verified the restore completion from the url (version:null)
Query Athena from s3 database - remove metadata/corrupted data
Hello, I was following along with the tutorials for connecting Tableau to Amazon Athena and got hung up when running the query and returning the expected result. I downloaded the student-db.csv from https://github.com/aws-samples/amazon-athena-tableau-integration and uploaded the csv to a S3 bucket that I created. I can create the database within Athena however when I create a table either with the bulk add or directly from the query editor and preview with a query the data gets corrupted. and includes unexpected characters and unexpected/unnecessary punctuations and sometimes all the data is aggregated into a single column and also contains metadata such as "1 ?20220830_185102_00048_tnqre"0 2 ?hive" 3 Query Plan* 4 Query Plan2?varchar8 @H?P?". Also with my Athena - Tableau connected receiving the same issues when I preview the table that was created with Athena and stored in my bucket. ``` CREATE EXTERNAL TABLE IF NOT EXISTS student( `school` string, `country` string, `gender` string, `age` string, `studytime` int, `failures` int, `preschool` string, `higher` string, `remotestudy` string, `health` string) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS INPUTFORMAT 'org.apache.hadoop.mapred.TextInputFormat' OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat' LOCATION 's3://jj2-test-bucket/' TBLPROPERTIES ( 'has_encrypted_data'='false', 'skip.header.line.count'='1', 'transient_lastDdlTime'='1595149168') ``` ``` SELECT * FROM "studentdb"."student" limit 10; ``` ![Enter image description here](/media/postImages/original/IMVPFHEHtDTo2aPRoGcWSBBg)
AWS S3 behavior when both transition and expiration is present
I read multiple documents in case when both expiration and transition is present in same rule. [This link](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-configuration-examples.html) says below 2 somewhat different things: * Example 4 in link: Transition takes precedence over creation of delete markers. * Example 4 in link: If an object has both tags, then Amazon S3 has to decide which rule to follow. In this case, Amazon S3 expires the object 14 days after creation. The object is removed, and therefore the transition action does not apply. * Also [other docs](https://docs.aws.amazon.com/AmazonS3/latest/userguide/lifecycle-transition-general-considerations.html) mention that "Glacier has 90 days minimum storage duration" I configured lifecycle policy on a "versioned bucket". I added single rule with expiration (current version) and transition policy (current version): * Transition the latest version of objects to Glacier Flexible Retrieval (formerly Glacier) after 10 days of object creation. * Delete (create Delete Marker since bucket is version enabled) the current version of object after 15 days of object creation. ![Enter image description here](/media/postImages/original/IMg22N_9v3QFSGp5WhJeFBbw) I want to understand AWS S3 behavior for above configuration. Will it 1. "change object's storage class to Glacier after 10 days" and later "delete the object (add delete marker) after 15 days" ? 2. or will it "change object's storage class to Glacier after 10 days" and after 90 days "delete the object (add delete marker)" ? 3. or something else ?
Enabling lifecycle rules on a huge S3 bucket
Hello all, I have a huge S3 bucket with almost 7 Billion objects and 270 Tb of data. This bucket receives almost 2 million objects per day, and the object size has an average of 37 kb. Enabling Intelligent tiering here is a no-go because it's really expensive scanning objects. I'm thinking in enabling Infrequent Access but I don't know if doing this could lead to high costs just for enabling it on this size of a bucket. Any advice on this would be helpful, thanks.
Creating an S3 Glacier Deep Archive vault with the CLI
I would like to create a Glacier Deep Archive to store a good chunk of files that I will rarely, if ever, access, and in such circumstance, I can wait a couple of days until the files are retrieved. So I would like to use Glacier Deep Archive. Checking the documentation, the only way to create vaults with the CLI that I've found is: ``` aws glacier create-vault --vault-name my_vault_name --account-id 123456789012 ``` Then in the Glacier management console, I can't find what type of storage my vault is (Instant Retrieval, Flexible Retrieval, Deep Archive). Two questions: where can I find the information about the type of storage for existing vaults, and how can I make sure my vault is a Deep Archive one? Many thanks in advance.
Using Google App Scripts to connect with Amazon S3
I have a Google Sheets add-on which let users get [stock price in Google Sheets](https://workspace.google.com/marketplace/app/stock_price_in_google_sheets_finsheet/574480000400). I am trying to copy users' request in the spreadsheet to S3. I am using the same library as in this [question](https://stackoverflow.com/questions/68776250/connect-to-amazon-s3-from-google-apps-script) on Stackoverflow and I encounter the exact same error: ```` AWS Error - SignatureDoesNotMatch: The request signature we calculated does not match the signature you provided. Check your key and signing method ```` Just wondering whether anyone has experience with this? The solution in that question does not work for me
How do I create a lifecycle rule to move to glacier?
Hi everyone, first time user of AWS... Thank you in advance for your help and your time. I created a lifecycle rule to move everything to Glacier Flexible but it hasn't done anything yet... Have I misconfigured something? Does versioning need to be enabled for this type of transition to work? ![Enter image description here](/media/postImages/original/IM4La8imPkT5S1l6qPlGAzAA) **Bonus Question:** Can I just select all objects and change the storage class? It said it would make a copy if I did that... I am assuming it means it will make a copy to Glacier and not maintain another copy in S3 standard. Thanks again!
Textract faster on some days rather than others
Previously about 2 weeks ago we were experiencing around 10-15 seconds per pdf when querying textract to translate our pdf. All the sudden now with the same code it takes give or take 5 minutes. Some key things I would like to point out are the PDFs are no larger and the code has not changed since then. The language I'm using is python and the library is boto3. Currently we are using the start_document_text_dection function that points to a pdf located in our s3 bucket. Once we determine the PDF is done reading (JobStatus changes from IN_PROGRESS), we then use the get_document_text_dection function to wrap the process up.