By using AWS re:Post, you agree to the Terms of Use
/Athena query error/

Athena query error


Getting this error:

Error opening Hive split s3://commoncrawl/cc-index/table/cc-main/warc/crawl=CC-MAIN-2017-17/subset=crawldiagnostics/part-00189-ac1cf8ef-3644-4b49-ac73-0fe6bef46adf.c000.gz.parquet (offset=0, length=36266581): Please reduce your request rate. (Service: Amazon S3; Status Code: 503; Error Code: SlowDown; Request ID: J88VWPD7BXAFP01E; S3 Extended Request ID: 89ZPRgZ/qx3n0Gs4zFonvfA50JUPjf9ep5vHxhCKHIFXwVr70vgbnLSL9Ctx22GNikrR+p/3gQU=; Proxy: null), S3 Extended Request ID: 89ZPRgZ/qx3n0Gs4zFonvfA50JUPjf9ep5vHxhCKHIFXwVr70vgbnLSL9Ctx22GNikrR+p/3gQU= This query ran against the "ccindex" database, unless qualified by the query. Please post the error message on our forum or contact customer support with Query Id: 066d4ff2-89ce-4683-a4c0-71e2f45348cb

asked 4 months ago64 views
1 Answers

This is a rate limiting error from S3, your query is probably making too many requests to S3 at the same time. This is usually a sign of your data set being too many small files, tens of thousands or more. Reduce the number of files by combining small files into bigger files.

answered 4 months ago
reviewed 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions