Athena - not respecting ResultReuseByAgeConfiguration correctly

0

Hi all,

I run some AWS Athena queries with the ResultReuseByAgeConfiguration option set. From a behaviour perspective, I would expect Athena to cache the results, but it seems Athena is caching it too well. I am using parameterised queries, and it seems that Athena is not taking the parameterised values in consideration when deciding what needs to be cached. Is this expected behaviour, or possibly a bug in how Athena manages caching?

To demonstrate, I create a very simple stored procedure in Athena.

PREPARE TEST_CACHE FROM
SELECT ?

Running the following code, you would expect all tests to pass (value inserted and returned are the same)

import boto3
import time

def query(v,reuse=False):
    athena = boto3.client('athena')
    query_id1 = athena.start_query_execution(
            QueryString           = 'execute TEST_CACHE',
            ExecutionParameters = [v],
            QueryExecutionContext = { 'Database' : 'default' },
            WorkGroup             = 'ca-main-athena',
            ResultReuseConfiguration={
            'ResultReuseByAgeConfiguration': {
                'Enabled': reuse,
                'MaxAgeInMinutes': 60
            }
        }
    )['QueryExecutionId']

    while True:
        finish_state = athena.get_query_execution(QueryExecutionId=query_id1)["QueryExecution"]["Status"]["State"]
        if finish_state == "RUNNING" or finish_state == "QUEUED":
            time.sleep(1)
        else:
            break

    athenaresult1 = athena.get_query_results(QueryExecutionId=query_id1)
    r = athenaresult1['ResultSet']['Rows'][1]['Data'][0]['VarCharValue']
    if  r == v:
        print(f"result match - expected {v} and got {r}")
    else:
        print(f"result does not match - expected {v} and got {r}")

# = works as expected - result reuse is turned off
query("alpha",False)
query("bravo",False)
query("charlie",False)

# = does not work as expected - result reuse is turned on
query("delta",True)
query("echo",True)
query("foxtrot",True)

The result does demonstrate that the caching is not correct.

$ python test_athena.py 
result match - expected alpha and got alpha
result match - expected bravo and got bravo
result match - expected charlie and got charlie
result does not match - expected delta and got charlie
result does not match - expected echo and got charlie
result does not match - expected foxtrot and got charlie
profile picture
massyn
질문됨 3달 전160회 조회
1개 답변
0

Hi massyn, this post has been brought to the Athena team's attention and added to our ticket queue for investigation. You will receive a followup here when we have more details. Thank you for your submission!

profile pictureAWS
mattmyo
답변함 3달 전
  • We have resolved this issue. You should be able to see the intended behaviour for parameterized queries with caching now.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인