1개 답변
- 최신
- 최다 투표
- 가장 많은 댓글
0
Hi,
Since I don't know your code, I would suggest that you check the following areas
- The authentication is only done once - Ensure that you are only doing the authentication once and the subsequent queries are made based on this initial authentication.
- Connected to the above issue, if you are using IAM roles, there is a limit on the number of calls you can make to get the instance metadata to get the access and secret key.
- Try putting logs at each points to understand exactly on which line the code takes time. If you have already done this and found out that query itself is taking time, then you will have to check exactly what and how you are querying it.
I'm not an expert but I hope this helps, if you have more information to add or if you have found out the solution by yourself, please provide a response so that we can have further discussion
Regards
Vignesh N
답변함 2년 전
관련 콘텐츠
AWS 공식업데이트됨 2년 전- AWS 공식업데이트됨 일 년 전
Hi Vignesh
Thank you for your response. Providing some snippet of what my code does ( lines are copied from my python code).
self.sess = boto3.Session (profile_name=profile, region_name=region) self.client = self.sess.client ('athena') execution_id = self.client.start_query_execution(QueryString = queryStr,QueryExecutionContext = { 'Database': '....'}, ResultConfiguration = { 'OutputLocation': 's3://...'}, WorkGroup='AmazonAthenaLakeFormation') while True: stats = self.client.get_query_execution(QueryExecutionId=execution_id['QueryExecutionId']) status = stats['QueryExecution']['Status']['State'] if status in ['SUCCEEDED', 'FAILED', 'CANCELLED']: break time.sleep (0.5) if status == 'SUCCEEDED': results = self.client.get_query_results(QueryExecutionId=execution_id['QueryExecutionId'], MaxResults=1000) """returns a python pandas frame and column names""" dataframe, columns = self.returnResultsAsPandaFrame(results) while 'NextToken' in results: """this has performance impact""" results = self.client.get_query_results(QueryExecutionId=execution_id['QueryExecutionId'], MaxResults=1000, NextToken=results['NextToken']) """this is not having a performance impact""" dataframe = self.appendData (dataframe, columns, results)
I believe that Authentication is happening only once. (comments continued in next comment ...)
Does AWS Athena create execution logs for a query? Perhaps looking at the logs may provide some clues. Appreciate any guidance on this