Amazon Polly Response Time

Question

Hi,
We're using Amazon Polly ( boto3 Python client ) for production  TTS. We have a "fixed" test text that we're using to monitor our system. Usually this text takes a few seconds to be completed. I've noticed that sometimes it takes more than 1 minute (!!) to complete successfully.
As we're using it as part of an IVR system where user is waiting on a phone, I would like to know if there is a way to speed it up or to prioritize such operations as well as to know how to audit/log/troubleshoot such operations (via cloudwatch or any other method). thanks in advanced !!

Ryan_A · Answer

Hi,

The 1 minute mentioned here is really close to the default boto3 connection timeout which is 60 seconds. I think what's happening here is that the client which is using boto3 tries to connect to the Polly endpoint but is unable to do so due to intermittent/transient networking issues. These are normally resolved by retrying the request. In the case of Boto3, it has to wait for 60 seconds before retrying the request again.

You can enable boto3 debug logs and follow the steps here in this documentation to check if all of the instances that are taking more than 1 minute is due to retries -> https://boto3.amazonaws.com/v1/documentation/api/latest/guide/retries.html#validating-retry-attempts

Please read this page for more information about this -> https://aws.amazon.com/premiumsupport/knowledge-center/lambda-function-retry-timeout-sdk/. This mentions Lambda but it can occur to any client which is invoking  any AWS API via the SDK.

You can update the default boto3 settings by following the example in the above article
```
# max_attempts: retry count / read_timeout: socket timeout / connect_timeout: new connection timeout

from botocore.session import Session
from botocore.config import Config

s = Session()
c = s.create_client('s3', config=Config(connect_timeout=5, read_timeout=60, retries={'max_attempts': 2}))
```

The idea here is to "fail fast" and ensure that boto3 times out quickly and retries the request immediately instead of waiting for the default timeout. Please make sure that the connection timeout is around 3x your expected or normal timeout to ensure healthy requests are not considered as timeouts.

That said, this behavior should only happen for a small number of requests. If a large number of your requests are experiencing timeouts or if the requests are taking more than 60 seconds without any retries then I would recommend logging the RequestID's and creating a support case with us so that a log dive can be performed on these requests.

Amazon Polly Response Time

Add your answer

Relevant content