We have a lambda function for which provisioned conurrency =1 is enabled. We insert records in Postgres Aurora RDS using this lambda. We are connecting to RDS proxy endpoint to connect to db . The db object creation is done in below manner :
@staticmethod
def init_database_dao():
if Dao._db_connection_dao is None:
_logger.info(msg=f"creating db connection , singleton")
ConnectionProvider._db_connection_metadata_dao = MetadataDao() // inside MetadataDao we are calling postgres.connect()
else:
_logger.info(msg=f"refreshing connection for MetadataDao singleton")
DatabaseConnectionProvider._db_connection_metadata_dao.reset_if_cursor_or_conn_is_none()
return DatabaseConnectionProvider._db_connection_metadata_dao
We getting 'SSL connection has been closed unexpectedly' while executing SELECT query once in few hours (can be in 2 hours ,sometimes in 6 hours). We have handled this Postgres Operational Error by resetting the connection in code and retrying and it works. Not able to find the root cause of this error. we are using RDS proxy, we increased the idle client timeout of the proxy from 30 mins to 60 mins , we observed the frequency of this error has reduced.
Also we are using the same db for other lambda (with same db object creation implementation) , those lambdas dont have provisioned concurrency enabled, we never encountered SSL connection closed error in those lambdas, Is it somewhat related to the lambda always warmed up(provisioned concurrency) and the connection getting shutup or destroyed. Any suggestions to get rid of this error?
Note : The lambda with provisioned concurrency is the entry point of the application . In this lambda we are getting connection object, then reading from db and inserting into db .Once it is executed the other lambdas are triggered .
AWS OFFICIALUpdated 2 years ago
It is called inside the handler method. The init_database_dao() is a static method and the class in which it is defined is imported from a different python package outside the handler . I have created reset_if_cursor_or_conn_is_none function to refresh the connection .
When init_database_dao() is called, either a fresh connection is created or old one's status is checked and if needed is reset .When i am getting SSL error while executing query (in my observation it comes when we are getting already created connection object) ,while calling init_database_dao() none of the condition mentioned in the if statement of reset_if_cursor_or_conn_is_none() is getting true and hence reset() is not executed. I am wondering if there is any other attribute whose value needs to be checked in case of db timeout etc.As per this https://stackoverflow.com/questions/1281875/making-sure-that-psycopg2-database-connection-alive , the closed attribute should be able to give info about connection open/close. Once I am getting SSL error, I am calling the reset() method in exception block and refreshing the connection. I want to check if any connection refresh is needed before executing any query (so that i don't encounter any Operational Error)