Redshift stored procs best practice for transactions locking tables and avoiding serialization isolation violations

0

I am facing occasional concurrency issues resulting in Serialization isolation violations where an sproc is invoked simultaneously for different tasks where the pseudo code for sp_main looks like -- BEGIN LOCK process_table ; -- has an IDENTITY event_id INSERT process_table VALUES(var_work, GETDATE()) ; SELECT INTO var_eid MAX(event_id) FROM process_table ; COMMIT ; END ; IF var_work = 'do_lots' THEN CALL sp_slow_proc ; -- Perform lots of slower statements that do not reference process_table ELSE CALL sp_quick_proc ; -- inserts new rows to process_table using a BEGIN LOCK COMMIT END construct END IF ; BEGIN LOCK process_table ; UPDATE process_table SET ended_at = GETDATE() WHERE event_id = var_eid ; COMMIT ; END ; It seems the slow running process always the one that chokes! I'm 99% confidant this can happen even if the fast process has finished (airflow job concluded having started after the slow process was commenced). Questions are: #1. is this an expected code layout ? #2. should the LOCK move outside the BEGIN END ? #3. when is a LOCK released - on COMMIT or ???

asked 2 years ago736 views
1 Answer
0

In Redshift, explicitly locking using the lock command is acquired in "ACCESS EXCLUSIVE" mode and is released when the transaction that contains the LOCK command completes.

To determine which transactions might be facing/causing serialization isolation error, you can refer system table STL_TR_CONFLICT , which has information to identify transaction conflicts that result from serializable isolation errors ->https://docs.aws.amazon.com/redshift/latest/dg/r_STL_TR_CONFLICT.html Based on which , you should get
-> a tablename( by determining the locks using svv_transactions and the associated relation id , which can then be used to determine table name in svv_table_info ) ---****

-> https://docs.aws.amazon.com/redshift/latest/dg/r_SVV_TRANSACTIONS.html -> https://docs.aws.amazon.com/redshift/latest/dg/r_SVV_TABLE_INFO.html

-> 2 or more transaction ids (xids) that are involved in the conflict (stl_tr_conflict) and are forming cycles. ---****

Using the above you can determine the sequence of statements executed in those by querying svl_statementtext which led to this error

-> https://docs.aws.amazon.com/redshift/latest/dg/r_SVL_STATEMENTTEXT.html

Moreover, managing transactions inside a stored procedure is a good walk-through of how to layout your code -> https://docs.aws.amazon.com/redshift/latest/dg/stored-procedure-transaction-management.html

AWS
SUPPORT ENGINEER
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions