In AWS Redshift, are there errors that can cause a stored procedure to stop, even when having a error handling block?

0

I wrote a stored procedure that calls other procedures inside multiple transaction blocks (BEGIN…END) with their own error handlers. I expected that if one block encounters an exception, it would go to the corresponding error handling section and then continue with the next block. However, the stored procedure is exiting entirely instead of continuing. What could be wrong?

Here's the error I'm receiving:

SQL Error [XX000]: ERROR: Cannot insert a NULL value into column process
  Detail: 
  -----------------------------------------------
  error:  Cannot insert a NULL value into column process
  code:      8007
  context:   query execution
  query:     54021995[child_sequence:1]
  location:  column:10
  process:   query0_3881_54022009 [pid=7602]
  -----------------------------------------------

The SQL query for the procedure is:

CREATE TABLE dummy_table (
  id BIGINT,
  process VARCHAR(100) NOT NULL
);

CREATE OR REPLACE PROCEDURE sp_dummy_etl()
LANGUAGE plpgsql
AS $$
DECLARE
  v_load_id BIGINT := 1;
  v_row_count INT;
BEGIN
  RAISE INFO 'Dummy ETL process started. Load ID: %', v_load_id;
  
  -- Step 1: Attempt to insert a row that violates the NOT NULL constraint.
  BEGIN
    RAISE INFO 'Step 1: Inserting record into dummy_table';
    INSERT INTO dummy_table (id, process)
    VALUES (v_load_id, NULL);  -- This will trigger error (NOT NULL violation).
    RAISE INFO 'Step 1 completed successfully';
  EXCEPTION WHEN OTHERS THEN
    RAISE INFO 'Step 1 error caught: %', SQLERRM;
    -- This call is intended to log the error.
    CALL dummy_error_log(v_load_id, SQLSTATE, SQLERRM);
  END;
  
  RAISE INFO 'Continuing to Step 2';
  
  -- Step 2: Dummy successful operation.
  BEGIN
    RAISE INFO 'Step 2: Calling dummy_step2';
    CALL dummy_step2(v_load_id, v_row_count);
    RAISE INFO 'Step 2 completed successfully. Row count: %', v_row_count;
  EXCEPTION WHEN OTHERS THEN
    RAISE INFO 'Step 2 error caught: %', SQLERRM;
    CALL dummy_error_log(v_load_id, SQLSTATE, SQLERRM);
  END;
  
  RAISE INFO 'Dummy ETL process finished';
  
END;
$$;

When calling the sp_dummy_etl(), it executes the first block, encounters the exception, handling it by writing in the error_log, and after that, it completely exit the procedure sp_dummy_etl instead of continuing to the next transaction block (Step 2).

asked 2 months ago131 views
3 Answers
0

The behavior you're experiencing with your stored procedure in Amazon Redshift is due to the limited support for exception handling in Redshift compared to standard PostgreSQL. Redshift doesn't support subtransactions, which affects how exception handling works within stored procedures.

In your case, even though you have separate BEGIN...END blocks with their own error handlers, the entire stored procedure is still considered as one transaction. When an error occurs in one of these blocks, it's caught by the EXCEPTION block, but it still affects the overall transaction state.

The error you're receiving ("Cannot insert a NULL value into column process") is a constraint violation. This type of error causes the entire transaction to be aborted, even if it's caught in an EXCEPTION block. After the error is caught and logged, the procedure exits because the transaction has been invalidated.

To address this issue and make your stored procedure continue execution after an error, you have a few options:

  1. Use a CONTINUE handler instead of an EXIT handler. The CONTINUE handler allows the procedure to continue execution after an error. However, be aware that this is specific to NONATOMIC procedures in Redshift.

  2. Move operations that don't need to be in the same atomic transaction outside of the transaction. This can help isolate errors and prevent them from affecting the entire procedure.

  3. Consider using separate stored procedures for each step, and then call these procedures from a main procedure. This way, if one step fails, it won't affect the execution of subsequent steps.

  4. If possible, validate data before attempting to insert it. This can help prevent constraint violation errors in the first place.

Remember, when working with stored procedures in Redshift, it's important to be aware of these limitations in exception handling and plan your error handling strategy accordingly.
Sources
Trapping errors - Amazon Redshift
Stored procedure limitations - Amazon Redshift

profile picture
answered 2 months ago
0

Add exception control block in your stored procedure:

CREATE OR REPLACE PROCEDURE sp_dummy_etl()
LANGUAGE plpgsql
AS $$
DECLARE
  v_load_id BIGINT := 1;
  v_row_count INT;
BEGIN
  RAISE INFO 'Dummy ETL process started. Load ID: %', v_load_id;
  
  -- Wrap the entire procedure body in an exception block
  BEGIN
    -- Step 1
    BEGIN
      RAISE INFO 'Step 1: Inserting record into dummy_table';
      INSERT INTO dummy_table (id, process)
      VALUES (v_load_id, NULL);
      RAISE INFO 'Step 1 completed successfully';
    EXCEPTION WHEN OTHERS THEN
      RAISE INFO 'Step 1 error caught: %', SQLERRM;
      CALL dummy_error_log(v_load_id, SQLSTATE, SQLERRM);
    END;
    
    RAISE INFO 'Continuing to Step 2';
    
    -- Step 2
    BEGIN
      RAISE INFO 'Step 2: Calling dummy_step2';
      CALL dummy_step2(v_load_id, v_row_count);
      RAISE INFO 'Step 2 completed successfully. Row count: %', v_row_count;
    EXCEPTION WHEN OTHERS THEN
      RAISE INFO 'Step 2 error caught: %', SQLERRM;
      CALL dummy_error_log(v_load_id, SQLSTATE, SQLERRM);
    END;
  EXCEPTION WHEN OTHERS THEN
    RAISE INFO 'Main procedure error caught: %', SQLERRM;
  END;
  
  RAISE INFO 'Dummy ETL process finished';
END;
$$;

Alternatively use the transactional control approach:

CREATE OR REPLACE PROCEDURE sp_dummy_etl()
LANGUAGE plpgsql
AS $$
DECLARE
  v_load_id BIGINT := 1;
  v_row_count INT;
BEGIN
  RAISE INFO 'Dummy ETL process started. Load ID: %', v_load_id;
  
  -- Step 1
  BEGIN
    SAVEPOINT step1;
    
    RAISE INFO 'Step 1: Inserting record into dummy_table';
    INSERT INTO dummy_table (id, process)
    VALUES (v_load_id, NULL);
    RAISE INFO 'Step 1 completed successfully';
    
    RELEASE SAVEPOINT step1;
  EXCEPTION WHEN OTHERS THEN
    ROLLBACK TO step1;
    RAISE INFO 'Step 1 error caught: %', SQLERRM;
    CALL dummy_error_log(v_load_id, SQLSTATE, SQLERRM);
  END;
  
  RAISE INFO 'Continuing to Step 2';
  
  -- Step 2
  BEGIN
    SAVEPOINT step2;
    
    RAISE INFO 'Step 2: Calling dummy_step2';
    CALL dummy_step2(v_load_id, v_row_count);
    RAISE INFO 'Step 2 completed successfully. Row count: %', v_row_count;
    
    RELEASE SAVEPOINT step2;
  EXCEPTION WHEN OTHERS THEN
    ROLLBACK TO step2;
    RAISE INFO 'Step 2 error caught: %', SQLERRM;
    CALL dummy_error_log(v_load_id, SQLSTATE, SQLERRM);
  END;
  
  RAISE INFO 'Dummy ETL process finished';
END;
$$;
profile pictureAWS
EXPERT
answered 2 months ago
profile pictureAWS
EXPERT
reviewed 2 months ago
0

Check out NONATOMIC stored procedures they won’t raise caught errors

answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions