Disable DynamoDB stream, restore data, not process any events during restore period with lambda trigger.

0

I'm attempting to prepare a DynamoDB disaster recovery plan. Our stack uses a DynamoDB table with a stream enabled. We also have a lambda trigger configured to listen for stream events. These events are processed and make a bunch of downstream changes.

In a DR situation Im looking for the ability to create a temp table from a snapshot, and then copy that restored data into a new 'DR' table that's managed by IAC. It's not ideal for the stream and lambda trigger to process events from the copy. That would cause a bunch of downstream issues.

I see it's possible to disable/ enable the dynamo stream, and although you'll receive a new arn after a stream is disabled/ enabled I can't find anything in documentation if behind the scenes you're connecting to the original stream log or not? If I configure my lambda trigger to listen to the stream will all the events from the copy start processing?

If I dont have the stream enabled while Im copying data to the table will the stream log be empty when I enable?

Another option Im looking into is lambda event source mapping, specifically the starting position. I might be able to use the 'latest' flag.

  • What kind of disaster scenario are you mitigating against? Are your efforts here really necessary?

asked 3 months ago65 views
1 Answer
2
Accepted Answer

I see it's possible to disable/ enable the dynamo stream, and although you'll receive a new arn after a stream is disabled/ enabled I can't find anything in documentation if behind the scenes you're connecting to the original stream log or not?

No, when you disable/enable a stream it changes ARN and is completely separate to the old stream.

If I configure my lambda trigger to listen to the stream will all the events from the copy start processing?

It depends how you "copy" the data. If your "copy" means restore the table from backup, then no the events will not be in the stream. If your "copy" is done by you calling PutItem or BatchWriteItems to insert data, then yes it will enter the stream.

If I dont have the stream enabled while Im copying data to the table will the stream log be empty when I enable?

Yes

Another option Im looking into is lambda event source mapping, specifically the starting position. I might be able to use the 'latest' flag.

LATEST will start reading from the the point of when you enable the trigger on the ESM. So if your copy puts events on the stream and you later create an ESM with iterator position as LATEST your Lambda will disregard all the data from the copy.

profile picture
answered 3 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions