Local DynamoDB Batch Write Causes Excess SELECT Statements

1

Hello, I have a dynamodb local instance running on my Windows 11 system and have been testing the batch write feature of DynamoDB. I'm using ASP.NET Core and am using the high-level DynamoDBContext class to perform the writes. The writes are working and that's not an issue. What I'm curious about is what the SQLLite logs are showing. The server states this:

INFO com.amazonaws.services.dynamodbv2.local.server.LocalDynamoDBServerHandler - target: BatchWriteItem And then dozens, if not hundreds of these lines show up immediately after: [SQLiteQueue[shared-local-instance.db]] DEBUG com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccessJob - SELECT StreamID FROM sm WHERE TableName = ? AND (DeletionDateTime IS NULL OR DeletionDateTime > ?) ORDER BY CreationDateTime DESC LIMIT 1;

And then, finally, I see the batch write HTTP request show up. Is this something I should be concerned about? Is this specific to the local dynamoDB implementation? I don't have any sort of code logic that's sending reads in parallel with the batch writes in this situation. Any insight on what these logs mean would be awesome.

UPDATE 7/21/2022: A user asked for the code snippet and dynamodb local version. I'm using version 1.18.0 and this is the downloaded version, not Docker.

Code snippet where the task is fired off in parallel. This is awaited later on in the method. Only 25 items are batched at once:

BatchWrite<DynamoBGCLog> batchWriter = this.dynamo.CreateBatchWrite<DynamoBGCLog>();
batchWriter.AddPutItems(logsToWrite);
logsToWrite.Clear(); // empty out the bag

writeTasks.Add(
    Task.Run(
            () => BGCUtilities.BackoffAsync(
                    () => batchWriter.ExecuteAsync(abortToken),
                            maxRetries: 30,
                            maxRetryTimeMS: 5000,
                            abortToken: token),
                        token));

Thanks!

  • Can you share the code snippet used for BatchWrite and also the version of DynamoDB Local which you are using and on which platform (docker or direct download).

  • I updated the original post with these details. Dmitry Balabanov provided an answer that seems to answer my question. But sharing this info seems helpful anyways. Thanks!

profile picture
ucrbgc
asked 2 years ago295 views
1 Answer
0
Accepted Answer

This is an implementation detail of the DynamoDBLocal package. It only applies to the local version of DynamoDB.

With this SELECT statement, the implementation queries the metadata looking for open table streams. If there is an open stream for the target table, the written items will be put into the stream as well.

profile pictureAWS
answered 2 years ago
  • Thanks for the info. It was my assumption it was something specific to the local version. Good to know that I'm not doing something wrong in the code though. Appreciate it!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions