Local DynamoDB Batch Write Causes Excess SELECT Statements

1

Hello, I have a dynamodb local instance running on my Windows 11 system and have been testing the batch write feature of DynamoDB. I'm using ASP.NET Core and am using the high-level DynamoDBContext class to perform the writes. The writes are working and that's not an issue. What I'm curious about is what the SQLLite logs are showing. The server states this:

INFO com.amazonaws.services.dynamodbv2.local.server.LocalDynamoDBServerHandler - target: BatchWriteItem And then dozens, if not hundreds of these lines show up immediately after: [SQLiteQueue[shared-local-instance.db]] DEBUG com.amazonaws.services.dynamodbv2.local.shared.access.sqlite.SQLiteDBAccessJob - SELECT StreamID FROM sm WHERE TableName = ? AND (DeletionDateTime IS NULL OR DeletionDateTime > ?) ORDER BY CreationDateTime DESC LIMIT 1;

And then, finally, I see the batch write HTTP request show up. Is this something I should be concerned about? Is this specific to the local dynamoDB implementation? I don't have any sort of code logic that's sending reads in parallel with the batch writes in this situation. Any insight on what these logs mean would be awesome.

UPDATE 7/21/2022: A user asked for the code snippet and dynamodb local version. I'm using version 1.18.0 and this is the downloaded version, not Docker.

Code snippet where the task is fired off in parallel. This is awaited later on in the method. Only 25 items are batched at once:

BatchWrite<DynamoBGCLog> batchWriter = this.dynamo.CreateBatchWrite<DynamoBGCLog>();
batchWriter.AddPutItems(logsToWrite);
logsToWrite.Clear(); // empty out the bag

writeTasks.Add(
    Task.Run(
            () => BGCUtilities.BackoffAsync(
                    () => batchWriter.ExecuteAsync(abortToken),
                            maxRetries: 30,
                            maxRetryTimeMS: 5000,
                            abortToken: token),
                        token));

Thanks!

  • Can you share the code snippet used for BatchWrite and also the version of DynamoDB Local which you are using and on which platform (docker or direct download).

  • I updated the original post with these details. Dmitry Balabanov provided an answer that seems to answer my question. But sharing this info seems helpful anyways. Thanks!

profile picture
ucrbgc
質問済み 2年前303ビュー
1回答
0
承認された回答

This is an implementation detail of the DynamoDBLocal package. It only applies to the local version of DynamoDB.

With this SELECT statement, the implementation queries the metadata looking for open table streams. If there is an open stream for the target table, the written items will be put into the stream as well.

profile pictureAWS
回答済み 2年前
  • Thanks for the info. It was my assumption it was something specific to the local version. Good to know that I'm not doing something wrong in the code though. Appreciate it!

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ