Dynamodb: Deleting all items & Read Write Capacity Query

Question

Hi,

I have a use case where I get about 20 links at a fixed time each day and want to store them in DynamoDb for analysis using AWS Lambda. I want to delete all prior stored links and then store new ones. I could not find a way to delete all items in Dynamodb like SQL so I am dropping the table and then re-creating it. I am not sure if this is the right way to go about it. Also, wanted to get inputs on required provisioned capacity for this exercise. I will have only about 20-30 rows with 5 columns. Below is my code which is currently working but wanted to get inputs if this is how this should be implemented and if there are AWS cost implications for this - I am new to AWS and currently on AWS Free Tier.

> def getLinks&Store():

links = getLinks()

dynamodb = boto3.resource('dynamodb', region_name = 'us-east-2')
   
    try:
        oldTable = dynamodb.Table('Links_Dynamic')
        if oldTable is not None:
            client = boto3.client('dynamodb')
            client.delete_table(TableName='Links_Dynamic')
            waiter = client.get_waiter('table_not_exists')
            waiter.wait(TableName='Links_Dynamic')
    except:
        print('Old Table is not available')
    linksDynamicTable = buildNewTable(dynamodb)
    
    for link in link: 
    	linksDynamicTable.put_item(Item= {'link': link})

> def buildNewTable(dynamodb):

table = dynamodb.create_table(
    TableName="Livestream_Dynamic",
        KeySchema=[
            {
                'AttributeName': 'LivestreamID',
                'KeyType': 'HASH'  #Partition key
            }
        ],
        AttributeDefinitions=[
            {
                'AttributeName': 'LivestreamID',
                'AttributeType': 'S'
            },
        ],
        ProvisionedThroughput= 
            { 
                'ReadCapacityUnits': 10,
                'WriteCapacityUnits': 10
            }    
        ,
        )
    table.meta.client.get_waiter('table_exists').wait(TableName='Livestream_Dynamic')
    return table

Regards,
Dbeings

Answer

A few things here, firstly you are correct, there is no way to delete all the items like you can in SQL I'm afraid. You can continue to drop the table as you have been doing, however it is likely a lot slower than deleting your items using the API. As you are on free tier, you are allowed to use up to 25WCU and 25RCU per region. If you have only a single table, you can allocate that capacity to the single table. I suggest removing Autoscaling should you wish to remain in the free tier.

As you only have a relatively small number of items, I so not see you exceeding 25WCU for the delete operations. You can delete the items in 2 ways, you can use [delete_item](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html#deleting-an-item) and run a loop to delete each item in your list, or you can add your `delete_item`s to a [batch_writer](https://boto3.amazonaws.com/v1/documentation/api/latest/guide/dynamodb.html#batch-writing) which will allow you to delete batches of items in single requests.

To understand you capacity requirements, keep an eye on your ConsumedCapacity metrics in CloudWatch, this will allow you to understand your consumption. Based on your described use-case, I believe consumed capacity will be relatively low.

Answer

As mentioned in AWS DynamoDB documentation, The **BatchWriteItem **operation puts or deletes multiple items in one or more tables. A single call to BatchWriteItem can write up to 16 MB of data, which can comprise as many as 25 *put or delete* requests.

DynamoDB doesn’t have a command that deletes multiple rows so you can perform a Scan API Operation, looping through the response to feed into a BatchWriteItem call. Here’s how in Python:

**How to utilize "BatchWriteItem" to delete multiple items (without deleting a table) is shown in the example on Stackoverflow:**

https://stackoverflow.com/questions/55169952/delete-all-items-dynamodb-using-python#answer-61641766

````
 with table.batch_writer() as batch:
        while page["Count"] > 0:
            counter += page["Count"]
            # Delete items in batches
            for itemKeys in page["Items"]:
                batch.delete_item(Key=itemKeys)
            # Fetch the next page
            if 'LastEvaluatedKey' in page:
                page = table.scan(
                    ProjectionExpression=projectionExpression, ExpressionAttributeNames=expressionAttrNames,
                    ExclusiveStartKey=page['LastEvaluatedKey'])
            else:
                break
````
As @Leeroy Hannigan mentioned, As you are on the free tier, you are allowed to use up to 25WCU and 25RCU per region.

**Check Free Tier Details:**
https://aws.amazon.com/dynamodb/pricing/provisioned/#DynamoDB_free_tier

Dynamodb: Deleting all items & Read Write Capacity Query

Contenuto pertinente