DynamoDB Read from Boto3 API Call

Question

I am currently attempting to analyze data from my DynamoDB database, with a focus on querying for both current and past dates. However, I have encountered some issues while using the AWS web app. Specifically, when I set the date as the primary key, the data is visible, but when attempting to scan it, the data does not appear.

In addition, I have also tried using the Boto3 API, but the output has shown an incorrect last evaluated key of 20230318, with a scanned count of 12279 that is not accurate. Below is an portion of the code to scan the table:
```
import boto3
import csv
from datetime import datetime, timedelta
from boto3.dynamodb.conditions import Attr

# Initialize the DynamoDB client
dynamodb = boto3.resource('dynamodb', region_name='us-east-2')

# Replace 'your_table_name' with the name of your DynamoDB table
table = dynamodb.Table('table_name')

# Get the current day timestamp
current_day = "20230405"
previous_day = "20230404" # datetime.now(timezone.utc).date().strftime("%Y%m%d")
#today = datetime.now()
#current_day = today.strftime('%Y%m%d')
#previous_day = (today - timedelta(days=1)).strftime('%Y%m%d')

# Scan the table
response_current_day = table.scan(
ProjectionExpression='ip_address, pk, c2_name, #counter',
ExpressionAttributeNames={
'#counter': 'counter',
},
FilterExpression=Attr('pk').eq(current_day)
)
```

Output of the request:
```
{'Items': [],
'Count': 0,
'ScannedCount': 12279,
'LastEvaluatedKey': {'pk': '20230318', 'sk': 'redacted'},
'ResponseMetadata': {'RequestId': 'redacted',
'HTTPStatusCode': 200,
'HTTPHeaders': {'server': 'Server',
'date': 'Wed, 05 Apr 2023 21:14:11 GMT',
'content-type': 'application/x-amz-json-1.0',
'content-length': '126',
'connection': 'keep-alive',
'x-amzn-requestid': 'REDACTED',
'x-amz-crc32': '3416724477'},
'RetryAttempts': 0}}
```

Leeroy Hannigan · Accepted Answer

I would first try to understand why you are doing a `Scan` if the partition key is date:

| pk | sk | data |
| --- | --- | --- |
| 20230318 | main | Data |
| 20230318 | active | Data |
| 20230318 | redacted | Data |
| 20230324 | main | Data |

If your data looks like above, and you want all the items which match the date `20230318` then you should use a [`Query`](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html) API call which means you do not have to read from the entire table, just the selection which matches your date.

```
response_current_day = table.query(
    ProjectionExpression='ip_address, pk, c2_name, #counter',
    ExpressionAttributeNames={
        '#counter': 'counter',
    },
    KeyConditionExpression=Key('pk').eq(current_day)
)
```

pk	sk	data
20230318	main	Data
20230318	active	Data
20230318	redacted	Data
20230324	main	Data

DynamoDB Read from Boto3 API Call

Relevant content