I have a dynamo db table that looks something like this:
RequestType (PK) | CreationTime (SK) | Other |
---|
A | 2 | Data |
A | 4 | Data |
B | 1 | Data |
B | 5 | Data |
B | 7 | Data |
My main requirement is to fetch the oldest item in a partition (based on CreationTime). Essentially, should behave like a FIFO queue.
I'm using DynamoDbMapper to query this table using the partition key value, scanIndexForward=true and limit=1. Now when i try to get the first item in the results, it takes around 25 seconds (there were ~15,000 items in the partition).
Here's the function used:
public Optional<MyDdbItem> getLastItemInPartition(@NonNull final String partitionKey) {
final DynamoDBQueryExpression<MyDdbItem> queryExpression =
new DynamoDBQueryExpression<MyDdbItem>()
.withHashKeyValues(MyDdbItem.builder()
.requestType(partitionKey).build())
// This is set true for ascending order of sort key (creationTime)
.withScanIndexForward(true)
// Fetch only one item
.withLimit(1);
Optional<MyDdbItem> result = Optional.empty();
try {
final PaginatedQueryList<MyDdbItem> results =
getDynamoDBMapper().query(MyDdbItem.class, queryExpression);
result = results.stream().findFirst();
} catch (final Exception exception) {
DDBExceptionHandler.handleDDBExceptions(exception);
}
return result;
}
Things i verified:
- The table metrics indicate that the query latency has never gone beyond 2 ms.
- Performed a similar query from a python script, and it took around 500 ms.
Python script:
def get_last_item_in_partition(table_name, partition_key_value):
query_params = {
'TableName': table_name,
'KeyConditionExpression': '#pk = :pkval',
'ExpressionAttributeNames': {'#pk': 'RequestType'},
'ExpressionAttributeValues': {':pkval': {'S': partition_key_value}},
'ScanIndexForward': True,
'Limit': 1
}
try:
response = dynamodb.query(**query_params)
items = response['Items']
if items:
return items[0]
else:
return None
except NoCredentialsError as e:
print(f"Error: {e}")
return None
Posting here to verify if the index-design accommodates my requirement.