DynamoDB query between dates

I'm trying to implement a query between two dates but from what I understood here https://stackoverflow.com/a/38790120/10586800 and from many other sources it's not possible. You must pass an id to filter with to use query.
The only way to only filter between dates is to use scan... then I thought, based on my table structure, save all the different sourceName at a different table that will hold all my sourceName and when I will want to query my table between dates, I will first query for all my sourceName from that table and then I will query my main table with each sourceName.

I'm just trying to understand if it's really better then scan or maybe there is a better option?

Here is my table structure:

Primary partition key recordID (String)

createdAt: Number,  
data: Map,  
recordId: String,  
sourceName: String,  
updatedAt: Number  

example:

All this process works on lambda, I'm getting from the user lastUpdateFrom (number) and lastUpdateTo (number)

then I want to do something like this:

 let params = {  
        TableName: process.env.RECORDS_TABLE_NAME,  
        IndexName: 'updatedAt-index',  
        KeyConditionExpression: "updatedAt BETWEEN :lastUpdateFrom AND :lastUpdateTo",  
        ExpressionAttributeValues: {  
          ":lastUpdateFrom":{  
            N: lastUpdateFrom,  
          },  
          ":lastUpdateTo":{  
            N: lastUpdateTo,  
          }  
        },  
        ProjectionExpression: "recordID",  
      };  

But it's impossible, then what I thought is to fetch all the sources and query each source with the lastUpdateFrom and lastUpdateTo and using query with the index I created (partition key sourceName and sort key updatedAt)

example:

const sources = await getAllRecords();  
const dbCalls = \[];  
for (var i = 0; i < sources.Count; i++) {  
   dbCalls.push(getRecordsBetweenDatesFilteredWithSource(sources.Items\[i].sourceName.S, lastUpdateFrom, lastUpdateTo, skip, limit));  
}  
return Promise.all(dbCalls);  

and then my query will look like this:

let params = {  
        TableName: process.env.RECORDS_TABLE_NAME,  
        IndexName: 'sourceName-updatedAt-index',  
        KeyConditionExpression: "sourceName = :sn AND updatedAt BETWEEN :lastUpdateFrom AND :lastUpdateTo",  
        ExpressionAttributeValues: {  
          ":lastUpdateFrom":{  
            N: lastUpdateFrom,  
          },  
          ":lastUpdateTo":{  
            N: lastUpdateTo,  
          },  
          ":sn":{S: sourceName}  
        },  
        ProjectionExpression: "recordID",  
};  

Edited by: ronmar on Oct 8, 2020 11:52 AM

Edited by: ronmar on Oct 8, 2020 11:53 AM

Edited by: ronmar on Oct 8, 2020 11:53 AM