Fetching Neptune stream records without commitNum

The Lambda function in the Neptune-to-Neptune replication setup doesn't necessarily start with a predetermined commitNum. Instead, it uses different iterator types to fetch stream records, especially when starting up or when there's no specific sequence number to begin with.

To fetch stream records manually without knowing the exact commitNum, you can use the following approaches:

1. Use the TRIM_HORIZON iterator type: This will start reading from the oldest available record in the stream. You can use a command like this:

curl -X GET "https://your-cluster-endpoint:8182/propertygraph/stream?limit=1&iteratorType=TRIM_HORIZON" -H "Accept: application/json"

2. Use the LATEST iterator type: This will fetch the most recent record in the stream. The command would look like:

curl -X GET "https://your-cluster-endpoint:8182/propertygraph/stream?limit=1&iteratorType=LATEST" -H "Accept: application/json"

Once you've successfully fetched a record using one of these methods, you can extract the commitNum and opNum from the response. These values will be in the "lastEventId" or within the "records" array of the response. You can then use these values for subsequent requests with the AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER iterator types.

The Lambda function in the Streams Consumer Application likely uses a similar approach, starting with TRIM_HORIZON or LATEST when it first runs, and then keeping track of the last processed commitNum and opNum for subsequent polls.

Remember to replace "your-cluster-endpoint" with your actual Neptune cluster endpoint in the curl commands. Also, ensure that your security groups and network settings allow for this connection from your local machine to the Neptune cluster.
Sources
Calling the Neptune Streams REST API - Amazon Neptune
Neptune Streams Examples - Amazon Neptune