DynamoDB Streams - Are shards ordered in the stream description?


I’m aware that the items in the stream are ordered. My principle question then is about whether or not the shards are ordered in the stream description. If they are, I can always just grab the latest shard and process records from there; if not, I would need to keep track of the shards already processed in order to get only the new shard from the next iteration when I grab the stream description—I.e., after the current shard is closed and I have to quit out of the getRecords loop for that shard.

Further Context:
I have already complete a KCL implementation, but am looking at doing a lower-level implementation, as that looks to be more suited to our particular needs. Ultimately, I just need to get the latest record from the table (broadcast amongst all of the service nodes). That said, my quick question is as follows: is the shard list returned from DescribeStreamResult.getShards() an ordered list? (I presume this follows the general describe stream API, but have yet to see anything indicating whether or not this list is ordered.)

질문됨 8년 전616회 조회
1개 답변
수락된 답변

The DynamoDB streams api is "intentionally" similar to Kinesis streams (from the documentation here). This also states that: Using the Kinesis Adapter is the recommended way to consume Streams from DynamoDB. The API documentation for the Kinesis DescribeStream API states that:

There are no guarantees about the chronological order shards returned in DescribeStream results. If you want to process shards in chronological order, use ParentShardId to track lineage to the oldest shard.

The DynamoDB DescribeStream API contains the parentSharedId as well as SequenceNumberRange.

답변함 8년 전
profile picture
검토됨 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠