Firehose Active Partition Count Exceeded

1

Hi,

I am writing data to S3 from an event bus using dynamic partitioning of the scheme data/events/!{partitionKeyFromQuery:source}/!{partitionKeyFromQuery:detail_type}/!{partitionKeyFromQuery:year}/!{partitionKeyFromQuery:month}/!{partitionKeyFromQuery:day}/!{partitionKeyFromQuery:hour}/

When there is an influx of data I experience an error in firehose of "Firehose Active Partition Count Exceeded" (default is 500 active partitions)

Q - Given one record in the buffer, how many active partitions would the above partition scheme create?

Thank you!

asked a year ago637 views
1 Answer
0

Can you anyone help?

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions