Delay in Kinesis video stream builds up over time

I am currently using https://github.com/awslabs/amazon-kinesis-video-streams-producer-sdk-cpp (gstreamer sample) to stream video from rpi to kinesis. The delay is around 10 seconds in the first hour. However, the delay gets built up and goes up to 30 to 40 sec which is not desirable for us. But if I refresh the stream it goes back to 10 sec delay. Could someone help me on how to reduce the delay build up or update the stream if delay is more than 10 sec?