Playback of Audio + Video Stream failing for Kinesis Video Stream Media Player

The error you're encountering is related to track compatibility with the Kinesis Video Streams playback console. The console's media player has specific requirements for the media formats it can play.

Based on your error message, you're using H.264 video (V_MPEG4/ISO/AVC) and AAC audio (A_AAC), which are generally supported formats. However, there are specific requirements for playback through different Kinesis Video Streams APIs.

For playback of streams with both audio and video tracks, you need to ensure that:

1. Your codec private data (CPD) in each fragment remains consistent throughout the stream
2. The track configuration remains consistent (e.g., not changing from video-only to audio+video)
3. You're using a compatible API for your track combination

If you're using the console's Media Playback feature, it likely uses one of these APIs: GetClip, GetDASHStreamingSessionURL, or GetHLSStreamingSessionURL. Each has specific requirements:

• GetClip supports H.264 video with optional AAC audio
• GetDASHStreamingSessionURL supports H.264 video and AAC audio for MP4 format
• GetHLSStreamingSessionURL supports H.264 video and AAC audio for HLS MP4 format

To troubleshoot this issue:

1. Try using the GetMedia or GetMediaForFragmentList APIs, which don't have track-type limitations
2. Verify your GStreamer pipeline is correctly formatting the MKV files for Kinesis
3. Test your pipeline by replacing kvssink with matroskamux and filesink to create a local file, then verify its compatibility

For GStreamer, ensure you're using at least version 1.14 and that the Kinesis Video GStreamer plugins are installed correctly. You can validate this by running gst-inspect-1.0 | kvssink.
Sources
Video playback track requirements - Amazon Kinesis Video Streams
Play back media from your Kinesis video stream - Amazon Kinesis Video Streams
Streaming videos on kinesis using kvssink | AWS re:Post