Transcribestreaming Java SDK error Your request timed out because no new audio was received for 15 seconds

The timeout you're experiencing is a built-in behavior of Amazon Transcribe Streaming, which will close the connection if no new audio is received for 15 seconds. This is by design and there isn't a direct "pause/resume" functionality for the standard streaming transcription service (unlike the Medical Scribe stream which does have this capability).

For your voice bot use case, you have a few options:

1. Continue with your current workaround of sending silent audio packets every ~10 seconds to keep the session alive. While this feels like a hack, it's actually a common approach for maintaining streaming connections when you need to pause audio input temporarily.

2. Create a new streaming session each time the user speaks. Although you mentioned this takes too long to initialize, you could potentially optimize this by:

• Pre-warming a new session in the background while the bot is responding
• Using connection pooling techniques
• Ensuring you're connecting to the closest AWS region to minimize latency

3. Implement a client-side buffer that continues to collect audio during the bot's response, but only sends it to Transcribe after the bot finishes speaking. This would require careful management of the audio stream.

4. Use WebSockets instead of the HTTP/2 streaming protocol if you're not already doing so, as they might provide more flexibility for your use case.

For your current workaround, you could structure your code like this:

private void sendSilentAudioToKeepSessionAlive() {
// Create a buffer of silence (all zeros)
ByteBuffer silentAudio = ByteBuffer.allocate(320); // Small chunk size

// Schedule sending silent packets every 10 seconds
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(1);
scheduler.scheduleAtFixedRate(() -> {
if (isSessionActive && isBotSpeaking) {
AudioEvent silentEvent = AudioEvent.builder()
.audioChunk(SdkBytes.fromByteBuffer(silentAudio))
.build();

// Send the silent audio through your publisher
audioPublisher.publish(silentEvent);
}
}, 0, 10, TimeUnit.SECONDS);
}

This approach, while not ideal, is a practical solution given the current limitations of the standard streaming transcription API.
Sources
AWS Transcribe Streaming error: HTTP/2 stream is abnormally aborted in mid-communication with result code 2 | AWS re:Post
Use StartStreamTranscription with an AWS SDK - AWS SDK Code Examples
Transcribing with the AWS SDKs - Amazon Transcribe
StartStreamTranscription - Amazon Transcribe