Amazon Transcribe Python API: Event handler processes audio only after the stream ends

0

I am sending streaming audio data from web browser as a blob via websocket. In backend, I am using Django Channels' AsyncWebsocketConsumer to receive it, then send it to Amazon Transcribe and attempt to reply to the browser with the transcript in near real-time.

I am able to send the transcription for a single blob, but couldn't send it in a streaming manner. I see the event handler is triggered only if the stream ends, which led me to think, maybe I am not using the API correctly.

What I have tried so far, apart from the code below:

  1. Created a separate stream for each audio chunk. The error amazon_transcribe.exceptions.InternalFailureException: An internal error occurred. is thrown.
  2. Used asyncio.gather function to group stream.input_stream.send_audio_event function and handler.handle_events function. The handler is not invoked.
  3. Used asyncio.create_task(handler.handle_events) to create a non-blocking task for the handler. The handler is not invoked and also it didn't wait for 15 seconds. The task got completed immediately.
stream_client = TranscribeStreamingClient(region="us-west-2")

class AWSTranscriptHandler(TranscriptResultStreamHandler):
    def __init__(self, transcript_result_stream):
        self.channel_layer = get_channel_layer()
        self.channel_name = "test"
        super().__init__(transcript_result_stream)

    async def handle_transcript_event(self, transcript_event: TranscriptEvent):
        results = transcript_event.transcript.results
        for result in results:
            if not (result.is_partial):
                for alt in result.alternatives:
                    await self.channel_layer.send(
                        self.channel_name,
                        {"type": "send_transcript", "message": alt.transcript},
                    )


class ChatConsumer(AsyncWebsocketConsumer):
    async def connect(self):
        await self.accept()
        await self.send(
            text_data=json.dumps(
                {"type": "connection_established", "channel_name": self.channel_name}
            )
        )
        self.stream = await stream_client.start_stream_transcription(
            language_code="en-US",
            media_sample_rate_hz=48000,
            media_encoding="ogg-opus",
        )
        self.handler = AWSTranscriptHandler(self.stream.output_stream)

    async def disconnect(self, close_code):
        await self.stream.input_stream.end_stream()

    async def receive(self, text_data=None, bytes_data=None):
        if bytes_data:
            await self.stream.input_stream.send_audio_event(audio_chunk=bytes_data)
            await self.handler.handle_events()

    async def send_transcript(self, event):
        await self.send(
            text_data=json.dumps({"type": "transcript", "message": event["message"]})
        )

asked 2 years ago95 views
No Answers

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions