Skip to content

Transcribe with React Native and Gen 2 Amplify

0

Hello,

I would like to send to Transcribe voice information from React Native app using Amplify Gen 2 according to the documentation.

The problem is that this code:

for (const chunk of chunks) {
            try {
              console.log(chunk)
              const { transcription } = await Predictions.convert({
                transcription: {
                  source: {
                    bytes: chunk
                  },
                  language: 'en-US',
                  region: 'eu-central-1'
                },
              });
              console.log(transcription)

returns in the transcription variable empty strings:

LOG {"fullText": ""}
LOG  {"fullText": ""}

although I am feeding data in (variable chunk) and I get empty string for each chunk:

{"data": [108, 1, ........... ..... 9, 0, 100, 1], "type": "Buffer"}

setup in backend.ts looks OK

const region = 'eu-central-1';

backend.auth.resources.authenticatedUserIamRole.addToPrincipalPolicy(
  new PolicyStatement({
    actions: [
      "transcribe:StartStreamTranscriptionWebSocket",
    ],
    resources: ["*"],
  })
);

backend.addOutput({
  custom: {
    Predictions: {
      convert: {
        transcription: {
          defaults: {
            language: "en-US",
          },
          proxy: false,
          region: region,
        },
      },
    },
  },
});

as well as configuration inside the App.tsx with the prediction API initialized.

If I increase the size of the chunk, I get error that stream is too big, meaning I am able to actively communicate with the service but it does not return any error upon the data I send, only empty strings.

Do you please have any ideas?

Thank you very much.

1 Answer
0

I apologize, but it seems there might be a misunderstanding in your approach to using Transcribe with React Native and Amplify Gen 2.

Amplify Gen 2 doesn't have built-in support for the Predictions category, which was used in Amplify Gen 1 for services like Transcribe. In Gen 2, Amplify offers built-in support for Auth, Data, Storage, and Functions.

For using Amazon Transcribe in a React Native app with Amplify Gen 2, you would need to take a different approach:

  1. Set up an AWS Lambda function that can interact with Amazon Transcribe.
  2. Create an API (REST or GraphQL) in your Amplify backend that triggers this Lambda function.
  3. From your React Native app, you would then call this API endpoint, sending the audio data.
  4. The Lambda function would process this data using Amazon Transcribe and return the results.

Your current code seems to be using the Predictions API from Amplify Gen 1, which is not compatible with Gen 2. This explains why you're not getting the expected results.

To implement this in Gen 2:

  1. Create a Lambda function in your backend that uses the AWS SDK to interact with Transcribe.
  2. Set up an API in your Amplify backend that invokes this Lambda function.
  3. In your React Native app, use the Amplify API category to make requests to your new endpoint, sending the audio data.
  4. Handle the response from the API in your app, which should contain the transcription results.

Remember to ensure your IAM roles are set up correctly to allow the Lambda function to access Transcribe.

This approach aligns with the Gen 2 architecture and should allow you to use Transcribe effectively in your React Native app with Amplify Gen 2.

Sources
FAQ - AWS Amplify Gen 2 Documentation

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.