Skip to content

ChimeSDK Audio Only - not working/can't figure out

0

I'm using the chime sdk + transcribe integration and it seems to work fine when a user joins with video, however if I don't call start video input, and only call start audio, the transcriptions never work. I can't figure out why that is, and I've run the system on and off just commenting out startvideoinput and that is always enough to cause this change in behavior. Maybe I'm missing a setting/configuration that is starting video has to be sent?

Here's my client code (minus some extraneous application specific code):

if (audioDevices.length > 0){
   meetingSession.audioVideo.startAudioInput(audioDevice);
   console.log(audioDevice)
   console.log("Audio device found, joining")
 } else {
   console.log("No audio devices found, unable to join");
   leaveConferenceCall();
   return;
 }
 
 const audioElement = $('#conf_audio')[0];
 meetingSession.audioVideo.bindAudioElement(audioElement);
 meetingSession.audioVideo.chooseVideoInputQuality(480, 270, 10);
 document.querySelector('#conferenceVideos').show();
 
 if (videoDevices.length > 0 && videoDevice != 'none'){
   await meetingSession.audioVideo.startVideoInput(videoDevice);
   const element = getVideoPanel('self');
   meetingSession.audioVideo.startVideoPreviewForVideoInput(element);
 } else {
   console.log("No video devices provided, joining with audio only");
 }

 const video_update_observer = {
   videoTileDidUpdate: (tileState) => {
     if (!tileState.boundAttendeeId || tileState.localTile ) {
       return;
     }
     meetingSession.audioVideo.bindVideoElement(
       tileState.tileId,
       getVideoPanel(tileState.tileId)
     );
   }
 }

 meetingSession.audioVideo.addObserver(video_update_observer);

 const video_remove_observer = {
   videoTileWasRemoved: (tileId) => {
     if (tileId in videoPanels) {
       meetingSession.audioVideo.unbindVideoElement(tileId);
       videoPanels[tileId].remove();
       delete videoPanels[tileId];
     }
   }
 }

 meetingSession.audioVideo.transcriptionController?.subscribeToTranscriptEvent(transcriptEventHandler);

 meetingSession.audioVideo.addObserver(video_remove_observer);
 
 if (videoDevice != 'none'){
   meetingSession.audioVideo.startLocalVideoTile();
 }

 meetingSession.audioVideo.start();

If I either set the video device to 'none' or comment out the start video call, joining with just audio doesn't work and no transcript events are produced (beyond the initial start event). With video, works fine.

asked 3 years ago773 views
1 Answer
0

Hi,

Thank you for reaching out on the AWS re:Post forums. My name is Sean from Premium Support and I'm here to follow up on your question.

During my investigation into this problem, I found what seems to be an identical GitHub issue, which may have been opened by you as the code is the same: https://github.com/aws/amazon-chime-sdk-js/issues/2658

For posterity, you were able to resolve this problem yourself. The issue turned out to be related to async handling with the audio device specific code. You mentioned that the video handler uses 'await' to wait for the video device to become available:

 if (videoDevices.length > 0 && videoDevice != 'none'){
   await meetingSession.audioVideo.startVideoInput(videoDevice);

On the other hand, the audio device code doesn't follow this logic:

if (audioDevices.length > 0){
   meetingSession.audioVideo.startAudioInput(audioDevice);

Please let us know if you have any more questions or concerns.

AWS
SUPPORT ENGINEER
answered 3 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.