Transcription not recognizing multiple speakers

I am using AWS Transcribe to generate text transcriptions of audio files that usually have multiple speakers. I am using a python lambda with Boto3 Transcribe to start the transcription jobs.

Two of the files that I have attempted to do this on in the past week have not had the multiple speakers recognized by AWS Transcribe. Both files had two speakers, but the text transcription coming out of AWS Transcribe only shows one speaker. When starting the transcription job, the MaxSpeakerLabels setting was set to 2.

I can see how in one of the files, the voices might sound similar, but in the other file, the voices are very clearly different (female/male). Is there any other way to set the number of speakers on a transcription job to make sure that it is recognizing 2 speakers? Instead of just a max number?