why does the whisper action on the polly console only work for the Mathew voice?

I recently tried the Polly console set to Neural and SSML and keyed in some text and tried several voices with the Whisper action and it works for the Mathew voice but does not work for any other one.

If its set to Normal instead of Neural the whisper does work for other voices but does not sound good. Is there a reason the whisper only works for that one voice using the Neural setting?