IoT broker: Publish mqtt messages with messageExpiry doesn't seem to work

I'm using boto3 to publish messages to some mqtt topic. I've tried both approaches retain=True and retain=False messages with a generally low value for messageExpiry (initially 10 seconds and then 1, 60, 120, and other values in case that was the issue) but it doesn't seem to work.

With retain=True I'd expect the message to expire and delete itself after messageExpiryseconds, but it doesn't (a client connecting after the publish does receive the message, of course). With retain=False I would expect a client that connects a few seconds after the publish operation (but before messageExpiry seconds) to receive the published message, but it doesn't.

So not sure if I'm missing some configuration, messageExpiry doesn't work as I would expect (though according to the table from https://docs.aws.amazon.com/iot/latest/developerguide/mqtt.html#mqtt5 it should work) or I'm doing something wrong.