Can not log out with IAM Identity Center SAML

I set up an application to use SAML SSO with IAM Identity Center. The login is working as expected, however, when I use the IAM Identity Center sign-out URL (from the Identity Center metadata), I only get redirected to the AWS access portal and the user is still logged in.

Expected behaviour (as it is working with other identity providers): the user logged out and it is being redirected to the application home page.

I configured the Application start URL as my application's URL but I haven't set any relay state. Is there anything else that needs to be configured?