Provide cross-account access to S3 buckets in Account B using users on IAM Identity center only.

If you would like AWS IAM Identity Center users to access resources in different AWS accounts that are within the same AWS Organization, you can assign user access to those accounts via Identity Center. As part of this process you have to select the appropriate permission set required to perform the actions you need (i.e. operating on the S3 bucket).

If the AWS accounts are not within the same AWS Organization, you can use cross-account IAM roles for console access to your resources. You can follow the cross-account IAM roles steps from this Knowledge Center article and switch to the role from the AWS Console when you require to perform operations on the S3 bucket.

Hope this helped.