disable read permission on certain folders on s3

I have a bucket with 2 folders inside, I want my IAM user to be able to access just one of them, I tried using group policy, but the actions s3:ListAllMyBuckets and s3:ListBucket (which is mandatory for listing objects) grants access to all objects in the bucket, denying the action s3:GetObject in the bucket policy doesn't help too. I'd like to create an exception for certain folders, is it possible?