You are correct, connectivity to an S3 Gateway endpoint would not work but connectivity to an S3 Interface endpoint should work. Suggest following troubleshooting steps mentioned here. Before trying the connectivity remotely from the ClientVPN client I suggest testing the connectivity locally from within the VPC where your ClientVPN endpoint is spun up. Also note that ClientVPN does an SNAT so traffic to the destination will be sourced by the ENI of the CVPN endpoint, so need to make sure those IPs are whitelisted as necessary.
You can use two types of VPC endpoints to access Amazon S3: gateway endpoints and interface endpoints (using AWS PrivateLink).
A gateway endpoint is a gateway that you specify in your route table to access Amazon S3 from your VPC over the AWS network. Endpoint connections cannot be extended out of a VPC. Resources on the other side of a VPN connection, VPC peering connection, transit gateway, or AWS Direct Connect connection in your VPC cannot use a gateway endpoint to communicate with Amazon S3.
Interface endpoints extend the functionality of gateway endpoints by using private IP addresses to route requests to Amazon S3 from within your VPC, on premises, or from a VPC in another AWS Region using VPC peering or AWS Transit Gateway.
Looks like VPC endpoint does not support static website ,it will need a proxy which does not fit our requirement
Cross account access from Athena to S3Accepted Answerasked 3 years ago
S3 static website domain not workingasked 2 months ago
Static website in S3 not working with HTTPSasked 2 years ago
Cross account copy from S3 to Deep ArchiveAccepted Answerasked 2 years ago
OAI or not OAI for serving a static website in S3 using CloudFrontasked 6 months ago
s3 static private website 403 forbidden over cross account client VPNasked 23 days ago
S3 Static Website Objects 403 Forbidden when Uploaded from Different Accountasked 5 months ago
Access bucket s3 from a role on another accountasked a year ago
S3 Static Website RoutingRules when using Cloudfront and a domain nameasked 3 years ago
Across Account S3 Bucket Get Access Odditiesasked 4 years ago