Create API GW Websocket API that is only accessible from within a VPC.

API Gateway Websockets APIs do not support private APIs so you can't really prevent access to the API from any location. Saying that, you mention CloudFront as the way to access the API. If you use CloudFront, the requests are not routed via a VPC.

One way to achieve that only requests that came from CloudFront are handled by API Gateway is to add sone secret between CloudFront and API GW. This can be done by adding a new header with a specific value in CloudFront and create a Lambda Authorizer in API GW that verifies the value in the header.