My instance is accessible from ipv4 address and public DNS. I want it to be accessible from domain name only.

Sounds like you're running a single WordPress instance with a public IP? The best approach is to make it private and put it behind an AWS load balancer or CloudFront distribution. Probably CloudFront if you're only running one instance. Then your DNS can be an A "Alias" record in Route 53 - a special link to an AWS service that isn't tied to a specific IP address (though of course, being DNS, it will still resolve on the fly to an IP address).

Note a custom origin like this behind CloudFront still requires the instance to have a public IP address but it can be locked down using an AWS-managed prefix list for CloudFront - it opens ingress only to CloudFront IPs so that's the only way people can access your site - through CloudFront.