Sub Domain Intermittently Dropping Requests Or Just Very Slow

I have a sub somain on my route53. Sometimes it works fine, other times, the request never hits the load balancer or is so slow in hitting it the client times out. I have checked my named servers and they match with the records. I have also ran the DNS through https://mxtoolbox.com/ and no issues. Any help would be greatly appreciated.