Allow AWS Amplify access a restricted site for npm builds

Setup

I am using AWS Amplify to host my NextJS app. My backend is a headless CMS called Craft CMS (PHP). The NextJS app is using the GraphQL endpoint from Craft CMS for data fetching. The CMS is in EC2 server. It is connected to an Application Load Balancer.

Requirement

I want to be the only one who can access the CMS since I'm the only one using it. So the EC2 security group's source for HTTP and HTTPs is the ELB security group ID. The ELB security group has my IP for the HTTP and HTTPS traffic. This configuration works. I'm the only one who can access the site.

The issue

The problem is that when Amplify tries to build my frontend app, it's always build error. It seems that the Amplify can not reach the GraphQL endpoint when it tries to build the app.

Action taken

I tried adding AmazonEC2FullAccess to the Amplify Service role and it didn't work.

Temporary workaround

I turned on the access logs of the CMS and found the AWS IP of the Amplify app. I then added the IP to the ELB security group to allow access. This works but I don't know how often AWS changes the IP address and I know at point some point it'll break down.

Question

How do we allow Amplify to access the GraphQL endpoint for npm builds? Any ideas is greatly appreciated. Thank you.