Limit IPs that can access an Elastic Beanstalk ALB listern or process

Our Elastic Beanstalk application has a three listeners 80 and 443 which forward to the default process that runs our web app, and another port that runs a monitoring app on a different port. I'd like to restrict which IPs can access the port of the monitoring app.

What's the best way to do this? Elastic Beanstalk makes the security group assigned to the ALB and it sets the allowed sources for the port of the monitoring app to 0.0.0.0/0. I can manually go in there and change it but I don't want to do that b/c then it will get blown away when I update our EB config laster.

I'm hoping for a solution that works well with EB and doesn't get blown away when the environment gets rebuilt.

Here's a bit of the relevant config for context

  AWSEBV2LoadBalancer.aws:elbv2:loadbalancer:
    AccessLogsS3Bucket: null
    AccessLogsS3Enabled: 'false'
    AccessLogsS3Prefix: null
    IdleTimeout: null
    SecurityGroups: sg-xxxx

AWSEBV2LoadBalancerListener5555.aws:elbv2:listener:xxxx:
    DefaultProcess: someprocess
    ListenerEnabled: 'true'
    Protocol: HTTPS
    Rules: null
    SSLCertificateArns: xxxx
    SSLPolicy: null

someprocess.aws:elasticbeanstalk:environment:process:someprocess:
    DeregistrationDelay: '20'
    HealthCheckInterval: '15'
    HealthCheckPath: /
    HealthCheckTimeout: '5'
    HealthyThresholdCount: '3'
    MatcherHTTPCode: '200'
    Port: 'xxxx'
    Protocol: HTTP
    StickinessEnabled: 'false'
    StickinessLBCookieDuration: '86400'
    StickinessType: lb_cookie
    UnhealthyThresholdCount: '5'