Issue to hit the endpoint of AWS docker build/push/run

Hi Team,

As a part of an in-house project, we are working with a few AWS services, and AWS Batch is one of them. The objective of our exercise was to hit the end-point of an AWS EC2 instance, through an API interface, from a docker running on the same EC2 instance. This docker has a front end tool up and running all the time and when a "Run" button is triggered from the tool, the API should hit our code base in the EC2 instance to run a python pipeline which will generate results in the tool.

In the process of doing the above we have tried hitting the codes directly by specifying the paths to the code base, but we faced issues related to the environment as the docker environment from which we are trying to hit the codebase and the EC2 environment had conflicts. So as another attempt, we tried using the DIND framework, which is a docker inside a docker setup to hit the codebase, but were faced with other issues as mentioned in the “aws_error” as below:

File "/usr/local/lib/python3.8/site-packages/botocore/httpsession.py", line 477, in send raise EndpointConnectionError(endpoint_url=request.url, error=e) botocore.exceptions.EndpointConnectionError: Could not connect to the endpoint URL: "https://batch.ap-south-1-MCBC.amazonaws.com/v1/describejobqueues"

Following is the logical flow of steps in the specified order: We have forecasting module which runs inside a docker, this module have below functionalities:

1. Building a docker image for AWS batch(docker image inside forecasting docker)
2. AWS CLI login inside the forecast docker
3. Creating a docker or creating layer for AWS docker
4. Pushing docker image of AWS to AWS batch
5. Running final forecasting model inside the AWS batch

When we run the main file in docker we are able to do step 1-4 but we are facing issue in step 5.

docker run -v $HOME/.aws:/root/.aws -e ENCODED_BATCH_CONFIG_JSON=$ENCODED_BATCH_CONFIG_JSON -e ENCODED_DATA_CONFIG_JSON=$ENCODED_DATA_CONFIG_JSON $IMG_NAME_TAG python launch_batch_jobs.py #/bin/bash

I would appreciate any kind of help in the Issue I'm currently facing. Please feel free to reach out in case of more queries about the issue. Thanks in advance!

Best Regards, Ajit Kumar Voraboina ajit.kv-v@adityabirla.com Aditya Birla Group GDNA floor