How to use private AWS docker images to build our custom images

Question

There are some private AWS docker images like the one used by sagemaker for environment sagemaker-sparkmagic. I know the ARN, which is arn:aws:sagemaker:eu-west-1:470317259841:image/sagemaker-sparkmagic.

I need to install extra packages in that image, I just want to extend it with my own Dockerfile. However I'm not able to docker pull that image. I tried the following command

```
- docker pull 470317259841.dkr.ecr.eu-west-1.amazonaws.com/sagemaker-sparkmagic
```

Loging in docker using:

```
- aws ecr get-login-password --region eu-west-1 | docker login --username AWS --password-stdin 470317259841.dkr.ecr.eu-west-1.amazonaws.com
```

I also tried to list the available images in there using the following, but it returns ACCESS DENIED:

```
curl -X GET -u AWS:$(AWS_PROFILE=at3w aws ecr get-login-password --region eu-west-1) https://470317259841.dkr.ecr.eu-west-1.amazonaws.com/v2/_catalog
```

Is it possible what I am trying to do?

Thank you!

Answer

ECR has a a [public gallery](https://gallery.ecr.aws/) where docker images can be pulled from. I tried [searching](https://gallery.ecr.aws/?searchTerm=sagemaker-sparkmagic&page=1) for the image you specified in there and could not find it. Along with the information you have shared that it is "private", that leads me to believe that this image is internal to the AWS system and is not available for public download. If that is the case, you won't be able to download it.

[This page](https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks-available-images.html) details some sagemaker images that are used by AWS. The one you are referencing is taken from [this github repository](https://github.com/jupyter-incubator/sparkmagic). So at the very least, you could build on top of the Dockerfile in that repository for what you are trying to accomplish.

Answer

Ok thank you very much for your answer! The problem is that to build the image from the Dockerfile is slow, it would be useful to have those images in the public gallery, but at least there is a solution :-)
Thanks!

How to use private AWS docker images to build our custom images

Relevant content