Accessing Elastic Cache Redis from Celery in a docker-compose locally

Question

Dear all,

I am developing a python FastAPI application that uses Celery and Redis. So far I have run these services locally using docker-compose and everything worked fine. I would now like to use the Elastic Cache Redis instance from my company, however I have problems connecting to it.

According to celery documentation https://docs.celeryq.dev/en/stable/userguide/configuration.html#std-setting-broker_url the CELERY_BROKER_URL should be `redis://:password@hostname:port/db_number` or `transport://userid:password@hostname:port/virtual_host `. Until now I simply had `redis://redis:6379/0` because I had redis running locally, however I am unsure about what to put when using the remote instance.

I have tried using the `Primary endpoint` as the `hostname`, the `Account ID` as `userid` and the token as the password, is this correct? Also, is it actually possible to access the remote instance locally from celery?

Answer

Hello, thank you for your post. Elasticache is designed to be used internally to your VPC and external access is not enabled. Should you need to access your Elasticache cluster from outside your VPC, you can do so via VPN. Please refer to the "Accessing ElastiCache resources from outside AWS" section on the page referenced below[1]. After you have established a VPN connection, you will be able to access the cluster simply using redis://redis:6379/0 as you do from within the VPC.

References:
[1] https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/accessing-elasticache.html

References:  
$$1$$ https://docs.aws.amazon.com/AmazonElastiCache/latest/mem-ug/accessing-elasticache.html

Accessing Elastic Cache Redis from Celery in a docker-compose locally

Relevant content