Amazon MQ: Using rabbitmq shovels to private brockers

  1. We have setup 2 private rabbitmq broker clusters(rabbitmq-prod & rabbitmq-test) in our VPC on the same subnets.
  2. Created security groups that allow traffic from to port 5671 and allow all outgoing traffic.
  3. Created shovel on rabbitmq-prod that sends traffic to rabbitmq-test.

But the shovel stays in "starting" state, and the logs says that the connection times out. Is there any more configuration that has to be done. Or doesn't the implementation support shovels to private brokers?

asked 2 years ago1004 views
3 Answers

Hi RickarK, Your observation is correct. Currently, our Amazon MQ for RabbitMQ brokers can only connect to public endpoints for Shovel or for Federation. Although you provision brokers in the same subnets, it is only the VPC endpoints for the brokers which are located in your subnets. The physical broker instances are in a separate managed VPC which are not connected to each other. They can only connect using a public endpoint like a NLB.

Thanks, Mithun

answered 2 years ago

Hi RickarK, i had a similar issue using a single private broker with different virtual hosts, where shovels kept starting and never running. Solution was using at the uri as amqps://user:[redacted]@ private amazonmq brokers have IP addresses. have you try to use the IP to allow shovel running between two different private brokers?

answered 8 months ago

Hi RickarK, Yes it's right Shovel does not transfer messages directly between private brokers. But you can make this happen with the help of one public broker between them. you need to create one public broker to act as a middleman between two brokers.

so there is basically two process: first, you have to transfer messages between your source private broker to the public broker, and second is transfer message between the public to destination private broker. in both cases, you need to create a shovel in the private broker.

first process: in your private source broker create one shovel. Give the queue name for the source and destination broker. but When specifying src-uri (amqp://), do not change this field because the rabbitmq automatically takes care of its own URI if you give the src-uri in this amqps://{broker-name}:{password}@{host}:{port} formate it won't work and stuck on the starting state. Now change dest-URI according to the public broker URI; for example amqps://{broker-name}:{password}@{host}:{port}. create the shovel and it starts in running state. so the first step is done all the messages are transferred from a private source to a public broker.

second process: in your destination broker create a shovel and now your public broker is your source broker and the destination broker is your private broker. so the src-uri is amqps://{broker-name}:{password}@{host}:{port} and dest-uri is amqp:// again do not change this fild. now create a shovel with the appropriate source and destination queue name. check that the shovel is in running state and all the messages are transferred between public to private.

so in this way, you transfer messages between your two private brokers without much effort.

answered 4 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions