Access Secrets using VPC EndPoint in Kafka Event-source in EventBridge Pipes

1

**Scenario: **

  • MSK Cluster is in private subnet of VPC-1 and I am trying to create an EventBridge Pipe in VPC2 that gets data from the MSK cluster
  • VPC-1 and VPC-2 are connected using VPC peering
  • Kafka authentication details are stored in Secrets Manager
  • I am using "Self managed Apache Kafka" for Event source as both VPCs are in differet AWS accounts.

It works if I configure NAT gateway in VPC2. Is it possible to achieve the same functionality using VPC Endpoints?

Problem

  • I have deleted NAT gateway and I have created VPC endpoint for Secrets Manager as Event Source will need to access Kafka authentication information.
  • Also, added STS VPC endpoint
  • This setup fails to start the EventBridge Pipe with error message "PROBLEM: Pipe VPC event source require outbound internet access to send events to Pipes"

EventSource Setup

1 Respuesta
1
Respuesta aceptada

Adding the answer for community:

EvenBridge supports VPC interface endpoints but not for MSK/Self-Managed Kafka /Amazon MQ. So this is not a network issue but rather service support.

We need route out to internet to be able to use "Self Managed Kafka Event Source" in EventBridge Pipe.

respondido hace un año
profile picture
EXPERTO
revisado hace 5 meses

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas