Access Secrets using VPC EndPoint in Kafka Event-source in EventBridge Pipes

1

**Scenario: **

  • MSK Cluster is in private subnet of VPC-1 and I am trying to create an EventBridge Pipe in VPC2 that gets data from the MSK cluster
  • VPC-1 and VPC-2 are connected using VPC peering
  • Kafka authentication details are stored in Secrets Manager
  • I am using "Self managed Apache Kafka" for Event source as both VPCs are in differet AWS accounts.

It works if I configure NAT gateway in VPC2. Is it possible to achieve the same functionality using VPC Endpoints?

Problem

  • I have deleted NAT gateway and I have created VPC endpoint for Secrets Manager as Event Source will need to access Kafka authentication information.
  • Also, added STS VPC endpoint
  • This setup fails to start the EventBridge Pipe with error message "PROBLEM: Pipe VPC event source require outbound internet access to send events to Pipes"

EventSource Setup

1 Respuesta
1
Respuesta aceptada

Adding the answer for community:

EvenBridge supports VPC interface endpoints but not for MSK/Self-Managed Kafka /Amazon MQ. So this is not a network issue but rather service support.

We need route out to internet to be able to use "Self Managed Kafka Event Source" in EventBridge Pipe.

respondido hace 9 meses
profile picture
EXPERTO
revisado hace 2 meses

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas