Use Neptune Streams as a lambda event source mapping?

0

Similar to, for example, using an SQS as a source to trigger a lambda that processes its messages, I would like to use the Neptune DB Stream as an event source for lambda. Does AWS Neptune support this?

I read here that the records from the stream are accessible via API. This is how I'm accessing them currently, but it's not as elegant of a solution as being able to 'natively' connect neptune with lambda via CDK configuration. https://docs.aws.amazon.com/neptune/latest/userguide/streams.html

2 Answers
2

I can attest that the AI response is mostly correct. I would add that Neptune already has a poller framework that you can extend (referenced in this blog post: https://aws.amazon.com/blogs/database/capture-graph-changes-using-neptune-streams/).

I've also seen patterns where customers will use the poller to then send events to Event Bridge. And then use Event Bridge and related rules to then trigger Lambda functions.

profile pictureAWS
answered a month ago
profile pictureAWS
EXPERT
reviewed a month ago
1

As of now, AWS Neptune does not natively support using Neptune Streams as a direct event source mapping for AWS Lambda, unlike services such as DynamoDB Streams or Kinesis Data Streams.

While Neptune Streams provide a way to capture changes in your Neptune database, they don't have built-in integration with Lambda in the same way that some other AWS services do. The current method of accessing Neptune Streams records via API, as you mentioned, is the standard approach.

To create a solution that mimics event source mapping, you would need to create a custom setup. This might involve:

  1. Setting up a Lambda function that periodically polls the Neptune Streams API.
  2. Processing the stream records in your Lambda function.
  3. Managing your own checkpointing to keep track of which records have been processed.

This approach, while not as streamlined as native event source mappings, can still be effective. You could schedule the Lambda function to run at regular intervals using Amazon EventBridge (formerly CloudWatch Events).

It's worth noting that AWS continuously evolves its services, so while this capability isn't available now, it could potentially be added in the future. If this feature is important to your use case, you might consider submitting a feature request to AWS.
Sources
Process Amazon DocumentDB events with Lambda - AWS Lambda
Tutorial: Using AWS Lambda with Amazon DocumentDB Streams - AWS Lambda
Process DynamoDB records with Lambda - AWS Lambda

profile picture
answered a month ago
profile pictureAWS
EXPERT
reviewed a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions