SQS with Dynamodb & AppSync

0

We have tables on DynamoDB and use it as a real-time database, connecting it with the application through AppSync. Can we use SQS to queue orders and ensure their validity before processing without affecting real-time capabilities? I'd like to know the correct method of integration ... should it be before AppSync, which connects directly to DynamoDB, or between AppSync and DynamoDB, and how to properly establish this?

2 Answers
0

If you introduce SQS in the middle, you are making a synchronous API call asynchronous, which means that the client will not get a response to their API call. If it is a mutation, it may not be an issue, but if it is a query, you will not the intended response.

If this is OK with what you need, you will probably need a Lambda resolver (instead of the DDB resolver) which will send the request to SQS and then another Lambda will process those requests.

If all you need is add additional business logic, you may use a Lambda resolve that will work directly with DDB, without the queue. This way you are still synchronous and you can add the additional code that you need.

profile pictureAWS
EXPERT
Uri
answered 12 days ago
0

Hi,

Depending on your actual requirement details, there are multiple patterns that you can use:

  1. As @Uri mentioned above, you can replace DynamoDB datasource with a Lambda datasource and add as many extra request validation steps as you would like to, before writing data to the DynamoDB. Keep in mind that, although this approach provides you with the most flexibility, you will have to take into account that a Lambda function will be invoked for each AppSync mutation, thus you will have to account for Lambda concurrency limits, cold-start time, etc.
  2. If the to task to "ensure their validity before processing" doesn't involve complex calculations or an external endpoint communication (other than the destination Amazon DynamoDB table), you can add your validation logic directly into the DynamoDB resolver's request transformation using JavaScript. See JavaScript resolvers for more details. These scripts will run in the context of the AppSync instance and are not Lambda functions, e.g. they don't count towards Lambda concurrency limits. It is important to understand that JavaScript resolvers are using APPSYNC_JS runtime which has more limitations on the available functionality, compared to the NodeJS Lambda functions;
  3. If you still need the power of using Lambda functions, you can consider using Pipeline Resolvers to perform request validation using a Lambda function, followed by a DynamoDB datasource resolver. This design will provide you with a clean segregation of concerns - Lambda function will be responsible for validation only and will return validated response to the pipeline, while DynamoDB datasource resolver will be responsible for saving data in the table.
  4. Adding a queue to the architecture doesn't solve the problem of validating the requests and you will still have to choose between the limited functionality of the JavaScript request transformations and Lambda functions. You can certainly create an HTTP Datasource and send messages to an SQS queue, EventBridge event bus, etc. , but that approach will add extra latency for saving data into the DynamoDB table. I would recommend it for use cases when the destination DynamoDB is being throttled and it negatively affects your user experience.
  5. If your task is to perform certain actions after information is saved in the DynamoDB table, that is a different story and you may consider using DynamoDB Streams or Kinesis Data Streams to send notifications to that extra business logic.

Regards.

AWS
answered 11 days ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions