By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Microservices with shared database

0

Hi there,

As is not recommended by "Database per service" design pattern, every integration between microservices should be done with any messaging system?

We have an application where users can upload videos. The API is available using GraphQL, and we have federation to route the video uploads to a cluster of servers responsible to create the video in the database (RDS).

Once the video is uploaded to S3, a service that is triggered by a S3 event start a MediaConvert job to create a HLS profile. Once completed, we need to mark the video as available to viewers (updating the table).

What is the best practice to do this? The convert service should connect to the database and update the record? Execute a service API to update the record? Send a SQS message that will be handle in the cluster that is connected to the database?

  • Antonio_Lagrotteria EXPERT
    a year ago

    If below helps, please upvote/accepted. It would be appreciated!

Topics
DatabaseServerlessApplication IntegrationMicroservices
Tags
Aurora PostgreSQLAmazon EventBridgeAmazon Simple Queue ServiceAmazon DynamoDBMicroservices
Language
English
Luciano
asked a year ago301 views
1 Answer
0

The answer is it depends. This is due to number of users, throughput request/seconds and other values.

A suggestion could be to send the S3 event notification to an SQS queue or an SNS Topic. Then the consumer could be a Step Function that has 2 states:

  1. the Media converter.
  2. update of the database. This could be a lambda function which has an RDS proxy integration in front of the RDS database.

In this serverless way, you could retry, keep an eye on the flow, ensure the converter does not get overwhelmed by requests.

profile picture
EXPERT
Antonio_Lagrotteria
answered a year ago
  • Luciano
    a year ago

    Great suggestion, thanks!

    My concern still on the 2nd state of the step function. So would the step function connect to RDS (proxy or not) and not use another service that perform operations in the table?

  • Antonio_Lagrotteria EXPERT
    a year ago

    In this scenario I would have the second step using a Lambda or container that connects directly to RDS via using an RDS proxy.

    I see your point of abstracting the db layer by having it responsible to handle db interactions. In my experience it just felt a bit over engineering as it would have pretty much the same role as an RDSProxy, but if you want to create an api that is used as api layer that then calls rds proxy, the there may be more latency and be potentially a bottleneck, but up to you.

    Hope the answer makes good sense and helps you!

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content