Service for storing frequently changing bitmap

0

I have a problem where I have a bitmap that is displayed to users on a webpage, each user can change the colour of a singular pixel. I intially thought about using S3 to store the bitmap and then using a lambda to get the bitmap, update one pixel and override the pervious bitmap, but it seems this will generate a pretty large cost due to lots of PATCH/PUT's. The updates to the bitmap will be streamed to each client (not the entire bitmap, just the pixel location and new colour)

The bitmap will be read everytime the user loads the page to display the current state, and it must be the most recent version (will be a bit of a mess if the clients get all out of sync).

I thought about putting this data into a DynamoDB but then its pretty expensive to retrieve all of the pixels and generate a bitmap (bitmap is 128x160 which makes for 20,480 entries!)

Is there a service that I am overlooking (highly likely) that fits this need pretty well

TLDR: Is there a service that is good (cost wise) for frequently (max 10 times per second) changing files

opugh
asked a year ago206 views
3 Answers
0

Super interesting problem! To throw maybe a curveball at you but have you considered kinesis? I'm assuming each cord is a hex value. So you could use API gw to receive the state change and put it to a kinesis stream. At which point you could have lambda consumers that write to a store (maybe elasticache write-throughs and only persist state to a storage system every so often (sync every minute or so). Then for syncing changes between clients, you'd have a step in the lambda consumer that posts a websocket event back to the API gw for others to consume. Using kinesis also guarantees event ordering so people should be experiencing nearly the same order of events.

answered a year ago
  • I had not even considered kinesis... Yeah, each coord would be a hex. That looks like a really nice solution. Then I am assuming that when users load the page for the first time and need the most up to date version of the bitmap I could have a lambda that could pull it out of the elasticache and return it back to the client, then they can begin to receive the kinesis updates to remain in sync with other users. That would work, right?

    Then I have the S3 with all of the backups of the bitmap.

  • I guess another way of getting the most up to date version of the bitmap could be to have another lambda behind API gw to get the most recent S3 version (a minute old max) and then apply all of the changes since the S3 bitmap was last updated (get these changes from an elasticache or some other fast storage) then return that bitmap, and that should be the most up to date, but I that isnt a massively scalable solution... as I guess if there were a thousands of changes in the last minute (highly unlikely) then that would be super slow.

    Also a lot of repeated work as the lambda would do this for every new user, not sure...

0

Have you considered having the entire bitmap in DynamoDB? Considering 16-bit color depth it would take 40 KB, which is well below the 400 KB maximum item size. You could have a single item table with a version field you'd use to do conditional updates.

Each update would take 40 WRUs and each read 10 RRUs, would that cost be manageable?

To implement the change notification, you could front DynamoDB with AppSync and use mutations. Those mutations would use a Lambda resolver that performs the conditional write. Using mutations you can have the clients subscribe to changes.

profile pictureAWS
EXPERT
Tasio
answered a year ago
0

Have you considered a relational database? The schema you are describing could be a very simple table in a database.

AWS
MODERATOR
philaws
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions