Design table for multi-tenant solution

0

Hi, I'd need to use DynamoDB to store some simple products metadata keys.

PKSKValue
product1key1123
product1key2456
product2key2746
product2key3946
product3key1678
product4key3987

The most common access pattern will be to write/read/update values specifying PK and SK ie: Get the value for product1 key1.
But also list all the products, what would be the best option for that without performing a full Scan ?

Should I then go multi-tenant for the same I was thinking to do something like:

Table-Multitenant

PKSKValueProductIDTenantID
tenant1#product1-product1tenant1
tenant1#product1key1123
tenant1#product1key2456
tenant1#product2-product2tenant1
tenant1#product2key2746
tenant2#product1-product1tenant2
tenant2#product1key1123
tenant2#product1-product2tenant2
tenant2#product2key2746

And create a GSI (GSI_Tenant_Product) with PK: TenantID and SK: ServiceID.
So for getting all the list of products for a specific tenant I can do:

SELECT ProductID FROM "Table-Multitenant"."GSI_Tenant_Product" WHERE TenantID = 'tenant1'

Accessing for reading or writing the single keys will simply be something like:

SELECT Value FROM "Table-Multitenant" WHERE PK = 'tenant1#product1' AND SK = 'key1'

I'm not totally convinced this is the best approach, mostly considering the introduction of the fake key (-). What would be other alternatives ?

Thanks

asked 2 years ago1738 views
4 Answers
2

But also list all the products, what would be the best option for that without performing a full Scan ?

You want all the products, you must do a full table Scan, why would you not want to?

The second approach would be better modeled like this:

PKSKAttr1Attr2
tenant1product1#key1DataData
tenant1product2#key1DataData
tenant2product1#key1DataData

Now you no longer need a GSI.

So for getting all the list of products for a specific tenant I can do

SELECT * FROM "Table-Multitenant" WHERE PK = 'tenant1'

Accessing for reading or writing the single keys will simply be something like:

SELECT * FROM "Table-Multitenant" WHERE PK = 'tenant1' AND SK = 'product1#key1'

profile pictureAWS
EXPERT
answered 2 years ago
  • Thanks, the list of products will need to be deduplicated in code with this approach, as the same product can have many keys, and all of them will be returned by the proposed query.

  • Couldn't this approach suffer from the hot partition issue ? The cardinality on the PK will be very low with only few tenants.

  • Each tenant can provide 1000 WCU's and 3000 RCU's. Does your system intend for a tenant to write at more than 1000 times per second? If so, you can consider write sharding:

    https://medium.com/@leeroy.hannigan/optimizing-dynamodb-queries-using-key-sharding-f3eb4d7f78f7

0

Why would the scan be a problem, if you need all the items anyway? Adding the tenant is a different problem than the 1st one you've proposed.

Unfortunately, you cannot just "leave tenants aside", as this severely affects your setup. You must make the choice of handling that using different tables or a single one now.

Single Table Approach

TenantProductKeyProductKeyValue
tenant1product1#key1product1key1123
tenant1product1#key2product1key2345
tenant1product2#key2product2key2456
tenant2product1#key1product1key1123
tenant2product2#key3product2key3567
tenant2product2#key4product2key4678

Having all the split fields gives you more flexibility to create new GSIs if needed.

  • Select all products for a tenant
    • SELECT Product, Value FROM Table WHERE Tenant = 'tenantX'
  • Select all keys for a product for a tenant
    • SELECT Key, Value FROM Table WHERE Tenant = 'tenantX' AND begins_with(ProductKey, 'productX')
  • Read Item
    • SELECT Value FROM Table WHERE Tenant = 'tenantX' AND ProductKey = 'productX#keyX'
  • All tenants that have a product (if you need it, GSI would be PK(Product) and SK(Tenant))... could do something similar with Keys
    • SELECT Value FROM Table.Product-Tenant-index WHERE Product = 'productX'

Multiple Tables for Tenants Approach

Product(PK)Key(SK)Value
product1key1123
product1key2345
product2key2456
product2key3567
product2key4678
  • Add a new product with some keys and values associated to those keys
    • Trivial, just do it
  • Retrive all the keys and values for a specific product
    • SELECT Key, Value FROM Table WHERE Product = 'ProductX'
  • Retrive the list of all the products (without the keys or values)
    • Scan table
  • Update/Remove/Add a specific key and associated value for a product
    • Trivial, just do it
answered 2 years ago
  • Thanks, the list of products will need to be deduplicated in code with this approach, as the same product can have many keys, and all of them will be returned by the proposed query.

  • On the single table approach, couldn't it suffer from the hot partition issue ? The cardinality on the PK will be very low with only few tenants.

0

Thanks for the quick reply.

In the first case I was thinking I needed to add a GSI to avoid having to deduplicate the productIds in code. But I might be missing something there.

In the second case with the new suggested approach if I run the query:

SELECT * FROM "Table-Multitenant" WHERE PK = 'tenant1'

That one will return all the keys as well, so I will then have to deduplicate them and extract just the product Ids.

answered 2 years ago
  • Why is there different values for each product? If you made the schema a bit more clear it would be easier to advise.

0

Just to clarify at a higher level my use case and what I need.

For each tenant there are different productsID, and for each ProductID one or more keys with an associated value. The keys are basically there to provide some metadata.

So what I need is (leaving alone the multi tenancy for now):

  • Add a new product with some keys and values associated to those keys
  • Retrive all the keys and values for a specific product
  • Retrive the list of all the products (without the keys or values)
  • Update/Remove/Add a specific key and associated value for a product

Example:
Product1 key1:abc key2:cbf key3:123
Product2 key1:hfg
Product3 key9:vbn

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions