AWS timestream performance is slow with large number of values for a dimension filter in query

Question

We have a SaaS product in which we have tenant wise statistics over time period. We tried ingesting them to AWS timestream with partitioning being on "tenant" dimension. In our query, we are supplying a filter on dimension with over 6k values e.g. filterDimension IN (val1, val2, .... val6000). We load tested this query and the response time always seem to be greater than 15 seconds, sometimes even reaching 25 seconds. 
We also tried partitioning on a dimension that contains the date (YYYY-mm-dd) for the stat, but the performance degraded further with this change with response timings crossing the 30 seconds mark.
Since, AWS timestream only allows custom partition over a single dimension only, is there any way to improve performance here?

Anshuman · Answer

In our use-case, the only filter that is constant across all queries is the tenant dimension, but whenever there will be a scenario of filtering on that additional filterDimension column with higher number of matching values, we are encountering higher query latencies. This additional filterDimension cannot be included in the custom partitioning keys since it only allows one dimension currently, which will be tenant for us. So what will be the possible solution for this particular scenario?

Anshuman

answered 3 years ago

Pushvitha · Answer

please refer - https://repost.aws/questions/QU4Baa0YKsQguMOYP-rzbWIw/choosing-timestream-dimension-and-query-performance

AWS timestream performance is slow with large number of values for a dimension filter in query

Add your answer

Relevant content