Opensearch improve first search

Hi Guys,

this is a bit of a long story...and I'm quite new to opensearch. I am trying to move away from my old solr box which host about 200Gb of indexes data to Opensearch Serverless Our application is a Ruby on Rails so each model has it own index (customers, bookings, jobs etc...), it is also a multi-tenancy which many customers sharing the same database.

The Solr box is a big one with 400Gb of RAM.The search is very fast, usually within 50ms

I've setup a collection in Opensearch Serverless with similar structure and start reindex our data, the indexes size is pretty much the same. However I've tried a search and the first search is always slow, sometime very slow, for an indexes with 20million records, it can take up to 7seconds sometime, subsequent search is faster with speed improving to 2sec, 1sec and then around 500ms (not as fast as the solr box but acceptable for us)

I am not sure what I can do to make the first few search faster so I tried... I partition our 20million record indexes into 100 smaller indexes. Now the search is faster, Generally within 1 second BUT somehow Opensearch Serverless is stuck with a high index OCU and search OCU. initial setup with max 10 OCU for indexing and searching works fine but after the partitioning, it is constantly at 18 OCU for indexing and 23 OCU for searching even though there is completely no load, I've tried remove all the partitioning indexes to see if it goes down but it seems that it stucks in that state. Not sure what has caused it.

I've read somewhere that say the shard size ideally should be 10Gb to 30Gb any my understanding is that every index in Opensearch Serverless is initially on 2 shards. I found that making each. index smaller (approximately 1million records) do improve the search but it has other issues.

is the ideal shard size still applicable or it is different for opensearch serverless. Is there anyway I can boost my first search performance. Since it is a multitenancy structure, the keywords varies and the servers are only busy during business hours. I'm just worried that they will have very bad experience first thing in the morning every day.