[Opensearch] current ES version doesn't allow Collapse with search_after

Question

Hi Experts :)
I wonder there is a workaround for using search_after with Collapse which isn't served on current AWS Opensearch Version 7.10
https://www.elastic.co/guide/en/elasticsearch/reference/current/collapse-search-results.html

We tried to use an aggregation command with top hit using writer as a key, however, the presented writers bucket had low scores. 
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations-metrics-top-hits-aggregation.html

Simply, we wanted to show the top 10 documents without duplication on the writer.

How can we find a workaround for the current Opensearch Version?

Answer

So the search_after parameter involves the point in time (PIT) API, which is an X-Pack/ElasticSearch exclusive feature due to licensing issues.

I can recommend you try one or both of the following workarounds. First, you can try querying the index using the search call, without the "PIT" part of the query. For instance, like this sample query.

-> GET sample/_search
{
  "size": 2,
  "query": {
    "match_all": {}
  },
  "sort": [ 
    {"timestamp": "asc"},
    {"FlightNum":"asc"}
  ]
}

Second, you may also find the scroll API useful. This will help you to retrieve large sets of results from a single scrolling search request.

https://www.elastic.co/guide/en/elasticsearch/reference/7.10/paginate-search-results.html#scroll-search-results
https://www.elastic.co/guide/en/elasticsearch/reference/7.10/scroll-api.html

[Opensearch] current ES version doesn't allow Collapse with search_after

Relevant content