How to perform scroll / scroll not working on Amazon Elasticsearch

Question

We used this guide: https://opensearch.org/docs/latest/opensearch/rest-api/scroll/, and the amazon elasticsearch docs: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/supported-operations.html to try and find out how to perform scroll search on amazon elasticsearch. We recently switched from elasticsearch on one of our local servers to amazon elasticsearch. We came to realize that scroll performs differently. See below a snippet on how we now try to perform scroll (in python3):
params = {"size": 5000, "q": query, "scroll": "5m"}
resp = requests.get(ELASTICSEARCH_URL + "/_search", params=params, auth=auth)
resp.status code:
200
scroll_id = resp.json().get("_scroll_id")
In [23]: resp = requests.get(ELASTICSEARCH_URL + "_search/scroll", params={"scro
    ...: ll": "5m", "scroll_id": scroll_id}, auth=auth)
In [24]: resp
Out[24]: 
In [25]: resp._content
Out[25]: b'{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/users/_search/scroll] contains unrecognized parameters: [scroll], [scroll_id]"}],"type":"illegal_argument_exception","reason":"request [/users/_search/scroll] contains unrecognized parameters: [scroll], [scroll_id]"},"status":400}'

We've tried about all the variations, remove "/"'s or tweaking params, but getting this same error.

Thank you in advance.

Answer

Syntax for scroll request:
GET _search/scroll/

https://opensearch.org/docs/latest/opensearch/rest-api/scroll/

When running the scroll request for the first time, we use the index name. Below is an example:

GET shakespeare/_search?scroll=10m
{
  "size": 10000
}

This will return scroll id:

"_scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAUWdmpUZDhnRFBUcWFtV21nMmFwUGJEQQ=="

From the second time use only  _search/scroll/. we don't pass index name the second time

sample:  GET _search/scroll
{
  "scroll": "10m",
  "scroll_id": "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAAUWdmpUZDhnRFBUcWFtV21nMmFwUGJEQQ=="
}

Note: Once the scroll id expires, scroll request will start failing. Please consider starting a new scroll.

params = {"size": 5000, "q": query, "scroll": "5m"} resp = requests.get(ELASTICSEARCH_URL + "/_search", params=params, auth=auth) resp.status code: 200 scroll_id = resp.json().get("_scroll_id")

Above Query works fine on versions 7.10, 7.9, 7.7 and 7.4. Since python says illegal argument exception, version of ES/OS is required to replicate the issue using the same python client.

How to perform scroll / scroll not working on Amazon Elasticsearch

Relevant content