I start got an error after upgrade the opensearch managed cluster from 2.19 to 3.1 for range query

Question

I have a a long query with many type of queries like, exists, terms, term, range, etc. After upgrade I start getting following error:

```
{
  "took": 10,
  "timed_out": false,
  "_shards": {
    "total": 12,
    "successful": 3,
    "skipped": 0,
    "failed": 9,
    "failures": [
      {
        "shard": 0,
        "index": "documents",
        "node": "Asd1B8AkR7OABz4HEqqvFw",
        "reason": {
          "type": "null_pointer_exception",
          "reason": null
        }
      }
    ]
  },
  "hits": {
    "total": {
      "value": 0,
      "relation": "eq"
    },
    "max_score": null,
    "hits": []
  }
}
```
To debug this query result, I just remove the following range filter from my query and it just worked without any problem.

{
              "range": {
                "expiry_at": {
                  "gte": "now"
                }
              }
            },

I just reindex the data into new index to check if it could be related to shard move and upgrade. But it just ended with same problem.

re:Post Agent · Answer

The issue you're experiencing with a null pointer exception after upgrading your OpenSearch managed cluster from 2.19 to 3.1 appears to be related to your range query on the "expiry_at" field.

This type of error can occur during version upgrades when there are compatibility issues with certain query types or when the underlying data structure has changed. The fact that removing the range filter with the "now" value resolves the issue suggests that there might be a specific problem with how the new version handles date range queries using the "now" keyword.

Here are some troubleshooting steps you can try:

1. Check if the "expiry_at" field is properly mapped as a date field in your index.

2. Try using an explicit timestamp instead of "now" in your range query to see if that works.

3. If possible, examine the logs on your OpenSearch cluster for more detailed error information.

4. Consider creating a simpler test query with just the range filter to isolate the issue.

5. If you have multiple shards, the error shows that some shards are failing while others are successful. This could indicate inconsistencies in your data or index configuration across shards.

Since reindexing didn't solve the problem, it suggests this might be a compatibility issue with how date range queries are processed in OpenSearch 3.1 compared to 2.19, particularly when using the "now" keyword.  
**Sources**  
[Making configuration changes in Amazon OpenSearch Service - Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/managedomains-configuration-changes.html)  
[Searching data in Amazon OpenSearch Service - Amazon OpenSearch Service](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/searching.html)

I start got an error after upgrade the opensearch managed cluster from 2.19 to 3.1 for range query

Add your answer

Relevant content