AWS opensearch bulk update with script issue

The error you're encountering with the OpenSearch bulk API is a null pointer exception that occurs when the update request's document is null. This typically happens when there's an issue with how the script or document update is structured in your bulk request.

Based on the error message "Cannot invoke 'org.opensearch.action.index.IndexRequest.source()' because the return value of 'org.opensearch.action.update.UpdateRequest.doc()' is null", it appears that your update request is missing the document to update or the script is not properly formatted.

When using scripts in bulk update operations, you need to ensure that:

1. The script is properly formatted and includes all required parameters
2. The script source is correctly specified
3. Any parameters needed by the script are provided

For example, a properly formatted bulk update with script should look something like:

{ "update": { "_index": "your_index", "_id": "document_id" } }
{ "script": { "source": "ctx._source.field = params.value", "params": { "value": "new_value" } } }

If you're using the Python client, make sure your bulk data structure includes both the action metadata and the script details for each document.

Additionally, check if there are any recent changes in the OpenSearch service configuration that might affect script execution. Some settings like rest.action.multi.allow_explicit_index can impact how bulk operations work, especially when specifying index names in the request body.

Since this works in your local OpenSearch instance but not in AWS OpenSearch Service, there might be differences in configuration or security settings between the environments. AWS OpenSearch Service might have stricter script execution policies or different default settings.

I recommend reviewing your bulk update request format, ensuring all required fields are present, and checking if there are any AWS OpenSearch Service specific configurations that might be affecting script execution.
Sources
Opensearch bulk API error: ConnectionError- failed to parse- label empty or too long (Opensearch domain endpoint URL) | AWS re:Post
Identity and Access Management in Amazon OpenSearch Service - Amazon OpenSearch Service