Skip to content

OpenSearch _reindex giving null_pointer_exception

0

I need to move an index from an ElasticSearch 6.8 domain to an OpenSearch 2.5 domain. They are both in a VPC.

For the ElasticSearch 6.8 domain in my account, I can see that I have an inbound connection under the Connections tab. For the OpenSearch 2.5, I can also see that I have the outbound connection with the aos-######.amazonaws.com link.

I run the reindex query:

POST /_reindex
{
  "source": {
    "remote": {
      "host": "https://aos-#########.us-west-2.es.amazonaws.com:443"
    },
    "index": "indexName-2023"
  },
  "dest": {
    "index": "indexName-2023"
  }
}

But receive back:

{
  "error": {
    "root_cause": [
      {
        "type": "null_pointer_exception",
        "reason": null
      }
    ],
    "type": "null_pointer_exception",
    "reason": null
  },
  "status": 500
}

I enabled the domain error logging and this is the stack trace:

java.lang.NullPointerException
	at __PATH__(StringBuilder.java:128)
__AMAZON_INTERNAL__
__AMAZON_INTERNAL__
__AMAZON_INTERNAL__
__AMAZON_INTERNAL__
	at org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:133)
	at org.apache.http.impl.nio.client.MainClientExec.prepareRequest(MainClientExec.java:520)
	at org.apache.http.impl.nio.client.MainClientExec.prepare(MainClientExec.java:146)
	at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.start(DefaultClientExchangeHandlerImpl.java:129)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.execute(CloseableHttpAsyncClientBase.java:116)
	at org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:138)
	at org.opensearch.client.RestClient.lambda$performRequestAsync$1(RestClient.java:421)
	at org.opensearch.client.Cancellable.runIfNotCancelled(Cancellable.java:93)
	at org.opensearch.client.RestClient.performRequestAsync(RestClient.java:419)
	at org.opensearch.client.RestClient.performRequestAsync(RestClient.java:406)
	at org.opensearch.index.reindex.remote.RemoteScrollableHitSource.execute(RemoteScrollableHitSource.java:190)
	at org.opensearch.index.reindex.remote.RemoteScrollableHitSource.lookupRemoteVersion(RemoteScrollableHitSource.java:112)
	at org.opensearch.index.reindex.remote.RemoteScrollableHitSource.doStart(RemoteScrollableHitSource.java:101)
	at org.opensearch.index.reindex.ScrollableHitSource.start(ScrollableHitSource.java:99)
	at org.opensearch.index.reindex.AbstractAsyncBulkByScrollAction.start(AbstractAsyncBulkByScrollAction.java:305)
	at org.opensearch.index.reindex.Reindexer.lambda$execute$0(Reindexer.java:156)
	at org.opensearch.index.reindex.BulkByScrollParallelizationHelper.executeSlicedAction(BulkByScrollParallelizationHelper.java:120)
	at org.opensearch.index.reindex.Reindexer.execute(Reindexer.java:136)
	at org.opensearch.index.reindex.TransportReindexAction$1.onResponse(TransportReindexAction.java:104)
	at org.opensearch.index.reindex.TransportReindexAction$1.onResponse(TransportReindexAction.java:101)
	at org.opensearch.index.reindex.BulkByScrollParallelizationHelper.initTaskState(BulkByScrollParallelizationHelper.java:158)
	at org.opensearch.index.reindex.Reindexer.initTask(Reindexer.java:131)
	at org.opensearch.index.reindex.TransportReindexAction.doExecute(TransportReindexAction.java:101)
	at org.opensearch.index.reindex.TransportReindexAction.doExecute(TransportReindexAction.java:58)
	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:219)
	at org.opensearch.indexmanagement.rollup.actionfilter.FieldCapsFilter.apply(FieldCapsFilter.kt:118)
	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:217)
	at org.opensearch.performanceanalyzer.action.PerformanceAnalyzerActionFilter.apply(PerformanceAnalyzerActionFilter.java:78)
	at org.opensearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:217)
	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:189)
	at org.opensearch.action.support.TransportAction.execute(TransportAction.java:108)
	at org.opensearch.client.node.NodeClient.executeLocally(NodeClient.java:110)
	at org.opensearch.index.reindex.AbstractBaseReindexRestHandler.lambda$doPrepareRequest$0(AbstractBaseReindexRestHandler.java:73)
	at org.opensearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:125)
	at org.opensearch.rest.RestController.dispatchRequest(RestController.java:312)
	at org.opensearch.rest.RestController.tryAllHandlers(RestController.java:398)
	at org.opensearch.rest.RestController.dispatchRequest(RestController.java:241)
__AMAZON_INTERNAL__
__AMAZON_INTERNAL__
__AMAZON_INTERNAL__
__AMAZON_INTERNAL__

Any suggestions, or a way to get a more helpful error message would be very much appreciated. I am hoping to avoid using a proxy, since the point of the VPC connection thing is to no longer need a proxy. At the very least, I'd like to find a way to get a meaningful error message from the system.

asked 3 years ago761 views
1 Answer
0

Hi, as source domain is in VPC you either will have to use a proxy or VPC Endpoint connection.

Please see "Reindex data between OpenSearch Service domains when the source is in a VPC" section from the docs. https://docs.aws.amazon.com/opensearch-service/latest/developerguide/remote-reindex.html

Hope it helps!

AWS
answered 3 years ago
  • I am using the VPC connection. Still getting that NPE.
    Have you seen this problem before, or have any ideas what could be causing it?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.