OpenSearch Serverless Cannot find document

Question

I have OpenSearch Serverless set up, with logs being emitted from ECS Fargate > Firelens / Fluent Bit > OpenSearch.  The collection is configured with collection type = Timeseries.

All the data is being streamed and appears in OpenSearch correctly. However, if you try to click on a log and "View single document" or "View surrounding documents", the UI will fail with "Cannot find document.

If I inspect the "_id" field, I see values like "1%3A0%3AOmGwvIsB-h7GakQ2ceEs". They all have the prefix "1%3A0%3A". If I try to query by that exact _id field in other places in the UI (like Filter by type or using the Console tool to query), nothing is returned. If I query by the URL decoded version "1:0:OmGwvIsB-h7GakQ2ceEs", nothing is returned. However, if I query by the suffix "OmGwvIsB-h7GakQ2ceEs", a value is returned.

I believe the _id is being generated by OpenSearch itself so I'm at a loss as to what is going on behind the scenes. If I try to set "Generate_ID" in my fluent config, I get an error since it seems like time-series collections do not support setting the ID.

Answer

Hello,

As you correctly mentioned that the "_id" is being generated by OpenSearch serverless itself for Timeseries collection. Thus, currently OpenSearch Serverless doesn't support query by id.

The Opensearch team is working on to address this issue and fix the "View Document" option in the dashboards which is invoking the same query by id API. However, if you have a need to search by document ids, we recommend to use get _doc by id, _mget APIs for getting desired results. We apologize for the inconvenience but will try to fix the issue as soon as possible.

If you need any help please do reachout to the AWS Support they will be able to assist with you with your query here.

OpenSearch Serverless Cannot find document

Relevant content