How do I implement semantic search with AWS OpenSearch service?

Question

I was trying to follow this tutorial from the OpenSearch documentation: https://opensearch.org/docs/latest/ml-commons-plugin/semantic-search/ 
but I am unable to modify the cluster settings with: 
`PUT _cluster/settings
{
  "persistent": {
    "plugins": {
      "ml_commons": {
        "only_run_on_ml_node": "false",
        "model_access_control_enabled": "true",
        "native_memory_threshold": "99"
      }
    }
  }
}`

Instead, I encounter the error: `_cluster/settings payload not allowed`
I have tried advice from this stack overflow post: https://stackoverflow.com/questions/76274221/does-aws-opensearch-support-ml-commons-library , particularly
- upgrading my instance to `m5.large.search` 
- enabling `FGAC` and mapping my main user (internal user database) to `ml_full_access` 
but to no success.

Accepted Answer

I found the solution to the 'not allowed' error: you must format the request to the configurations API like so:

`PUT /_cluster/settings
{
  "persistent":{
    "plugins.ml_commons.only_run_on_ml_node": false,
    "plugins.ml_commons.model_access_control_enabled": "true",
    "plugins.ml_commons.native_memory_threshold": "99"
  }
}`

I had to dig into this lab to find this solution: https://catalog.workshops.aws/semantic-search/en-US/module-6-neural-search/lab1-local-mode/opensearch-dashboard-dev-tools . This must be the 'API differences' described in this page: https://docs.aws.amazon.com/opensearch-service/latest/developerguide/supported-operations.html#version_api_notes but I feel that it's difficult to determine how to follow this convention. The error message being 'payload not allowed' does not help either. It would great if the documentation was updated to be clearer about this!

Answer

Dear customer,
 
I understood that there was a problem while updating cluster setting.
If my understand is incorrect, please let me know that.

Below is the result of internally testing to reproduce your problem.

=============================================================
>> PUT /_cluster/settings
{
  "persistent": {
    "plugins": {
      "ml_commons": {
        "only_run_on_ml_node": "false",
        "model_access_control_enabled": "true",
        "native_memory_threshold": "99"
      }
    }
  }
}

>> Output:
{
  "Message": "Your request: '/_cluster/settings' payload is not allowed."
}
=============================================================
Generally, the "ml_commons" plugin is supported by the AWS Opensearch service.

[+] Plugins by engine version in Amazon OpenSearch Service

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/supported-plugins.html

However, AWS Opensearch is a managed service, and on the user side, only certain API tasks are supported by AWS Opensearch.
Information on specific API operations in the supported Opensearch can be found through the link below.

[+] Supported operations in Amazon OpenSearch Service

https://docs.aws.amazon.com/opensearch-service/latest/developerguide/supported-operations.html

Therefore, you need to request the internal AWS Opensearch team to change that information.
Please refer to the link below and create a technical support case and request it.

[+] Creating a support case

https://docs.aws.amazon.com/awssupport/latest/user/case-management.html#creating-a-support-case

I hope this information has been helpful. 
If there is anything I have missed or if there is anything else we can assist, 
please feel free to let us know and we'll be more than happy to help.

How do I implement semantic search with AWS OpenSearch service?

Relevant content