Zero-ETL OpenSearch Serverless Collection Endpoint hosts

Question

I'm trying to set up a pipeline for the Zero-ETL Ingestion to connect Dynamo Db to my OpenSearch Serverless Collection.

When inserting my OpenSearch endpoint into the `host` it is flagging as invalid. I have copied over my endpoint from the OpenSearch console.

Here is how my endpoint looks:
`https://l01wuooe8ah8z1fk0021.eu-west-1.aoss.amazonaws.com`

This is the error that it's presenting itself with:

`
The "opensearch.hosts" https://search-mydomain-l36wuros9dh8z6fk9616.aos.eu-west-1.on.aws does not conform to the standard regex pattern ^https:\/\/(vpc-|search-)(?[0-9a-z\-]{3,28})\-[a-z0-9]{26}\.(?[a-z0-9\-]{4,20})\.es\.amazonaws\.com$ or ^https:\/\/(vpc-|search-)(?[0-9a-z\-]{3,28})\-[a-z0-9]{26}\.aos\.(?[a-z0-9\-]{4,20})\.on\.aws$ for an Amazon OpenSearch domain endpoint. Custom domain endpoints are not compatible with Amazon OpenSearch Ingestion at this time.
`

Any help would be greatly appreciated!

Answer

We had the same issue previously, because our hosts url was wrong and we did not set "serverless: true".

Solution: Based on the doc, we need to use correct collection id and set  "serverless: true:" (shown in below).

```
 sink:
    - opensearch:
        hosts: [ "https://{collection-id}.us-east-1.aoss.amazonaws.com" ] 
        index: "my_logs"
        aws:
          sts_role_arn: "arn:aws:iam::{your-account-id}:role/PipelineRole"
          region: "us-east-1"
          serverless: true   # set it to true
```

Reference :  https://docs.aws.amazon.com/opensearch-service/latest/developerguide/osis-serverless-get-started.html#osis-serverless-get-started-pipeline

Answer

Where does the “search-mydomain-” string come from?

Zero-ETL OpenSearch Serverless Collection Endpoint hosts

Relevant content