Main issue: Data is not written to OpenSearch

• I have data coming in on IoT Core, I can see the data on the MQTT Test Client
• I have a rule: SELECT * FROM test, My data already have a timestamp formatted correctly
• The rule have permission: "Effect": "Allow","Action": ["osis:*","aoss:*","es:*"],"Resource": "*"
• The OpenSearch have the same permission with "Principal": { "AWS": "*" }, added
• I can POST to the OpenSearch from the OpenSearch DEV TOOLS.
• I can POST using curl from my home:

url -i -u "OpenSearch login username: and password" -H 'Content-Type: application/json' -X PUT  -d \
'{ "Timestamp": "2023-08-16T08:44:47Z", "Location": "HOME"} ' \
https://search-****.us-east-1.es.amazonaws.com/test/_doc

• When my rules run, I get this error in CloudWatch from a IoT Core Rule Error action:

{
    "ruleName": "rule",
    "topic": "test",
    "cloudwatchTraceId": "some generated number",
    "clientId": "test",
    "base64OriginalPayload": "base64 contents=",
    "failures": [
        {
            "failedAction": "OpenSearchAction",
            "failedResource": "https://search-***.us-east-1.es.amazonaws.com",
            "errorMessage": "Failed to index document in OpenSearch.  The error received was Bad Request. Message arrived on: aq, Action: openSearch, Endpoint: https://search-***.us-east-1.es.amazonaws.com, Index: test, type: device, id: 182-generated-number-10"
        }
    ]
}

• And this error from the IoT Core Log (in CloudWatch):

{
    "timestamp": "2023-08-17 15:18:12.452",
    "logLevel": "ERROR",
    "traceId": "910...be7",
    "accountId": "...",
    "status": "Failure",
    "eventType": "RuleExecution",
    "clientId": "test",
    "topicName": "test",
    "ruleName": "rule",
    "ruleAction": "OpenSearchAction",
    "resources": {
        "Endpoint": "https://search-***us-east-1.es.amazonaws.com",
        "Index": "test",
        "Type": "device",
        "DocumentId": "94a...49d"
    },
    "principalId": "ce5...819",
    "details": "Bad Request"
}

Bad Request is a somewhat vague description

• OpenSearch is configured with:
  • Fine-grained access control with a master user
  • No SAML nor Cognito
  • domain level access policy as per above

I don't know if it is an access problem, if so, I imagine it might be the master user thing that should be IAM ARN as master user instead. If so, what IAM ARN? If it is bad format, I have no idea, as I have posted exact same valid JSON as I get from IoT Core. I have even decoded the base64 and stuffed it into OpenSearch using Curl POST without changing a comma ... successfully.

Any material I find on this is dated around 2016 and nothing looks like that anymore. I could probably hack it in using a rule with HTTP POST and username password from OpenSearch, but find that must be the very wrong of doing this.

Oh, btw, I am a newbie, I only played around with AWS for a week.