IoT Core OpenSearch Action Rule & "Mapper parsing exception"

0

Hi

I was following this AWS tutorial but with my own sensor data. I've created the following IoT Core Rule OpenSearch Action:

OpenSearchTopicRule:
    Type: AWS::IoT::TopicRule
    Properties:
      TopicRulePayload:
        Actions:
          - OpenSearch:
              Endpoint: !Join ['', ['https://', !GetAtt OpenSearchServiceDomain.DomainEndpoint]]
              Id: '${newuuid()}'
              Index: sensors
              RoleArn: !GetAtt IoTOSActionRole.Arn
              Type: sensor_data
        Sql: SELECT *, timestamp() as ts FROM 'Greenhouse/+/Sensor/Status'

The IoTOSActionRole has propper es:ESHttpPut permission. But when I try to create an index with following command send from Postman that would match the Type: sensor_data attribute:

curl --location --request PUT 'https://search-iot***-avt***i.eu-west-1.es.amazonaws.com/sensors' \
--header 'Content-Type: application/json' \
--data-raw '{
  "mappings": {
    "sensor_data": {
      "properties": {
        "ts": { "type": "long",
          "copy_to": "datetime"},
        "datetime": {"type": "date",
          "store": true},
        "deviceID": {"type": "text",
          "store": true},
        "humidity": {"type": "integer",
          "store": true},
        "temperature": {"type": "integer",
          "store": true},
        "lux": {"type": "integer",
          "store": true},
        "soil": {"type": "integer",
          "store": true}
}}}'

I receive an error:

{
    "error": {
        "root_cause": [
            {
                "type": "mapper_parsing_exception",
                "reason": "Root mapping definition has unsupported parameters:  [sensor_data : {properties={datetime={store=true, type=date}, temperature={store=true, type=integer}, humidity={store=true, type=integer}, soil={store=true, type=integer}, deviceID={store=true, type=text}, lux={store=true, type=integer}, ts={copy_to=datetime, type=long}}}]"
            }
        ],
        "type": "mapper_parsing_exception",
        "reason": "Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters:  [...]",
        "caused_by": {
            "type": "mapper_parsing_exception",
            "reason": "Root mapping definition has unsupported parameters:  [...}]"
        }
    },
    "status": 400
}

I've tried removing the 'type' "sensor_data" attribute and that allowed me to create an index with that mapping,

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "sensors"
}

and then index pattern in OpenSearch Dashboard, but what happens then is that the IoT Core Rule even though it gets triggered does not result in any data ingestion to the OpenSearch domain. So I guess IoT Core Action tries to send that data with type sensor_databut there's no corresponding type in OS. Additionally, when I open the Discovery tab in OS dashboard I get this notice:

"undefined" is not a configured index pattern ID

Showing the default index pattern: "sensors*" (d97775d0-cc44-11ec-acd4-bf72aa2fd725)

I'm using the OpenSearch 1.2 (latest) version. Sample data:

{
  "deviceID": "Tomatoes",
  "Greenhouse": 1,
  "date": "05-05",
  "time": "09:35:39",
  "timestamp": 1651743339,
  "humidity": 60,
  "temperature": 33.3,
  "lux": 9133.333,
  "soil": 78
}

What PUT call I have to make to create a sensor_data type mapping in OS that would match the type specified in IoT Core OpenSearch Action Rule?

Lez
質問済み 2年前284ビュー
1回答
0

Hi Lez. It seems mapping types were dropped in ElasticSearch 7.x, a couple of years after that blog.

https://www.elastic.co/guide/en/elasticsearch/reference/7.0/removal-of-types.html

I think types have never existed in OpenSearch. I think the answer is here: https://opensearch.org/docs/latest/opensearch/rest-api/document-apis/index-document/#response-body-fields

The document’s type. OpenSearch supports only one type, which is _doc.

I wanted to try this out before answering, but ran out of time today. I hope this helps.

profile pictureAWS
エキスパート
Greg_B
回答済み 2年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ