Skip to content

How to use date/float values in OpenSearchServerless PropertyMapping CDK

0

I see that the AWS::OpenSearchServerless::Index PropertyMapping only accepts text | knn_vector:

Type
    The field data type. Must be a valid OpenSearch field type.
    Required: Yes
    Type: String
    Allowed values: text | knn_vector

But then how do we represent things liked dates and numbers?

This is an issue that leads to cdk deploy errors like this one:

[#/Mappings/Properties/store_id/Type: keyword is not a valid enum value,
#/Mappings/Properties/reservation_start_window_timestamp/Type: long is not a valid enum value, 
#/Mappings/Properties/jane_store/Type: object is not a valid enum value,
#/Mappings/Properties/jane_store/Properties/id/Type: keyword is not a valid enum value,
#/Mappings/Properties/total_price/Type: float is not a valid enum value,
#/Mappings/Properties/created_at_timestamp/Type: long is not a valid enum value, #/Mappings/Properties/created_at/Type: date is not a valid enum value,
#/Mappings/Properties/reservation_start_window/Type: date is not a valid enum value,
#/Mappings/Properties/type/Type: keyword is not a valid enum value,
#/Mappings/Properties/account_id/Type: keyword is not a valid enum value,
#/Mappings/Properties/objectID/Type: keyword is not a valid enum value,
#/Mappings/Properties/status/Type: keyword is not a valid enum value]
1 Answer
0

When working with OpenSearch Serverless Index PropertyMapping in CDK, you're correct that the documentation indicates only "text" and "knn_vector" as allowed values for the Type property. However, this limitation doesn't mean you can't work with dates, numbers, and other data types in OpenSearch Serverless.

For date fields in OpenSearch, you have two options:

  1. Use the "text" type and then define a custom mapping template to properly interpret the field as a date. For example, you can map a date field to an OpenSearch date by using a custom mapping template like this:
{
"template": {
"mappings": {
"properties": {
"myDateField": {
"type": "date",
"format": "epoch_second"
}
}
}
}
}
  1. By default, date and timestamp values map to integers in OpenSearch. You can use this default mapping and then query accordingly.

For numeric values like floats or decimals, you can similarly use custom mapping templates:

{
"template": {
"mappings": {
"properties": {
"myDecimalField": {
"type": "double"
}
}
}
}
}

For string values that should be treated as keywords (exact match strings), you can define a custom mapping:

{
"template": {
"mappings": {
"properties": {
"PartType": {
"type": "keyword"
}
}
}
}
}

When working with CDK, you'll need to provide these mappings as part of your template configuration. The error you're seeing occurs because you're trying to directly use types like "keyword", "long", "float", and "date" which aren't in the allowed values list for the Type property. Instead, use "text" as the base type and provide the appropriate mapping template to specify how OpenSearch should interpret and index these fields.

Remember that OpenSearch Service dynamically maps data types in each incoming document to the corresponding data type in your data source, but you need to provide the proper mapping templates to ensure the fields are indexed correctly for searching and aggregation.
Sources
Using an OpenSearch Ingestion pipeline with Amazon DocumentDB - Amazon OpenSearch Service
Using an OpenSearch Ingestion pipeline with Amazon DynamoDB - Amazon OpenSearch Service

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.