Skip to content

Create mappings for an opensearch index with cdk

1

I have been trying to add OpenSearch Serverless to my CDK (I use ts). But when I try to create a mapping for an index it fails.

Here is the mapping CDK code:

const indexMapping = {
                properties: {
                    account_id: {
                        type: "keyword"
                    },
                    address: {
                        type: "text",
                    },
                    city: {
                        fields: {
                            keyword: {
                                type: "keyword",
                            },
                        },
                        type: "text",
                    },
                    created_at: {
                        format: "strict_date_optional_time||epoch_millis",
                        type: "date",
                    },
                    created_at_timestamp: {
                        type: "long",
                    },
                    cuopon: {
                        type: "text",
                    },
                    customer: {
                        fields: {
                            keyword: {
                                ignore_above: 256,
                                type: "keyword",
                            },
                        },
                        type: "text",
                    },
                    delivery_time_window: {
                        fields: {
                            keyword: {
                                ignore_above: 256,
                                type: "keyword",
                            },
                        },
                        type: "text",
                    },
                    email: {
                        fields: {
                            keyword: {
                                ignore_above: 256,
                                type: "keyword",
                            },
                        },
                        type: "text",
                    },
                    jane_store: {
                        properties: {
                            id: {
                                type: "keyword",
                            },
                            name: {
                                type: "text",
                            },
                        },
                        type: "object",
                    },
                    objectID: {
                        type: "keyword",
                    },
                    order_number: {
                        fields: {
                            keyword: {
                                ignore_above: 256,
                                type: "keyword",
                            },
                        },
                        type: "text",
                    },
                    reservation_start_window: {
                        format: "strict_date_optional_time||epoch_millis",
                        type: "date",
                    },
                    reservation_start_window_timestamp: {
                        type: "long",
                    },
                    status: {
                        type: "keyword",
                    },
                    store_id: {
                        type: "keyword",
                    },
                    total_price: {
                        type: "float",
                    },
                    type: {
                        type: "keyword",
                    },
                },
            };

            this.opensearchIndex = new aoss.CfnIndex(this, "OpenSearchIndex", {
                collectionEndpoint:
                    this.environmentConfig.aoss.CollectionEndpoint,
                indexName: prefix,
                mappings: indexMapping,
            });

And, this is the error I got in codebuild:

[#/Mappings/Properties/store_id/Type: keyword is not a valid enum value,
#/Mappings/Properties/reservation_start_window_timestamp/Type: long is not a valid enum value, 
#/Mappings/Properties/jane_store/Type: object is not a valid enum value,
#/Mappings/Properties/jane_store/Properties/id/Type: keyword is not a valid enum value,
#/Mappings/Properties/total_price/Type: float is not a valid enum value,
#/Mappings/Properties/created_at_timestamp/Type: long is not a valid enum value, #/Mappings/Properties/created_at/Type: date is not a valid enum value,
#/Mappings/Properties/reservation_start_window/Type: date is not a valid enum value,
#/Mappings/Properties/type/Type: keyword is not a valid enum value,
#/Mappings/Properties/account_id/Type: keyword is not a valid enum value,
#/Mappings/Properties/objectID/Type: keyword is not a valid enum value,
#/Mappings/Properties/status/Type: keyword is not a valid enum value]

And the frustrating part is that when I create the exact mapping in the collection Dashboard using the Dev Tool, it works just fine.

Can anyone spot the issue here or show me some working examples of a mapping creation in the CDK?

Thanks in advance.

2 Answers
1

According to the current AWS::OpenSearchServerless::Index PropertyMapping documentation, Type property is limited only to text or knn_vector values. You can create a custom Lambda function that will use one of the OpenSearch clients to create a new index and register that Lambda as a custom resource in your CDK stack. This approach will allow you to use any type definition that is supported by the OpenSearch Create Index API.

AWS

answered a year ago

-2

The issue you're encountering is related to how the CDK expects the mapping structure to be defined for OpenSearch Serverless indexes. The error messages suggest that the CDK is not recognizing the field types you've specified as valid enum values. To resolve this, you need to adjust your mapping structure to match the CDK's expectations for the CfnIndex construct.

Here's how you can modify your code to create a valid mapping:

  1. Instead of using an object literal for the mapping, use the MappingsProperty interface provided by the CDK.

  2. For each field in your mapping, use the PropertyMappingProperty interface to define its properties.

  3. The type property for each field should be a string that matches the OpenSearch field types.

Here's an example of how you can rewrite your mapping:

import * as aoss from 'aws-cdk-lib/aws-opensearchserverless';

const indexMapping: aoss.CfnIndex.MappingsProperty = {
  properties: {
    account_id: {
      type: "keyword"
    },
    address: {
      type: "text"
    },
    city: {
      type: "text",
      properties: {
        keyword: {
          type: "keyword"
        }
      }
    },
    created_at: {
      type: "date"
    },
    created_at_timestamp: {
      type: "long"
    },
    // ... other fields ...
    jane_store: {
      type: "object",
      properties: {
        id: {
          type: "keyword"
        },
        name: {
          type: "text"
        }
      }
    },
    // ... remaining fields ...
  }
};

this.opensearchIndex = new aoss.CfnIndex(this, "OpenSearchIndex", {
  collectionEndpoint: this.environmentConfig.aoss.CollectionEndpoint,
  indexName: prefix,
  mappings: indexMapping
});

In this revised version:

  1. We're using the MappingsProperty structure expected by the CDK.
  2. Each field's type is specified as a string.
  3. For fields with multiple types (like city with both text and keyword), we use nested properties.
  4. The object type fields (like jane_store) are defined with their own properties.

This structure should align with what the CDK expects for OpenSearch Serverless index mappings. Make sure to import the necessary types from the aws-cdk-lib/aws-opensearchserverless module.

Remember that some advanced mapping features might not be directly supported through the CDK and may require using escape hatches or custom resources if needed.
Sources
interface PropertyMappingProperty · AWS CDK
interface MappingsProperty · AWS CDK
interface CfnIndexProps · AWS CDK

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.