- Newest
- Most votes
- Most comments
Based on the information provided, it appears that you're encountering an issue with the Kendra API when trying to create a WEBCRAWLERV2 data source. Let's address your questions:
-
Is this request body valid? The request body you've provided seems to be structured correctly for the most part. However, the ValidationException suggests that there are some required keys missing.
-
What are the four missing required keys? Unfortunately, the error message you're receiving is not providing specific information about which keys are missing. This appears to be a bug in the error reporting, as the key names are not being properly displayed in the error message.
-
Why didn't the key names evaluate in the error message? This seems to be an issue with how the error is being generated or formatted by the AWS Kendra service. It's not a problem with your code, but rather with how the service is reporting the error.
Given that your template configuration passes validation against the JSON schema, the issue might be related to how the AWS SDK for Ruby is processing or sending the request, or how Kendra is interpreting it.
To troubleshoot this issue, you could try the following:
- Double-check that all required fields for the WEBCRAWLERV2 data source type are present in your configuration.
- Ensure that the structure of your request matches exactly what's expected by the API, including any nested objects or arrays.
- Try creating the data source using a different method, such as the AWS CLI or another SDK, to see if the issue persists.
- If possible, start with a minimal configuration and gradually add more fields to identify which part of the configuration is causing the issue.
If the problem continues, it would be advisable to contact AWS support, as this appears to be an issue with how the Kendra service is handling or reporting errors. They may be able to provide more specific guidance or address the bug in their error reporting.
Sources
[Class: Aws::Kendra::Types::ValidationException
— AWS SDK for Ruby V3](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Kendra/Types/ValidationException.html)
[Class: Aws::Kendra::Types::InvalidRequestException
— AWS SDK for Ruby V3](https://docs.aws.amazon.com/sdk-for-ruby/v3/api/Aws/Kendra/Types/InvalidRequestException.html)
There is a json schema on this page showing the required fields: https://docs.aws.amazon.com/kendra/latest/dg/ds-schemas.html#ds-confluence-schema
{
"$schema": "http://json-schema.org/draft-04/schema#",
"type": "object",
"properties": {
"connectionConfiguration": {
"type": "object",
"properties": {
"repositoryEndpointMetadata": {
"type": "object",
"properties": {
"hostUrl": {
"type": "string",
"pattern": "https:.*"
},
"type": {
"type": "string",
"enum": [
"SAAS",
"ON_PREM"
]
},
"authType": {
"type": "string",
"enum": [
"Basic",
"OAuth2",
"Personal-token"
]
}
},
"required": [
"hostUrl",
"type",
"authType"
]
}
},
"required": [
"repositoryEndpointMetadata"
]
},
"repositoryConfigurations": {
"type": "object",
"properties": {
"space": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": [
"STRING",
"STRING_LIST",
"DATE"
]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": [
"fieldMappings"
]
},
"page": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": [
"STRING",
"STRING_LIST",
"DATE",
"LONG"
]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": [
"fieldMappings"
]
},
"blog": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": [
"STRING",
"STRING_LIST",
"DATE",
"LONG"
]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": [
"fieldMappings"
]
},
"comment": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": [
"STRING",
"STRING_LIST",
"DATE",
"LONG"
]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": [
"fieldMappings"
]
},
"attachment": {
"type": "object",
"properties": {
"fieldMappings": {
"type": "array",
"items": [
{
"type": "object",
"properties": {
"indexFieldName": {
"type": "string"
},
"indexFieldType": {
"type": "string",
"enum": [
"STRING",
"STRING_LIST",
"DATE",
"LONG"
]
},
"dataSourceFieldName": {
"type": "string"
},
"dateFieldFormat": {
"type": "string",
"pattern": "yyyy-MM-dd'T'HH:mm:ss'Z'"
}
},
"required": [
"indexFieldName",
"indexFieldType",
"dataSourceFieldName"
]
}
]
}
},
"required": [
"fieldMappings"
]
}
}
},
"additionalProperties": {
"type": "object",
"properties": {
"usersAclS3FilePath": {
"type": "string"
},
"isCrawlAcl": {
"type": "boolean"
},
"fieldForUserId": {
"type": "string"
},
"inclusionSpaceKeyFilter": {
"type": "array",
"items": {
"type": "string"
}
},
"exclusionSpaceKeyFilter": {
"type": "array",
"items": {
"type": "string"
}
},
"pageTitleRegEX": {
"type": "array",
"items": {
"type": "string"
}
},
"blogTitleRegEX": {
"type": "array",
"items": {
"type": "string"
}
},
"commentTitleRegEX": {
"type": "array",
"items": {
"type": "string"
}
},
"attachmentTitleRegEX": {
"type": "array",
"items": {
"type": "string"
}
},
"isCrawlPersonalSpace": {
"type": "boolean"
},
"isCrawlArchivedSpace": {
"type": "boolean"
},
"isCrawlArchivedPage": {
"type": "boolean"
},
"isCrawlPage": {
"type": "boolean"
},
"isCrawlBlog": {
"type": "boolean"
},
"isCrawlPageComment": {
"type": "boolean"
},
"isCrawlPageAttachment": {
"type": "boolean"
},
"isCrawlBlogComment": {
"type": "boolean"
},
"isCrawlBlogAttachment": {
"type": "boolean"
},
"maxFileSizeInMegaBytes": {
"type":"string"
},
"inclusionFileTypePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"exclusionFileTypePatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"inclusionUrlPatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"exclusionUrlPatterns": {
"type": "array",
"items": {
"type": "string"
}
},
"proxyHost": {
"type": "string"
},
"proxyPort": {
"type": "string"
}
},
"required": []
},
"type": {
"type": "string",
"pattern": "CONFLUENCEV2"
},
"enableIdentityCrawler": {
"type": "boolean"
},
"syncMode": {
"type": "string",
"enum": [
"FULL_CRAWL",
"FORCED_FULL_CRAWL"
]
},
"secretArn": {
"type": "string",
"minLength": 20,
"maxLength": 2048
}
},
"version": {
"type": "string",
"anyOf": [
{
"pattern": "1.0.0"
}
]
},
"required": [
"connectionConfiguration",
"repositoryConfigurations",
"syncMode",
"additionalProperties",
"secretArn",
"type"
]
}
answered a year ago
