Glue OpenSearch connection
I am looking for a way to programmatically create a Glue data connection to OpenSearch. I cannot find find any info about this connection type being supported by the AWS API, SDK, CloudFormation etc ... The docs consider only the creation from the console UI.
Am I missing something?
- 最新
- 投票最多
- 评论最多
The connection itself is generic metadata, what makes it a specific connection is the ConnectionType and ConnectionProperties.
You can list the values of a connection created via the console (e.g. aws glue get-connection --name your_connection) and then replicate the configuration programmatically
For creating connections to data stores that are not explicitly supported through AWS Glue's API or SDK, one common approach is to use a custom connector or to integrate via a more generic JDBC connection if applicable. OpenSearch, being a fork of Elasticsearch, might be accessible through JDBC with the right driver and configuration. This approach would involve:
- Finding a JDBC driver that can connect to OpenSearch.
- Creating a custom connection in Glue that uses this JDBC driver, specifying the connection details to your OpenSearch cluster.
This process typically requires constructing the connection string manually and ensuring that your Glue jobs have the necessary permissions to access the OpenSearch cluster. This might involve setting up VPC endpoints, security groups, and IAM roles appropriately.
If this has answered your question or was helpful, accepting the answer would be greatly appreciated. Thank you!
The JDBC solution may be an option but is out of scope. AWS supports OpenSearch connection from the console UI so it should be supported at the API and SDK level as well.
相关内容
AWS 官方已更新 3 年前- AWS 官方已更新 3 年前
Both the terraform AWS provider and CloudFormation require a connection type and validate it to match the API constraints. From the docs: