Glue OpenSearch connection
I am looking for a way to programmatically create a Glue data connection to OpenSearch. I cannot find find any info about this connection type being supported by the AWS API, SDK, CloudFormation etc ... The docs consider only the creation from the console UI.
Am I missing something?
- 최신
- 최다 투표
- 가장 많은 댓글
The connection itself is generic metadata, what makes it a specific connection is the ConnectionType and ConnectionProperties.
You can list the values of a connection created via the console (e.g. aws glue get-connection --name your_connection) and then replicate the configuration programmatically
For creating connections to data stores that are not explicitly supported through AWS Glue's API or SDK, one common approach is to use a custom connector or to integrate via a more generic JDBC connection if applicable. OpenSearch, being a fork of Elasticsearch, might be accessible through JDBC with the right driver and configuration. This approach would involve:
- Finding a JDBC driver that can connect to OpenSearch.
- Creating a custom connection in Glue that uses this JDBC driver, specifying the connection details to your OpenSearch cluster.
This process typically requires constructing the connection string manually and ensuring that your Glue jobs have the necessary permissions to access the OpenSearch cluster. This might involve setting up VPC endpoints, security groups, and IAM roles appropriately.
If this has answered your question or was helpful, accepting the answer would be greatly appreciated. Thank you!
The JDBC solution may be an option but is out of scope. AWS supports OpenSearch connection from the console UI so it should be supported at the API and SDK level as well.
Both the terraform AWS provider and CloudFormation require a connection type and validate it to match the API constraints. From the docs: