Glue job access opensearch serverless failed

I have setup a aws opensearch severless collection. I also enable vpc endpoint and allow access from a IAM role. I created a AWS glue job in the same VPC and use elasticsearch glue connector from marketplace. The glue is to write index to my opensearch. But i always got error saying, `An error occurred while calling o115.pyWriteDynamicFrame. Cannot detect ES version - typically this happens if the network/Elasticsearch cluster is not accessible or when targeting a WAN/Cloud instance without the proper setting 'es.nodes.wan.only'

Gonzalo Herreros ESPERTO
un anno fa
Are you sure your Glue job has connectivity to that serverless cluster?, you could try to open an http connection in the code using Python "requests" to verify
rePost-User-4946701
un anno fa
I have a vpc endpoint setup. I can verify that there is connectivity between vpc subnet and opensearch serverless vpc endpoint.
Gonzalo Herreros ESPERTO
un anno fa
I would check the connectivity from the Glue script, not all types of endpoints work correctly when Glue runs inside a VPC