Glue to Snowflake connector, is it mandatory?
I’m reading a blog posted last year around moving data from Snowflake to S3, I shared the link with customer who need to move data between Snowflake and S3. https://aws.amazon.com/blogs/big-data/performing-data-transformations-using-snowflake-and-aws-glue/
Now customer comes back saying they need spin up EKS/ECS containers and because the instruction mentioned marketplace and they’ve tried to use the official connector from marketplace called “Snowflake Connector for AWS Glue”.
Because I don’t have Snowflake environment, so I’m not sure if the marketplace snowflake connector is mandatory ? My understanding is it doesn’t, as the blog sounds quite clear to load the JDBC jar to access Snowflake. But it’s still quite confusing for me. If it doesn’t, why we release that connector on marketplace?
If there are 2 approaches to do so (one using custom connector, one using marketplace connector), what's the difference between them? (performance, scalability, support)
- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
Hi,
The “Snowflake Connector for AWS Glue” is delivered as a container and as such the MarketPlace page shows that the supported services are ECS and EKS, BUT there is no need for the customer to spin an ECS/EKS cluster at all.
As describe in the Glue documentation on how to subscribe to a marketplace connector once you have subscribed to the Snowflake Connector you will be asked to configure and then llaunch it. At that point you will be brought back to Glue Studio where you will be asked to create the connection for the connector.
As you notice in this flow there are no steps in which an ECS/EKS cluster will be deployed.
To understand the difference between a Spark and a JDBC connector please refer to this blog post
hope this helps
Relevanter Inhalt
AWS OFFICIALAktualisiert vor 3 Jahren- AWS OFFICIALAktualisiert vor 2 Jahren