By using AWS re:Post, you agree to the Terms of Use

Glue to Snowflake connector, is it mandatory?

0

I’m reading a blog posted last year around moving data from Snowflake to S3, I shared the link with customer who need to move data between Snowflake and S3. https://aws.amazon.com/blogs/big-data/performing-data-transformations-using-snowflake-and-aws-glue/

Now customer comes back saying they need spin up EKS/ECS containers and because the instruction mentioned marketplace and they’ve tried to use the official connector from marketplace called “Snowflake Connector for AWS Glue”.

Because I don’t have Snowflake environment, so I’m not sure if the marketplace snowflake connector is mandatory ? My understanding is it doesn’t, as the blog sounds quite clear to load the JDBC jar to access Snowflake. But it’s still quite confusing for me. If it doesn’t, why we release that connector on marketplace?

If there are 2 approaches to do so (one using custom connector, one using marketplace connector), what's the difference between them? (performance, scalability, support)

asked 7 months ago67 views
1 Answer
1
Accepted Answer

Hi,

The “Snowflake Connector for AWS Glue” is delivered as a container and as such the MarketPlace page shows that the supported services are ECS and EKS, BUT there is no need for the customer to spin an ECS/EKS cluster at all.

As describe in the Glue documentation on how to subscribe to a marketplace connector once you have subscribed to the Snowflake Connector you will be asked to configure and then llaunch it. At that point you will be brought back to Glue Studio where you will be asked to create the connection for the connector.

As you notice in this flow there are no steps in which an ECS/EKS cluster will be deployed.

To understand the difference between a Spark and a JDBC connector please refer to this blog post

hope this helps

EXPERT
answered 7 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions