How do I use OIDC-based authentication with Snowflake in an AWS Glue job using Workload Identity Federation?

Short description

You want to connect to Snowflake from an AWS Glue job using OIDC-based Workload Identity Federation (WIF) instead of static credentials. This article explains how to use the Snowflake Python connector library with WIF authentication in a Glue job, including how to resolve the common "No AWS region was found" error.

Resolution

You can use the snowflake-connector-python library in a Glue job to authenticate to Snowflake using OIDC/Workload Identity Federation with IAM roles.

Prerequisites

• An IAM role with a trust policy allowing glue.amazonaws.com to assume it
• A Snowflake user configured for Workload Identity Federation with the IAM role ARN (see Snowflake WIF documentation)
• Glue version 4.0 or above (earlier versions use Python 3.9 which is incompatible with the required Snowflake driver version)
• The snowflake-connector-python library specified as an additional Python module in your Glue job

Resolving the "No AWS region was found" error

In the Glue execution environment, boto3's automatic region detection may not work as expected due to the Glue proxy layer. There are two approaches to resolve this:

Option A: Use a Glue Network Connection (recommended)

Create a Glue Network Connection attached to a VPC. This implicitly provides the AWS region context to the Snowflake Python driver, eliminating the region detection issue entirely.

aws glue create-connection --connection-input '{
  "Name": "sf-vpc-conn",
  "Description": "Connection to VPC for Snowflake WIF",
  "ConnectionType": "NETWORK",
  "PhysicalConnectionRequirements": {
    "SubnetId": "subnet-example",
    "SecurityGroupIdList": ["sg-example"],
    "AvailabilityZone": "us-west-2a"
  }
}' --region us-west-2

Then reference the connection in your Glue job configuration:

--connections Connections=sf-vpc-conn

VPC requirements:

• NAT Gateway (for outbound internet to Snowflake)
• S3 VPC Endpoint (for script/temp access)
• Security group allowing HTTPS (port 443) outbound and a self-referencing rule for all TCP traffic

Option B: Set region_name explicitly in boto3

If you are not using a Glue Network Connection, explicitly set region_name in any boto3 session or client calls within the Glue job:

import boto3
session = boto3.Session(region_name='us-west-2')  # Set your region explicitly

Without this, you may encounter a "No AWS region was found" error even though the same code works in EC2 or Lambda environments.

Example: Connecting to Snowflake with WIF in a Glue job

import snowflake.connector
import sys

print("Python:", sys.version)
print("snowflake-connector-python version:", snowflake.connector.__version__)

conn = snowflake.connector.connect(
    account='YOUR_ORG-YOUR_ACCOUNT',
    authenticator='WORKLOAD_IDENTITY',
    workload_identity_provider='AWS',
    role='YOUR_ROLE',
    warehouse='YOUR_WAREHOUSE'
)

cur = conn.cursor()
cur.execute("SELECT CURRENT_USER(), CURRENT_ROLE(), CURRENT_REGION()")
print("Session context:", cur.fetchone())

cur.execute("SELECT * FROM your_database.your_schema.your_table LIMIT 10")
rows = cur.fetchall()
for r in rows:
    print(r)

cur.close()
conn.close()

Creating the Glue job (CLI required)

The Glue job must be created via CLI. The console defaults to Python 3.9 which is incompatible with the required Snowflake driver version.

aws glue create-job \
  --name glue-sf-wif-job \
  --role arn:aws:iam::ACCOUNT_ID:role/YOUR_GLUE_ROLE \
  --command '{"Name":"glueetl","ScriptLocation":"s3://YOUR_BUCKET/scripts/your-script.py","PythonVersion":"3"}' \
  --glue-version 4.0 \
  --connections '{"Connections":["sf-vpc-conn"]}' \
  --worker-type G.1X \
  --number-of-workers 3 \
  --default-arguments '{"--additional-python-modules":"snowflake-connector-python==4.0.0","--TempDir":"s3://YOUR_BUCKET/temp"}' \
  --region us-west-2

Snowflake user setup

On the Snowflake side, create a service user with Workload Identity tied to your IAM role:

CREATE USER your_wif_user
  WORKLOAD_IDENTITY TYPE AWS
  ARN 'arn:aws:iam::ACCOUNT_ID:role/YOUR_GLUE_ROLE'
  TYPE SERVICE
  DEFAULT_ROLE your_role
  DEFAULT_WAREHOUSE your_warehouse;

Key considerations

• The snowflake-connector-python library is a public, third-party library not managed by AWS. AWS does not provide support for the library itself.
• AWS Glue supports running custom Python libraries in jobs. Using third-party libraries with additional Python modules is a supported execution pattern.
• The region_name requirement is specific to the Glue execution environment when not using a Network Connection, and does not affect other AWS compute services.
• The Glue Network Connection approach (Option A) is recommended as it avoids the region issue entirely and provides proper VPC networking for Snowflake connectivity.

Related information

• AWS Glue Snowflake connection authentication options
• Snowflake Workload Identity Federation
• Using Python libraries with AWS Glue
• AWS Glue connection types