What are the benefits when I run a Glue job inside VPC?
I am having a Glue job and without VPC, the job work fine. However, I want to ask:
- What is the benefits if I move it to be inside a VPC?
- If I continue use the job outside VPC, will I face security issues such as leak data, etc.?
Thank you so much!
Hi. That's a great question.
If you run a job outside of a VPC, the job potentially has direct access to the internet, and a rouge engineer could write code that would write data to some endpoint on the internet that is outside of your organization. There are various ways to address this risk, but one of them is to ensure the job runs on a VPC where you control all data egress.
The other common reason to use a VPC endpoint with your Glue jobs is to enable access to other resources in your VPC (like RDS servers if you need to ingest data from those), or resources on your corporate network (if you have a connection between your VPC and your corporate network).
See the IAM Policies that Control Settings Using Condition Keys
in the AWS Glue documentation at the following link. This includes an example of how you can use an IAM policy to ensure that only Glue jobs that have a specific VPC connection are able to be created.
https://docs.aws.amazon.com/glue/latest/dg/using-identity-based-policies.html
All the best with your AWS Glue data engineering!
Relevant questions
Call a glue Job from within another without using vpc endppoints or SG
asked 4 months agoAWS Glue ETL Job: IllegalArgumentException: Missing collection name.
asked 5 days agoStep function state to execute a Glue job seems to be stalling
asked a year agoWhat are the benefits when I run a Glue job inside VPC?
Accepted Answerasked a month agoAccess CloudDirectory from inside a VPC
asked 3 years agoAWS Glue Job schedule missing
asked 18 days agocall a stored procedure from within glue job
Accepted Answerasked 4 months agoHow to retrieve job run id from the script itself in python shell glue job
asked a month agoHow do I get the output of an AWS Glue DataBrew job to be a single CSV file?
Accepted Answerasked a year agoHow to pass parameters from an event rule through a glue workflow trigger to a job
asked 4 days ago
Thank you so much for your answer.