Can you export Glue DataBrew recipies as code?

Question

Customer likes how easy DataBrew is to work with, but wonders if they could get the resulting Glue Job code so they can tweak it even further.   They are looking for starter code like you get with a standard Glue Job when running something like the convert to Parquet.

AWS-User-9164621 · Accepted Answer

My understanding is that DataBrew is meant to be a no-code alternative to writing your own Spark code. As best I can tell, you can only export the declarative YAML/JSON definition of a DataBrew recipe. I created a recipe and couldn't find any way to export it as a "regular" Glue job definition, and the recipe does not appear in the standard Glue jobs window. I don't think there's any way to see the spark code.

If your customer wants visual design + ability to view/edit code, I think they'll need to look at AWS Glue Studio, instead: https://docs.aws.amazon.com/glue/latest/ug/what-is-glue-studio.html

If your customer wants visual design + ability to view/edit code, I think they'll need to look at AWS Glue Studio, instead: 
https://docs.aws.amazon.com/glue/latest/ug/what-is-glue-studio.html

Deen_P · Answer

DataBrew is not a  code generator. You can export the recipe generated by Data Brew project to be used by other Data Brew Project  or other Data Brew jobs but you cannot mix jobs Glue Studio and DataBrew jobs. But you can build your data pipeline flow which can use DataBrew Jobs and Glue Jobs independently , stitching them together with Step Functions or other orchestration tool.

Gabriel_S · Answer

In addition to what AWS-User-9164621 and Deen_P have aptly shared, in my experience, it is worthwhile to become acquainted with [implementing the recipes into an AWS CloudFormation template](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-databrew-recipe.html#:~:text=%22Steps%22%20%3A%20%5B%20RecipeStep%2C%20...%20%5D%2C), in conjunction with, various Glue DataBrew resource types, such as: [projects](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-databrew-project.html#:~:text=%22RecipeName%22%20%3A%20String%2C), [datasets](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-databrew-dataset.html#cfn-databrew-dataset-input:~:text=%22FormatOptions%22%20%3A%20FormatOptions%2C%0A%20%20%20%20%20%20%22Input%22%20%3A%20Input%2C), [jobs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-databrew-job.html#:~:text=%3A%20String%2C-,%22Recipe%22%20%3A%20Recipe%2C,-%22RoleArn%22), and/or [schedules](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-databrew-schedule.html#:~:text=%22JobNames%22%20%3A%20%5B%20String%2C%20...%20%5D%2C), that you may need for their composition and/or orchestration.  [Available "conditionals" in AWS Glue DataBrew recipe structure](https://docs.aws.amazon.com/databrew/latest/dg/recipes.html#recipes.structure:~:text=DataBrew%20supports%20the%20following%20conditions%3A).

Can you export Glue DataBrew recipies as code?

Contenuto pertinente