By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Available SQL Functions in Glue

0

Using the step: "Derived Column" where I can use SQL functions to derive new columns I found SUBSTRING works but CHARINDEX it doesn't. And as usual with AWS documentation, there's no place or indication which functions are available in this particular kind of glue node.

Topics
AnalyticsDatabase
Tags
AnalyticsAWS GlueExtract Transform & Load Data
Language
English
blanchma
asked a year ago334 views
1 Answer
2
Accepted Answer

Are you using DynamicFrames?

You can convert it to DataFrame and then use SparkSQL functions.

https://spark.apache.org/docs/latest/api/sql/index.html

profile pictureAWS
acmanjon
answered a year ago
profile picture
EXPERT
Sedat Salman
reviewed a year ago
  • blanchma
    a year ago

    The goal is to use "Derived Column" and not dabble with the script itself.

  • blanchma
    a year ago

    Actually that list is correct. Thanks, @acmanjon

  • Gonzalo Herreros EXPERT
    a year ago

    Yes, that node allows using SparkSQL functions based on the Glue version you are using (3.3.0 for Glue 4 and 3.1.1 for Glue 3)

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content