- Newest
- Most votes
- Most comments
From your question, it is not really clear where the data is stored and what tools you are using to access the data. Depending on where the data is available (Redshift vs RDS vs S3 vs Kinesis etc) and depending on the tool used to query the data, the answer would change.
I will list a couple of ways you can accomplish this:
- In Athena when the data is cataloged in Glue tables (Eg. the data is in S3): you could execute this query -
select cast(date_parse('20220915','%Y%m%d') as date);
- In Glue when you are accessing the data as a Glue dynamicFrame or Pyspark dataframe:
from pyspark.sql.functions import *
#df is the dataframe that you add logic here to read/transform or convert from Glue Dynamicframe
df.select(col("input"),to_date(col("input"),"yyyyMMdd").alias("date_column")).show()
- In Redshit:
select to_date('20220915', 'YYYYMMDD', FALSE);
Reference: https://docs.aws.amazon.com/redshift/latest/dg/r_TO_DATE_function.html
Hi there,
Thanks so much for your help. So I'm using AWS to query the database and I have tried option 1 with no success. Error message reads as:
INVALID_FUNCTION_ARGUMENT: Invalid format: ""
The field I am trying to perform this operation on is a result of a substring function which extracts the last 8 characters of another field that contains the date as part of a longer unusable string for date purposes. The result of the substring is in 'YYYMMDD' format.
Thanks again
@JD is the format "YYYYMMDD" or "YYYMMDD", I think you have a typo in the comment. aside from that could you provide your athena query? and some info on the table format (DDL)?
Relevant content
- asked 10 months ago
- Accepted Answerasked 2 years ago
- asked 2 years ago
- AWS OFFICIALUpdated 3 years ago
- AWS OFFICIALUpdated 2 years ago
Hi which service do you plan to use to work with the data?