Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
I am considering Glue to connect to a third party application's database (Oracle) and bring a data set (in excess of 1M rows) obtained by joining multiple tables at source end. The destination...
1
answers
0
votes
368
views
asked 5 months agolg...
Hi all! I need some guidance on the proper way to connect to an Oracle Free Tier Autonomous Database from a Glue ETL Job. I’ve been using the following code snippet, but I encountered an error -...
0
answers
0
votes
626
views
asked 5 months agolg...
* EMR Version: 6.15.0
* Spark Conf
* "spark.sql.catalog.spark_catalog": "org.apache.iceberg.spark.SparkSessionCatalog"
* "spark.sql.catalog.spark_catalog.catalog-impl":...
1
answers
0
votes
297
views
asked 5 months agolg...
I have multiple Visual ETL configured correctly, but if go back to the previous screen and then try to see the job again, the display editor will lost the configuration and it will highlight some...
0
answers
0
votes
112
views
asked 5 months agolg...
I am trying to read data from redshift schema_a and write its output into another redshfit table in schema_b. Below is the code I am using to read from...
3
answers
0
votes
426
views
asked 5 months agolg...
I am working with .sas7bdat file stored in my s3 bucket
I want to convert the sas7bdat file to csv but in glue visual etl I cannot see an option for sas7bdat file format
Can someone please help me...
1
answers
0
votes
330
views
asked 5 months agolg...
Trying to figure out if it's possible to use AWS Glue crawler to parse the spark stderr logs that are dumped from emr-serverless.
The logs are space delimited. I tried running a crawler against the...
0
answers
0
votes
169
views
asked 5 months agolg...
Hello,
While building a job in AWS Glue (Amazon S3, Change Schema, AWS Glue Data Catalog), I had a surprising cost for data preview session (AWS Glue GlueInteractiveSession) of 91% of the total...
1
answers
0
votes
228
views
asked 5 months agolg...
I encountered the following error, “Parquet column cannot be converted in file, Pyspark Expected string Found: INT32.”
I tried to convert the column to INT32 (Applying withColumn(), but the error...
1
answers
0
votes
981
views
asked 5 months agolg...
Hi All,
I set up a crawler, which is giving me headaches when it comes to the "Include path". My path looks currently something like this:
databaseName/schema/%_qt_%
This works fine, meaning that the...
1
answers
0
votes
178
views
asked 5 months agolg...
I want to use Glue Studio for creating a glue ETL job. This job needs to filter out the data in its first step based on the input parameters given to it at run time. Is there a way with visual ETL...
Accepted AnswerAWS Glue
2
answers
0
votes
511
views
asked 5 months agolg...
I have data currently partitioned on a key (say cluster) and I'm repartitioning to a new key 'date'. So I do (in Python)
```
df = glueContext.create_dynamic_frame.from_options(...)
df =...
1
answers
0
votes
194
views
asked 5 months agolg...