Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I tried to add through 2 ways
```
newdf = newdf.withColumn('filename2', input_file_name())
```
also tried
```
AmazonS3_node = glueContext.create_dynamic_frame.from_options(
...
1
answers
0
votes
443
views
asked 4 months agolg...
For a DeltaTarget, glue.create_crawler is not recognizing the parameter "CreateNativeDeltaTable"lg...
Unknown parameter in Targets.DeltaTargets[0]: "CreateNativeDeltaTable", must be one of: DeltaTables, ConnectionName, WriteManifest.
However, in the documentation it shows the parameter (see...
1
answers
0
votes
159
views
asked 4 months agolg...
Hi,
I am trying to use Athena to query the s3 file which is in csv format and contains the json objects.
**S3 CSV file:**
```
Id, data, last_update
001, {"key1":"value1"}, 10-01-2024
002,...
1
answers
0
votes
565
views
asked 4 months agolg...
Hi,
could you please help me creating crawler in aws glue. when we create aws crawler with all IAM access then we get error account account id is denied access.
1
answers
0
votes
198
views
asked 4 months agolg...
I am trying to create a glue notebook using an IAM role that has policies 'AWSGlueConsoleFullAccess', 'AWSGlueServiceNotebookRole' and a custom policy as below:
```
{
"Version": "2012-10-17",
...
1
answers
0
votes
183
views
asked 4 months agolg...
Just curious when will AWS Glue support Java 11 or Java 17? AWS Glue 4.0 only supports Java 8 so far and due to this our project can't use the new features of Java 11.
1
answers
0
votes
506
views
asked 4 months agolg...
I am trying to connect to an external s3 bucket as a data source for a glue crawler. The bucket has the necessary arn permissions as i can connect to it via s3 browser as an external bucket. The...
1
answers
0
votes
601
views
asked 4 months agolg...
I have a Delta Lake table saved in s3. I am running the following command:
spark.sql("""
CREATE EXTERNAL TABLE db.my_table
USING DELTA
LOCATION 's3://path/to/delta/table
""")
Everything...
2
answers
0
votes
327
views
asked 4 months agolg...
Hi there,
I am trying to establish ETL processing from our existing AWS Kinesis data stream to snowflake. We are currently using KCL which is extremely difficult to maintain as we do not have the...
1
answers
0
votes
1292
views
asked 4 months agolg...
I've loaded some CSV files from an S3 bucket into tables in a Glue DB. I'm trying to query the tables using Athena, but I keep getting this...
1
answers
0
votes
382
views
asked 4 months agolg...
Hello.
I am currently completing the AWS Industry Quest: Financial Industry but I'm unable to do so as 2 modules produce errors that prevent me from continuing. The modules "Securing a Banking Data...
1
answers
0
votes
440
views
asked 4 months agolg...
This error: `Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException`
I have an organisation account, I have access to the root user, but I'm trying to deploy using a PowerUser. I have...
0
answers
0
votes
123
views
asked 4 months agolg...