Unanswered Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Facing the following error when using 2 jobs and 1 crawler:
Job 1: changing the schema and saving the csv file as parquet in S3.
Job 2: ETL Process
Crawler: Saving it in the AWs Glue Datacatalog...
0
answers
0
votes
23
views
asked 4 hours agolg...
I'm creating a role in AWS Glue to read CSV files from an S3 bucket. I'm granting full access to S3, but I can't seem to avoid this error. I contacted support, and they suggested increasing the usage...
0
answers
0
votes
30
views
asked 4 days agolg...
This was working before, as recently as a week or two ago but Athena now fails with "INVALID_PARAMETER_USAGE: Incorrect number of parameters: expected 207 but found 0." when the query has more than...
0
answers
0
votes
36
views
asked 5 days agolg...
AWS Glue Job Errorlg...
Im trying to convert CSV files in S3 to Parquet in another S3 bucket. So first I read the CSV files using a crawler, load the data into a Table, and then use a Job to convert from the Table to S3 in...
0
answers
0
votes
49
views
asked 6 days agolg...
Hello,
We set up AWS DMS, where the source is MS SQL Server 2019, and the target is S3 (with parquet). Setting up CDC copying. And it is important for us to check that DDLs on source work as well:
1)...
0
answers
0
votes
215
views
asked 11 days agolg...
I've been trying to test out Iceberg tables with Amazon Redshift Spectrum and have come across a major issue.
Here is my setup:
1. I create an iceberg table via spark (emr 7.0) and insert data across...
0
answers
1
votes
204
views
asked 13 days agolg...
I got the error when write glue dynamic data frame to redshift table .
The code I have used :-
my_conn_op={
"dbtable":"public.xxxx",
...
0
answers
0
votes
155
views
asked 14 days agolg...
Hello.
I am trying to configure specific iam permission for an user. I need a permission for only read tables from existing Data Catalog.
So, I have configured this policiy:
```
{
"Version":...
0
answers
0
votes
185
views
asked 15 days agolg...
I've successfully set up AWS Glue with an RDS database serving as the data source and a Snowflake database as the data target. In this setup, I've configured AWS Glue crawlers to catalog the metadata...
0
answers
0
votes
177
views
asked 15 days agolg...
Hello,
I work as a data engineer and business intelligence specialist for a fintech startup. We've entered into a new agreement with a supplier to provide a technological solution for managing their...
0
answers
0
votes
219
views
asked 19 days agolg...
I am using aws lakeformation workflow to create a data lake following [this](https://docs.aws.amazon.com/lake-formation/latest/dg/getting-started-tutorial-jdbc.html) guide. everything is setup as...
0
answers
0
votes
96
views
asked 21 days agolg...
In our ETL process we are building out a pipeline where someones job is to take input files (ex. csv) and map the columns to existing column names. After the mapping is complete a glue workflow will...
0
answers
0
votes
177
views
asked 22 days agolg...