Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hello,
We set up AWS DMS, where the source is MS SQL Server 2019, and the target is S3 (with parquet). Setting up CDC copying. And it is important for us to check that DDLs on source work as well:
1)...
0
answers
0
votes
215
views
asked 11 days agolg...
I am getting json files to my s3. For example:
```
{
"name" : "John",
"lastname": "Doe",
"meta" : {
"x": "a",
"y": "b",
"unwanted_field": {
"some":...
1
answers
0
votes
77
views
asked 11 days agolg...
say i have couple of json files in s3, I would to set up a crawler or a glue job, such that i can create table in aws rds (mysql or postgre) , such that in table 1, it creates a autogenerated id and...
1
answers
0
votes
656
views
asked 12 days agolg...
How to build AWS Glue ETL Jobs or Data Quality Jobs, if access to console is not allowed as per company policy. Does not having AWS Console access defeats the purpose AWS Glue? What features cannot be...
2
answers
0
votes
124
views
asked 13 days agolg...
I've been trying to test out Iceberg tables with Amazon Redshift Spectrum and have come across a major issue.
Here is my setup:
1. I create an iceberg table via spark (emr 7.0) and insert data across...
0
answers
1
votes
205
views
asked 13 days agolg...
Hi,
I want to read a tar file from s3, uncompress it and load it to another s3 bucket using Glue job. But I am facing "fileobj must implement read".
obj=s3.getObject(bucketname,key)
objbuffer =...
1
answers
0
votes
133
views
asked 13 days agolg...
In AWS Glue jobs, within the Targets node, I am unable to see the data types such as struct, array or map while changing the schema. Does AWS Glue not support these data types?
1
answers
0
votes
159
views
asked 14 days agolg...
I got the error when write glue dynamic data frame to redshift table .
The code I have used :-
my_conn_op={
"dbtable":"public.xxxx",
...
0
answers
0
votes
155
views
asked 14 days agolg...
Hi,
I've created Glue Crawler to determine data sctructure from XML file uploaded to s3, and write Table into Data Catalog.
I tried 2 approaches:
1. Use Glue default classifier - this is preferred...
1
answers
0
votes
124
views
asked 14 days agolg...
Hello.
I am trying to configure specific iam permission for an user. I need a permission for only read tables from existing Data Catalog.
So, I have configured this policiy:
```
{
"Version":...
0
answers
0
votes
185
views
asked 15 days agolg...
I've successfully set up AWS Glue with an RDS database serving as the data source and a Snowflake database as the data target. In this setup, I've configured AWS Glue crawlers to catalog the metadata...
0
answers
0
votes
177
views
asked 15 days agolg...
I am running a glue job with python script shell(version 3.9) and glue version is 3.0. I am passing 8 arguments to the glue job and accessing it using getResolvedOptions(args, options). One of the...
2
answers
0
votes
156
views
asked 17 days agolg...