Questions tagged with Data Lakes
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
We are utilizing AWS Glue Catalog along with Athena v3 and Iceberg for our data solution.
We create tables using a CREATE TABLE query in Athena, setting the ‘table_type’ parameter to ‘ICEBERG’. Our...
1
answers
0
votes
627
views
asked a year agolg...
Hello everybody,
I'm working for a small retail company, where we want to move our data to aws. Our data can be from different sources (postgresDB, MySQL DB or other unstructured data sources or...
2
answers
0
votes
333
views
asked a year agolg...
The goal is to create an ETL job that can be altered and executed by non-technical users in our organization, which is why we are sticking to only visuals and not code.
The problem is that the nodes...
1
answers
0
votes
831
views
asked a year agolg...
Hi TEam,
Me ew to AWS Data lake configuration. As per the doc, it refers to create the Glue connection which will be internally used by AWS BluePrint.
My database is running on EC2 instance of...
2
answers
0
votes
526
views
asked a year agolg...
Error to view tablelg...
After using a crawler to create a table it shows the following error which I don't know how to fix.
![Error](/media/postImages/original/IMr9UNFbk4Q-qth24BoQLcPQ)
1
answers
0
votes
248
views
asked a year agolg...
Hello, I'm looking to push Salesforce data to a Data Lake. This data lake table needs to hold different versions of the record. I experimented with AppFlow, but I couldn't really get the control I...
1
answers
0
votes
484
views
asked a year agolg...
Hi folks,
Curious to know if it is possible to expose an AWS-hosted Snowflake data lake/warehouse directly using an API Gateway? I found documentation on Snowflake's site which mentions the use of a...
2
answers
0
votes
472
views
asked a year agolg...
Hi,
Is there a way to perform time travel with Athena queries on Apache Hudi tables in a way similar to the one described here [Implement a cdc based upsert in a data lake using Apache Iceberg and...
2
answers
0
votes
859
views
asked 2 years agolg...
I am having trouble connecting MWAA to snowflake. I used the MWAA UI to automatically create a VPC, security group, and IAM for my airflow environment. I can not get snowflake to show as an option in...
1
answers
0
votes
1510
views
asked 2 years agolg...
I am building an ETL pipeline using primarily state machines, Athena, and the Glue catalog. In general things work in the following way:
1. A table, partitioned by "version", exists in the Glue...
1
answers
0
votes
295
views
asked 2 years agolg...
I created the tutorial from this...
1
answers
0
votes
563
views
asked 2 years agolg...
I'm a beginner in tasks of date engineer.
I have a taks to create a data lakehouse and i'm tryding undestand how to do it using these tools: DMS, S3, Glue and Hudi.
I already created a simple data...
1
answers
0
votes
649
views
asked 2 years agolg...