Questions tagged with Data Lakes
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Is it possible to integrate opentelemetry(ADOT) and promethus managed by AWS with a data lake. The data lake could be both on edge or on AWS? If yes then how does it work? How can oepntemetry and...
0
answers
0
votes
31
views
asked 12 hours agolg...
At every 30 minutes it saying "ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 20000 files in this round of vacuum" but when I calculate the my table metadata size it didn't changed before and after the...
2
answers
0
votes
436
views
asked 3 months agolg...
Hi! I have been searching and playing around with services and cannot seem to find what I need.
I am using the following architecture to guide me in building out my end-to-end solution:...
2
answers
0
votes
316
views
asked 7 months agolg...
When executing a task the last step is validating the data migrated with the source against target apparently using Athena, I have the following error:
2023-11-07T22:09:04 [VALIDATOR_TARGE ]E: Not...
1
answers
1
votes
748
views
asked 7 months agolg...
Hello
I have created a resource link to a shared database from a different account. I am able to query the tables within the database but "Show tables from <database>" and "View Tables" on the AWS...
1
answers
0
votes
464
views
asked 8 months agolg...
My Glue 4.0 jobs have suddenly stopped working with error message below. As it is related to boto3, I am unable to make any changes to library config. Pls advise.
NB: I noticed that urllib3 released...
0
answers
0
votes
255
views
asked 8 months agolg...
Hello everyone,
### 1. Context
We have a Delta Lake where we write our tables in **S3** in **Delta format**, and we use the **Glue Catalog** for queries in **Athena**. Tables are created both with...
1
answers
1
votes
682
views
asked 9 months agolg...
I am trying to create a Delta Table from spark sql using the Glue meta catalog.
I can correctly query a Delta table using the Glue metastore:
```
%%sql
select * from `my_table` VERSION AS OF 1 limit...
2
answers
0
votes
1648
views
asked 10 months agolg...
Looks like attempting to write to a Delta Lake table from a DynamicFrame is not working. The Visual Glue interface generates a script like:
```
s3 =...
2
answers
0
votes
592
views
asked 10 months agolg...
Hello team,
We are planning to build a data lake in AWS that will contain regularly extracted data from an on-prem data warehouse. The purpose of this data lake is to serve the following purposes in...
2
answers
0
votes
520
views
asked a year agolg...
This CDK code produces a "Resource did not stabilize" error:
```
data_location = lakeformation.CfnPrincipalPermissions.DataLocationResourceProperty(
catalog_id=Aws.ACCOUNT_ID,
...
1
answers
0
votes
431
views
asked a year agolg...
In CDK, I am registering a datalake location with the following code:
```
lakeformation.CfnResource(scope, "S3BucketRegistrationResource",
...
1
answers
0
votes
678
views
asked a year agolg...