Questions tagged with AWS Data Pipeline
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I'm a beginner in tasks of date engineer.
I have a taks to create a data lakehouse and i'm tryding undestand how to do it using these tools: DMS, S3, Glue and Hudi.
I already created a simple data...
1
answers
0
votes
645
views
asked 2 years agolg...
Hello! I am looking for an equivalent to this solution that MIcrosoft has flaunted called IDP intelligent data platform, it is governance + operations + analytics in one. they flaunt synapse with aml...
0
answers
0
votes
131
views
asked 2 years agolg...
I am trying to build a data pipeline for a data engineering project With the help of S3, Glue, Athena, etc., I am stuck when setting up glue crawler for indexing over data. Even I set up the role...
1
answers
2
votes
1790
views
asked 2 years agolg...
hi all,
I have a Dag that I schedule to run every quarterly using: **'schedule_interval='@quarterly'**, but for some reason it runs every day.
I don't see the run in managed airflow UI but the...
1
answers
0
votes
630
views
asked 2 years agolg...
My aws account got hacked and my lightsail instace deleted- is there any recover option for that instance -please help
Now we got our account back but the instance is no more .And i don't have any...
1
answers
0
votes
621
views
asked 2 years agolg...
Using AWS Pydeequ in databricks I am performing Data Quality checks. When I run this below mentioned code it provide only metrics results as my output (like Check_level, check_status, constraint,...
0
answers
0
votes
166
views
asked 2 years agolg...
As a publisher of data set on AWS Data Exchange, we would like the data asset downloaded by customers on their server from AWS Data Exchange to be technically un-usable when they terminate the...
1
answers
0
votes
284
views
asked 2 years agolg...
Would like to know what would be the way to migrate per user data currently hosted at on-prem vdi to AWS workspaces? Will this be a case to case migration since the on-prem VDI data might be only...
3
answers
0
votes
484
views
asked 2 years agolg...
I am setting up a new data lake and have been tasked with creating the master data tables in the data bricks delta lake component. I'm trying to do this in a use-case agnostic way (or as agnostic as...
1
answers
0
votes
387
views
asked 2 years agolg...
How can DMS do more complex mapping that is not simply 1-1 from source to target?
For example, if you need to check for data across multiple tables for some business logic and based on those results,...
2
answers
0
votes
765
views
asked 2 years agolg...
AWS DataPipeline Is Failed for EMrClusterBackup with Error Resource Not Healthy:- JobFlow retiredlg...
AWS DataPipeline Is Failed for EMrClusterBackup with Error Resource Not Healthy:- JobFlow retired
1
answers
0
votes
656
views
asked 2 years agolg...
I have a dashboard and I do not want to have any access restrictions for Amazon employees. Is it possible? any ANT group that provides access to all Amazon employees?
1
answers
0
votes
319
views
asked 2 years agolg...