Questions tagged with AWS Data Pipeline
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I am trying to build a data pipeline for a data engineering project With the help of S3, Glue, Athena, etc., I am stuck when setting up glue crawler for indexing over data. Even I set up the role...
1
answers
2
votes
1743
views
asked 2 years agolg...
hi all,
I have a Dag that I schedule to run every quarterly using: **'schedule_interval='@quarterly'**, but for some reason it runs every day.
I don't see the run in managed airflow UI but the...
1
answers
0
votes
599
views
asked 2 years agolg...
My aws account got hacked and my lightsail instace deleted- is there any recover option for that instance -please help
Now we got our account back but the instance is no more .And i don't have any...
1
answers
0
votes
593
views
asked 2 years agolg...
Using AWS Pydeequ in databricks I am performing Data Quality checks. When I run this below mentioned code it provide only metrics results as my output (like Check_level, check_status, constraint,...
0
answers
0
votes
159
views
asked 2 years agolg...
As a publisher of data set on AWS Data Exchange, we would like the data asset downloaded by customers on their server from AWS Data Exchange to be technically un-usable when they terminate the...
1
answers
0
votes
270
views
asked 2 years agolg...
Would like to know what would be the way to migrate per user data currently hosted at on-prem vdi to AWS workspaces? Will this be a case to case migration since the on-prem VDI data might be only...
3
answers
0
votes
455
views
asked 2 years agolg...
I am setting up a new data lake and have been tasked with creating the master data tables in the data bricks delta lake component. I'm trying to do this in a use-case agnostic way (or as agnostic as...
1
answers
0
votes
369
views
asked 2 years agolg...
How can DMS do more complex mapping that is not simply 1-1 from source to target?
For example, if you need to check for data across multiple tables for some business logic and based on those results,...
2
answers
0
votes
726
views
asked 2 years agolg...
AWS DataPipeline Is Failed for EMrClusterBackup with Error Resource Not Healthy:- JobFlow retiredlg...
AWS DataPipeline Is Failed for EMrClusterBackup with Error Resource Not Healthy:- JobFlow retired
1
answers
0
votes
641
views
asked 2 years agolg...
I have a dashboard and I do not want to have any access restrictions for Amazon employees. Is it possible? any ANT group that provides access to all Amazon employees?
1
answers
0
votes
303
views
asked 2 years agolg...
I have AWS DB of Production and AWS DB of UAT environment. I want whenever needed I can get Production data of few tables to my UAT Database by clicking button from GUI. So GUI means we will design...
1
answers
0
votes
280
views
asked 2 years agolg...
I am trying to write data from S3 to BigQuery using Glue, this is working perfectly fine -** have used the Glue BQ connector from Marketplace**. I am able to write the data to the table at BQ but when...
1
answers
0
votes
827
views
asked 2 years agolg...