Ab Initio Integration with AWS (for ETL workloads)
IHAC that uses Ab Initio on an enterprise scale to do on-prem ETL workloads. They are now wanting to build a Data Lake on AWS, and would prefer using this already established tool to write from source into S3 buckets, and then perhaps into Redshift. Before we go down the road of steering the customer to Glue, Lambda, SDLF frameworks, I want to explore if they can use Ab Initio as a first choice.
Hence does Ab Initio recognise AWS a source (and target) for ETL workloads, and can it be easily used within AWS (similar to how Informatica Cloud fulfills this pattern)?
i used Ab>initio for multiple years for different customers. It has a lot of functionality when it comes to ETL. It also has it's own repo EME. If i understand your question correctly in first phase you want to check if Abinitio can load load in S3 and may be RedShift... the answer is Yes. We can use S3 and Redshift as targets using different S3 components. Even when Abinitio installation is on-prem. If plan is to move Abinitio in AWS then it can be implement on EC2 (min - co>op, EME) with large highly compute machines (C-series). But it may be very costly solution (licensing + EC2 cost).
Converting Ab>initio jobs in Glue, Lambda, SDLF frameworks - logically yes but this will need deeper understanding of the logic implemented and may need multiple AWS services to come together for effective solution.
Ab Initio Integration with AWS (for ETL workloads)Accepted Answerasked 2 years ago
Can AWS Glue handles complex transformation jobs like Talend. Can any one share their observations around AWS Glue vs any standard ETLasked 6 months ago
What's the best way to filter out duplicated records in a Glue ETL Job with bookmarking enabled?asked 6 months ago
Optimizing AI/ML for Sustainability Workloads: Ask the ExpertAccepted AnswerMODERATORasked 21 days ago
What are the recommended data integration tools for NetSuite ERP?Accepted Answerasked 2 years ago
How fast can glue ETL convert data to parquet?Accepted AnswerMODERATORasked 3 years ago
Sagemaker and Data on DatabasesAccepted Answerasked 4 years ago
Data ingestion using DMS or Glue?Accepted Answerasked 2 years ago
I need to read S3 data, transform and put into Data Catalog. Should I be using a Crawler?Accepted Answer
I am trying to write an ETL job to the Data Catalog but its writing the Headers as DataAccepted Answer