Ab Initio Integration with AWS (for ETL workloads)
IHAC that uses Ab Initio on an enterprise scale to do on-prem ETL workloads. They are now wanting to build a Data Lake on AWS, and would prefer using this already established tool to write from source into S3 buckets, and then perhaps into Redshift. Before we go down the road of steering the customer to Glue, Lambda, SDLF frameworks, I want to explore if they can use Ab Initio as a first choice.
Hence does Ab Initio recognise AWS a source (and target) for ETL workloads, and can it be easily used within AWS (similar to how Informatica Cloud fulfills this pattern)?
i used Ab>initio for multiple years for different customers. It has a lot of functionality when it comes to ETL. It also has it's own repo EME. If i understand your question correctly in first phase you want to check if Abinitio can load load in S3 and may be RedShift... the answer is Yes. We can use S3 and Redshift as targets using different S3 components. Even when Abinitio installation is on-prem. If plan is to move Abinitio in AWS then it can be implement on EC2 (min - co>op, EME) with large highly compute machines (C-series). But it may be very costly solution (licensing + EC2 cost).
Converting Ab>initio jobs in Glue, Lambda, SDLF frameworks - logically yes but this will need deeper understanding of the logic implemented and may need multiple AWS services to come together for effective solution.
Relevant questions
Ab Initio Integration with AWS (for ETL workloads)
Accepted Answerasked 2 years agoCan AWS Glue handles complex transformation jobs like Talend. Can any one share their observations around AWS Glue vs any standard ETL
asked 6 months agoWhat's the best way to filter out duplicated records in a Glue ETL Job with bookmarking enabled?
asked 6 months agoOptimizing AI/ML for Sustainability Workloads: Ask the Expert
Accepted AnswerWhat are the recommended data integration tools for NetSuite ERP?
Accepted Answerasked 2 years agoHow fast can glue ETL convert data to parquet?
Accepted AnswerSagemaker and Data on Databases
Accepted Answerasked 4 years agoData ingestion using DMS or Glue?
Accepted Answerasked 2 years agoI need to read S3 data, transform and put into Data Catalog. Should I be using a Crawler?
Accepted Answerasked 3 months agoI am trying to write an ETL job to the Data Catalog but its writing the Headers as Data
Accepted Answerasked 3 months ago