This seems like relatively light load use case, in that case, this all can be implemented within pure-Python libs with Python Glue Shell.
- Use python libs for excel to extract/process data from excel files.
- Use python REST API for Sharepoint to extract Sharepoint data.
- Append columns to track source for data lineage.
- Get processing counts and sent data over SNS after processing
AWS Glue ETL from SQL Server DB on premises to Snowflake on AWS cloudasked 4 months ago
Informatica to AWS Glue Migration PathAccepted Answerasked 3 years ago
ETL automation and testingasked 7 months ago
ETL Workflow Orchestration Step functions and/or Glue Workflows??Accepted Answerasked 3 years ago
ETL Replication from Oracle to Postgresasked 3 months ago
Talend to Glue migrationAccepted Answerasked 2 years ago
Glue ETL Job with external connection to Redshift - filter then extract?Accepted Answerasked 4 years ago
ETL using AWS Glueasked 3 months ago
How fast can glue ETL convert data to parquet?Accepted AnswerMODERATORasked 3 years ago
Using Glue for dimensional model ETL into RedshiftAccepted Answerasked 2 years ago