I am new to AWS and looking for guidance to design a FTP solution
Infrastructure :
A zipped file (encrypted) plus a checksum file will be available on FTP server in Data Center 1 (once daily at around 1 am). Data Centre 1 cannot be reached via Public internet.
But it has a connectivity to Data Centre 2 via MPLS.
Datacentre 2 has Direct Connect Link Set up with AWS Ireland.
Requirement:
Get the zipped file from on premise server in DC!, and perform following : DEcrypting, perform check on checksum and DEcompression.
Store the flat files (from zip file) in AWS London region in S3. These files will be required for 12 months and then deleted. These flat files wont be accessed frequently and will be saved for audit purposes.
Only need to run the SFTP operation once on daily basis
Pre Reqs
Firewall ports will be opened
No agent can be installed on any of the On Premise server
Backup / DR solution required as well
What is the best way to achieve this. I thought of using Lambda function but how will network side of things work. Can Lambda function be able to reach to FTP server in DC1 which is sitting behind a firewall. Can all the above operations (checksum, decrypt and decompression) be performed using Lambda function. We can create separate Lambda function for each operation.
or to use EC2 instance and get node.js installed.
Thanks for your response. Can this solution we used for file sizes in the region 40-50MB. There will be just one 40-50MB file and a checksum file which needs fetching from FTP server. Will VPC have a unique static IP address, as this IP address will need to be whitelisted.