"For a Food procuring company which will procure thousands of products from farmers each product having a UPC(Unique Product Code) that are listed in an excel sheet with 500000 records and this sheet has to be read and sent to third party databases outside AWS through API Gateway however those databases can only process one record at a time(this is the limitation of those 3 external databases) and will return metadata of a product for each individual UPC as we do not know which database that product can be found so 3 calls has to be made for one UPC and likewise 500000*3 calls(only one call at a time and NOT batch processing) and store the metadata in DynamoDB and then we need to be able to query and display the product UPC with metadata in a dashboard for a BusinessAnalyst to easily refresh or query the product metadata details in a dashboard.All AWS Services must be accessibel only through Intranet using a VPN connection.
Please let us know what would be the efficient architecture for this use case with all the services, protocols and implementation details."
