Logging system in AWS

There are several ways to design a logging system in AWS to track unprocessed or errored out payloads. One common approach is to use a combination of AWS services such as Amazon Kinesis, AWS Lambda, and Amazon S3.

In terms of "dos and don'ts," some things to keep in mind include:

Do ensure that your logging system is designed to handle high volumes of data and can scale as needed. Don't store sensitive data in plaintext in your logs. Do use encryption to protect your logs in transit and at rest. Don't store your logs indefinitely. Consider setting up a retention policy to automatically delete old logs after a set period of time. Do test your logging system regularly to ensure that it is working as expected and can detect and handle errors and failures.