CloudWatch vs CloudTrail based on specific questions

<<Sorry for long verbiage answer>>

I understand that sometimes it becomes little hard to differentiate between cloudwatch and cloudtrail, and when to use what, specially when we just start our cloud journey in AWS.

To best clarify that, I'd say this to remember the difference:

In summary, CloudWatch is primarily for monitoring the performance and health of AWS resources, while CloudTrail is focused on auditing and logging API activity within your AWS account. They can complement each other for comprehensive AWS resource management and security monitoring.

Cloudtrail persists API history for 90 days by default but cloudtrail logs can be stored in S# bucket for persistent storage of those logs. Cloudwatch keeps logs indefinitely, unless/until, you set up a shorter retention duration for that cloudwatch log group.

In a very layman term, Cloudtrail is like a person sitting at main door of a gated society,recording all the in-outs(doesn't control anything) in the register for auditing purposes where as cloudwatch is like an individual for each home within thats society, that monitors and record detailed logs of all the activities for that home.

AWS CloudWatch and AWS CloudTrail serve different purposes in monitoring and tracking activities within your AWS environment:

1. AWS CloudWatch is a Monitoring Service and is primarily a monitoring service that collects and tracks metrics and logs from various AWS resources and applications. Metrics and Alarms allows you to create custom metrics, set up alarms, and visualize performance data through dashboards. CloudWatch provides real-time monitoring and can help you troubleshoot issues by providing detailed insights into the performance of your resources. It is resource focused, which focuses on the operational health of resources, such as EC2 instances, RDS databases, and Lambda functions.

2. AWS CloudTrail is an Auditing and Logging Service and is designed for auditing and logging AWS account activity. It records API calls made on your AWS account and provides a historical record of these actions. Trail Logs are used for security analysis, resource change tracking, and compliance auditing. User Activity Tracking tracks who did what, when, and from where in your AWS account.

Now to answer your questions:

CloudTrail - user from the outside is accessing a resource : It records API activity(management and data events) of AWS resources, irrespective of whether resources/data is accessed from within AWS account or from outside. Organization trail captures all the API activity for all of it's child accounts.

CloudWatch - looking at the health of an AWS resource, ie, EC2 instance. : That's right, with cloudwatch metric, AWS resource monitoring happens, be it EC2 instance, RDS DB, lambda function etc.

CloudTrail - handles error codes. But it is unclear because what if this error code isn't caused by a user but that an EC2 instance isn't reached by another? Isn't this the health of an AWS resource? : Like I mentioned above, cloudtrail doesn't handle anything, it just records the events, like when was lambda function created, deleted, if a resource was accessed then whether that was successfully accessed or there were some access denied errors.

CloudTrail/CloudWatch - logs are needed and I see from here (see relevant quote below) that even though there might be errors in the users, the logs need to be uploaded to CloudWatch to view them.: Until now, you'd have clear understanding what both of them do. For example, you run a lambda function, logs of that execution would be auto uploaded to cloudwatch logs and you can access it from cloudwatch lambda log groups but for any customer application, you may need to place a logic to push logs to cloudwatch. Almost all the AWS resources log to cloudwatch. On the other hand cloudtrail doesn't monitor any execution/error, it simply records overall activities within account. For monitoring, you may want to integrate cloudtrail logs with cloudwatch logs and then monitor cloudwatch logs via cloudwatch alarm. Also there is very famous/common integration between AWS cloudtrail and AWS config, refer Logging AWS Config API Calls with AWS CloudTrail

Hope this helps.

Comment here if you have additional questions, happy to help.

Abhishek