Common AWS monitoring and logging tools
CloudWatch, CloudTrail are your best bets
In any AWS environment it is always a good idea to enable logging and monitoring of the components in your workloads. But since there are components to monitoring services that incur charges it is more cost-efficient to enable more detailed logging on say a production workload rather than a development one.
Assume we are looking at a production workload that needs to track what happens, and who does it. This is where AWS CloudWatch and CloudTrail come in, which are two fully managed services that can help you find both of these.
The key differences between the 2 is that CloudWatch tells you what is happening with your resources and CloudTrail tells you who is doing it and from where. When used in conjunction with each other they can provide you a detailed overview across all of your resources.
Lets take a look at the aforementioned production workload, a simple 3 tier web app, which looks like this:
- Public subnet with an instance running a front end web server
- Private subnet with both a logic instance and a database server instance
CloudWatch works at the system and application level and also allows you to have custom log files. You can monitor in near real time for specific values or patterns.
An example for our scenario here is that it would allow us to track CPU utilization across the instances, and if they surpassed a defined utilization threshold for a defined period of time then CloudWatch will trigger an alarm. That alarm can then be sent out to another service like EC2 auto scaling or SQS. It will also store the log data that occurred during this sequence of events that can be accessed later with the CLI for analysis.
So with a single service you can both automate your architecture to react to events, and also have a reliable stream of event logs you can analyze later on to improve efficiency moving forward.
CloudTrail works at the user activity and API level. It will allow you to continuously monitor and retain account activity related to actions across your AWS infrastructure. Note that the events it monitors for are only related to management events like create, modify, and delete API calls and account activity.
If we have CloudTrail enabled and a developer as an IAM user calls the CloudTrail StartLogging API from the CloudTrail console before making changes to the instances, it will be kept as a single log record. Anytime any user makes a management related API call it will be kept as a record.
If you combine the two you are able to track how your infrastructure is being utilized and to what degree, and who is doing what inside of your AWS account and resources. This gives you a ton of visibility into your day to day operations and allows you to focus less on manually monitoring your IT/Cloud environment and focusing more on your business. It is also important to remember that for security and compliance reasons AWS provides many compliance reports for its services via AWS Artifact.
So you now have a service that is incredible useful as well as compliant with many governing bodies. The documentation for CloudWatch can be found here, and for CloudTrail here.
As always, best of luck on your continued journey through cloud computing.
