CloudWatch Agent on EC2
It allows to access certain performance counters of the operating system itself.
CloudWatch Agent allows to look at the processes running on an instance and their stats (like memory consumption), has access to certain operating system level performance metrics that you cannot see outside the instance. You also might want to allow access to system and application logging from within the EC2 instance.
Neither CloudWatch nor CloudWatch Logs can natively capture any data or any logs that’s happening inside an EC2 instance. The CloudWatch agent is required to run inside an EC2 instance
Logically for the CloudWatch agent to function it needs to have the configuration and permissions to be able to send that data into both of those products.
We can attach this IAM role to the EC2 instance (See Instance Profile), providing anything running on the instance with access to the CloudWatch and CloudWatch logs service.
Example Configuration
You can configure one log group for every log file that you want to inject into the product and then within each log group there’ll be a log stream for each instance performing this logging.
You can automate this by using CloudFormation to include that agent configuration for every single instance that you provision.
CloudWatch agent comes with a number of ways to obtain and store the configuration that it will use to send this data into CloudWatch logs: one of those ways is using the parameter store and store the agent configuration as a parameter.
Scaling the instances configuration
Once you have competed the wizard started with sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-config-wizard a configuration file will be available at /opt/aws/amazon-cloudwatch-agent/bin/config.json. The wizard will itself ask you if you want to store the configuration in the Parameter Store.
Storing the configuration in the Parameter Store will allow the former to be picked by a number of instances, maybe from an ASG, with no admin overhead. It’ll be as simple as adding a line in cfn-init or User Data:
$ sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c ssm:<the-name-given-to-the-config-during-the-wizard> -s