CloudWatch Logs

CloudWatch is hosted in the AWS Public Zone, so it can be used on-premise without particular networking configuration.

Concepts

  • Log Class:

    • The Standard log class is a full-featured option

    • The Infrequent Access log class is a lower-cost option for logs that you access less frequently. It supports a subset of the Standard log class capabilities.

  • Log events: a record of some activity recorded by the application or resource being monitored. E.g.: a line in the apache access log

  • Log streams: a sequence of log events that share the same source. E.g.: all the lines coming from the apache access logs from one instance.

  • Log Group: groups of log streams that share the same retention, monitoring, and access control settings. E.g.: All the streams from an ASG of EC2 instances running the Apache webserver whose logs are being streamed. You can export the log group data to an S3 bucket (encryption of the bucket is supported but DSSE-KMS). Data can also be exported to OpenSearch but you may incur in high usage charges for large amounts of data.

  • Metric filters: filters to log events that capture data to export as a metric. You can give dimensions and a unit to the metric.

Log Classes

Standard Log Class

  • Fully managed log ingestion and storage

  • Cross-account features

  • Encryption with AWS KMS

  • CloudWatch Logs Insights query commands

  • CloudWatch Logs Insights discovered fields

  • Natural language query assist

  • CloudWatch Logs Anomaly Detection

  • Compare to previous time range

  • Subscription filters

  • Export to Amazon S3

  • GetLogEvents and FilterLogEvents API operations

  • Metric filters

  • Container Insights log ingestion

  • Lambda Insights log ingestion

  • Sensitive data protection with masking

  • Embedded metrics format

Infrequent Access Log Class

  • Fully managed log ingestion and storage

  • Cross-account features

  • Encryption with AWS KMS

  • CloudWatch Logs Insights query commands (not all commands)

Use Cases

  • You need to query your log data

  • You need to be able to detect and debug using Live Tail

  • Monitoring EC2 instances

  • Monitor AWS CloudTrail logged events

  • Audit and mask sensitive data

  • Log Route 53 DNS queries