BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage News Amazon CloudWatch Supports JSON Logs and Integrates AWS CloudTrail

Amazon CloudWatch Supports JSON Logs and Integrates AWS CloudTrail

This item in japanese

Bookmarks

Shortly after releasing the AWS CloudTrail Processing Library (CPL), Amazon Web Services has also integrated AWS CloudTrail with Amazon CloudWatch Logs to enable alarms and respective "notifications from CloudWatch, triggered by specific API activity captured by CloudTrail". The implied support for monitoring JSON-formatted logs has recently been officially released as well.

AWS CloudTrail records all API calls made in an AWS account for logging and auditing use cases including security analysis, change tracking, compliance aid and operational troubleshooting, as explained in more detail in our previous coverage. Amazon CloudWatch monitors AWS resources by recording metrics and offering alarms to provide "system-wide visibility into resource utilization, application performance, and operational health". It also provides log monitoring and storage for application, operating system and custom logs.

CloudTrail Integration with CloudWatch Logs

As outlined in our previous coverage, CloudWatch Logs focuses on alerting rather than a search interface by "monitoring specific phrases, values or patterns", which also applies to the CloudTrail integration. Amazon Web Service's Chief Evangelist Jeff Barr illustrates a common use case by using a metric filter to continuously match CloudTrail log events with patterns originating from failed sign-in attempts so that "each occurrence of 'AccessDenied' or 'UnauthorizedOperation' will increase the value (count) of the metric by 1":

Amazon CloudWatch Logs Metric Filter

As usual with CloudWatch, users can then create a custom alarm for this metric to perform "one or more actions based on the value of the metric relative to a given threshold over a number of time periods". Actions can be Amazon SNS notifications, Auto Scaling actions or EC2 actions that stop, terminate or (as of recently) recover an instance. SNS notifications trigger messages to SNS topic subscribers in turn, like sending an email when there is an alert for one or more authorization failures per minute in the example at hand.

CloudWatch Logs JSON support

The initial term based metric filter and value extraction from space-delimited log events already enables important use cases like extracting transferred bytes from an Apache HTTP log. However, CloudTrail logs are delivered in JSON format, just like an increasing number of other AWS and third party log events. Consequently, CloudWatch Logs now also offers JSON based metric filters.

The JSON metric filter pattern syntax is straight forward and supports string comparisons based on '=, !=', including support for the asterisk '*' wildcard character to match any text before or after a search term, precise numeric comparisons based on '<, >, >=, <=, =, !=', and also compound conditions using OR '||' and AND '&&' – here are a few examples:

  • '{ $.eventType = "UpdateTrail" }' – Filter on the event type being UpdateTrail
  • '{ $.sourceIpAdress != 10.42.* }' – Filter on the IP address being outside the subnet 10.42 prefix
  • '{ $.objectList[1].id = 2 }' – Filter on the second entry in objectList having a property called id with a value of 2
  • '{ $.latency > 42 }' – Filter on the latency being higher than 42
  • '{ ($.user.id = 2 && $.users[0].email = "nonmatch") || $.actions[2] = "GET" }' – Filter conditionally

Counting the number of matching log events covers many use cases, but the most enabling feature is probably the extraction of metric values from JSON logs. For example, a simple metric filter like "{ $.latency = * }" will match all log events with respective latency values, which can then be published to CloudWatch by referring to the JSON selector "$.latency" – the following JSON log event would yield a metric value of 42 in CloudWatch:

{
"latency": 42,
"requestType": "GET"
}

 

The CloudTrail user guide explains how to integrate with CloudWatch Logs monitoring using the AWS Management Console or the AWS CLI and provides examples how to create CloudWatch alarms for console sign-in failuresnetwork access control list changes or security group configuration changes. Support is available via the Amazon CloudWatch and AWS CloudTrail forums. The integration itself is free of charge, CloudWatch Logs, CloudWatch alarms and Amazon SNS notifications incur the standard usage based cost for these services.

Rate this Article

Adoption
Style

BT