Flow Logs
Flow Logs capture packets METADATA (NOT CONTENTS) at various levels: VPC, Subnet, Interface.
Thy’re NOT REAL-TIME: within an aggregation interval, also referred to as a capture window (default max 10 mins, can be configured as low as 1 min). The smaller the capture window the more logs they’ll generate. for Nitro-based instances the interval is always 1 min, regardless of the configuration.
Performance is not impacted: metadata is captured outside the path of the network traffic, and therefore does not affect network throughput or latency.
Architecture
Wherever you enable Flow Logs, they capture data downwards. Meaning that:
-
Enabling flow logs on an ENI will only capture traffic on that specific interface.
-
Enabling flow logs on a subnet will also capture interfaces traffic
-
Enabling flow logs on a VPC will also capture traffic from all the subnets and interfaces within.
They can capture metadata from ACCEPTED connections, REJECTED connections or BOTH.
Flow Log Records
Available Fields
Version 2 fields:
-
Version
-
account-id
-
interface-id
-
srcaddr
-
dstaddr
-
srcport
-
dstport
-
protocol
-
packets
-
bytes
-
start
-
end
-
action
-
log-status
Version 3 fields:
-
vpc-id
-
subnet-id
-
instance-id
-
tcp-flags
-
type
-
pkt-srcaddr
-
pkt-dstaddr
Version 4 fields:
-
region
-
az-id
-
sublocation-type
-
sublocation-id
Version 5 fields:
-
pkt-src-aws-service
-
pkt-dst-aws-service
-
flow-direction
-
traffic-path
Version 7 fields:
-
ecs-cluster-arn
-
ecs-cluster-name
-
ecs-container-instance-arn
-
ecs-container-instance-id
-
ecs-container-id
-
ecs-second-container-id
-
ecs-service-name
-
ecs-task-definition-arn
-
ecs-task-arn
-
ecs-task-id
With the default format, the flow log records include the Version 2 fields.
If a field is not applicable or could not be computed for a specific record, the record displays a '-' symbol for that entry.
You cannot customize or change the default format. To capture additional fields or a different subset of fields, specify a custom format instead.
Metadata fields that do not come directly from the packet header are best effort approximations, and their values might be missing or inaccurate.
Example:
2 123456789010 eni-1235b8ca123456789 203.0.113.12 172.31.16.139 0 0 1 4 336 1432917027 1432917142 ACCEPT OK2 123456789010 eni-1235b8ca123456789 172.31.16.139 203.0.113.12 0 0 1 4 336 1432917094 1432917142 REJECT OKsrc and dst port are 0 because protocol is 1 (ICMP, which doesn’t use ports).
These two records are related: the second is from a response connection to the first’s connection. This is probably a misconfigured NACL because it’s not allowing outgoing traffic while allowing incoming.