Azure Top Traffic Monitor (10467 / 20467)

Description

This Module identifies Azure VMs with the most traffic. It consolidates NSG Flow Logs records over a period of time (Data Collection Interval) which all have the same combination of the following fields:
    Source IP address
    Destination IP address
    Source port number
    Destination port number
    Layer 3 protocol
This information is provided per Virtual Network (Exporter). The Module also enriches them with Azure data not reported in NSG Flow Logs natively.
De-duplication: optionally the Module can report consolidated flows only from authoritative Virtual Network. Authoritative NSG is determined as follows. The Module sums up bytes, packets, and connections between two communicating peers over data collection interval reported by each Virtual Network. A Virtual Network with most connections (flows) for each consolidated flow is considered authoritative, and flows reported for the same two peers by all other Virtual Networks are discarded.

Parameters

Parameter Name
Description
Comments
Data Collection Interval, sec
Module logic execution interval
min = 5 sec, max = 1800 sec, default = 30 sec
N – number of reported hosts
The number of top hosts reported per Virtual Network
min = 0, max = 100000, default = 50 (0 indicates all hosts are reported)
Share of total traffic reported, %
Reported percent of total traffic per Virtual Network
e.g. 98 - indicates that reported consolidated flows consuming 98% of total NetFlow exporter traffic; min = 1%, max = 100%, default = 95%
Not more than N consolidated flows will be reported
Enable(1) or disable (0) reporting by authoritative exporters only
If set to 1 (de-duplication enabled), the Module reports flows only from authoritative Virtual Networks (exporters)
default = 0
Azure VM Instances
VMs with IPv4, IPv6, Subscription ID, Subscription Name, VM Name, NSG Name, Virtual Network Name, etc
Provided by EDF agent
Azure IPv4 Routes
IP range, source and destination Virtual Network hash
Provided by EDF agent
Azure IPv6 Routes
IP range, source and destination Virtual Network hash
Provided by EDF agent
Azure IPv4 Ranges
IPv4 ranges, Service name, Region
Provided by EDF agent
Azure IPv6 Ranges
IPv6 ranges, Service name, Region
Provided by EDF agent

Input

Azure NSG Flow Logs

Syslog/JSON Message Fields

Key
Field Description
Comments
nfc_id
Message type identifier
“nfc_id=20467”
exp_ip
Exporter Ipv4 address
<IPv4 address> (added for compatibility with other flows)
protocol
Transport Protocol (TCP = 6, UDP = 17)
<number>
direction
The direction of the traffic flow
<string>
decision
Whether traffic was allowed or denied
<string>, valid values are “A” for allowed and “D” for denied
src_ip
Source VM instance IPv4 address
<IPv4 address>
[src_ip6]
Source VM instance Ipv6 address
<IPv6 address>
[src_host]
Source host name
<string>, included when FQDN is on
[src_subs_id]
Source Subscription ID
<string>
[src_subs_name]
Source Subscription Name
<string>
[src_vm_name]
Source VM name
<string>
[src_nsg_name]
Source NSG name
<string>
[src_vnet_name]
Source Virtual Network name
<string>
[src_subnetwork_name]
Source Subnet name
<string>
[src_region]
Source Region
<string>
[src_res_grp_name]
Source Resource Group Name
<string>
src_port
Source port number
<number>
dest_ip
Destination VM instance IPv4 address
<IPv4 address>
[dest_ip6]
Destination VM instance Ipv6 address
<IPv6 address>
[dest_host]
Destination host name
<string>, included when FQDN is on
[dest_subs_id]
Destination Subscription ID
<string>
[dest_subs_name]
Destination Subscription Name
<string>
[dest_vm_name]
Destination VM name
<string>
[dest_nsg_name]
Destination NSG name
<string>
[dest_vnet_name]
Destination Virtual Network name
<string>
[dest_subnetwork_name]
Destination Subnet name
<string>
[dest_region]
Destination Region
<string>
[dest_res_grp_name]
Destination Resource Group Name
<string>
dest_port
Destination port number
<number>
packets_in
Total number of packets in the consolidated flows from the source to the destination
<number>
bytes_in
Total number of Layer 3 bytes in the packets of the consolidated flows from the source to the destination
<number>
packets_out
Total number of packets in the consolidated flows from the destination to the source
<number>
bytes_out
Total number of Layer 3 bytes in the packets of the consolidated flows from the destination to the source
<number>
flow_count
Number of consolidated flows
<number>
percent_of_total
Percent of Total (bytes)
<decimal>, e.g. 25.444% is 25.444
flow_start_time
Start time of the first consolidated flow
<time>
flow_end_time
End of the last consolidated flow
<time>
t_int
Observation time interval, msec
<number>
Last modified 2mo ago