Azure Top Traffic Monitor (10467 / 20467)
Description
This Module identifies Azure VMs with the most traffic. It consolidates NSG Flow Logs records over a period of time (Data Collection Interval) which all have the same combination of the following fields:
- Source IP address
- Destination IP address
- Source port number
- Destination port number
- Layer 3 protocol
This information is provided per Virtual Network (Exporter). The Module also enriches them with Azure data not reported in NSG Flow Logs natively.
De-duplication: optionally the Module can report consolidated flows only from authoritative Virtual Network. Authoritative NSG is determined as follows. The Module sums up bytes, packets, and connections between two communicating peers over data collection interval reported by each Virtual Network. A Virtual Network with most connections (flows) for each consolidated flow is considered authoritative, and flows reported for the same two peers by all other Virtual Networks are discarded.
Parameters
| Parameter Name | Description | Comments |
|---|---|---|
| Data Collection Interval, sec | Module logic execution interval | min = 5 sec, max = 1800 sec, default = 30 sec |
| N – number of reported hosts | The number of top hosts reported per Virtual Network | min = 0, max = 100000, default = 50 (0 indicates all hosts are reported) |
| Share of total traffic reported, % | Reported percent of total traffic per Virtual Network | e.g. 98 - indicates that reported consolidated flows consuming 98% of total NetFlow exporter traffic; min = 1%, max = 100%, default = 95%. Not more than N consolidated flows will be reported |
| Enable(1) or disable (0) reporting by authoritative exporters only | If set to 1 (de-duplication enabled), the Module reports flows only from authoritative Virtual Networks (exporters) | default = 0 |
| Azure VM Instances | VMs with IPv4, IPv6, Subscription ID, Subscription Name, VM Name, NSG Name, Virtual Network Name, etc | Provided by EDF agent |
| Azure IPv4 Routes | IP range, source and destination Virtual Network hash | Provided by EDF agent |
| Azure IPv6 Routes | IP range, source and destination Virtual Network hash | Provided by EDF agent |
| Azure IPv4 Ranges | IPv4 ranges, Service name, Region | Provided by EDF agent |
| Azure IPv6 Ranges | IPv6 ranges, Service name, Region | Provided by EDF agent |
Input
Azure NSG Flow Logs
Syslog/JSON Message Fields
| Key | Field Description | Comments |
|---|---|---|
| nfc_id | Message type identifier | “nfc_id=20467” |
| exp_ip | Exporter Ipv4 address | <IPv4 address> (added for compatibility with other flows) |
| protocol | Transport Protocol (TCP = 6, UDP = 17) | <number> |
| direction | The direction of the traffic flow | <string> |
| decision | Whether traffic was allowed or denied | <string>, valid values are “A” for allowed and “D” for denied |
| src_ip | Source VM instance IPv4 address | <IPv4 address> |
| [src_ip6] | Source VM instance Ipv6 address | <IPv6 address> |
| [src_host] | Source host name | <string>, included when FQDN is on |
| [src_subs_id] | Source Subscription ID | <string> |
| [src_subs_name] | Source Subscription Name | <string> |
| [src_vm_name] | Source VM name | <string> |
| [src_nsg_name] | Source NSG name | <string> |
| [src_vnet_name] | Source Virtual Network name | <string> |
| [src_subnetwork_name] | Source Subnet name | <string> |
| [src_region] | Source Region | <string> |
| [src_res_grp_name] | Source Resource Group Name | <string> |
| src_port | Source port number | <number> |
| dest_ip | Destination VM instance IPv4 address | <IPv4 address> |
| [dest_ip6] | Destination VM instance Ipv6 address | <IPv6 address> |
| [dest_host] | Destination host name | <string>, included when FQDN is on |
| [dest_subs_id] | Destination Subscription ID | <string> |
| [dest_subs_name] | Destination Subscription Name | <string> |
| [dest_vm_name] | Destination VM name | <string> |
| [dest_nsg_name] | Destination NSG name | <string> |
| [dest_vnet_name] | Destination Virtual Network name | <string> |
| [dest_subnetwork_name] | Destination Subnet name | <string> |
| [dest_region] | Destination Region | <string> |
| [dest_res_grp_name] | Destination Resource Group Name | <string> |
| dest_port | Destination port number | <number> |
| packets_in | Total number of packets in the consolidated flows from the source to the destination | <number> |
| bytes_in | Total number of Layer 3 bytes in the packets of the consolidated flows from the source to the destination | <number> |
| packets_out | Total number of packets in the consolidated flows from the destination to the source | <number> |
| bytes_out | Total number of Layer 3 bytes in the packets of the consolidated flows from the destination to the source | <number> |
| flow_count | Number of consolidated flows | <number> |
| percent_of_total | Percent of Total (bytes) | <decimal>, e.g. 25.444% is 25.444 |
| flow_start_time | Start time of the first consolidated flow | <time> |
| flow_end_time | End of the last consolidated flow | <time> |
| t_int | Observation time interval, msec | <number> |