NetFlow Optimizer, a high performance flow processing engine, receives flow data from network devices and virtual public clouds, consumes and enriches flow information with other data, translates it to syslog, JSON, or other formats, and sends it to other systems where it is then correlated with other machine data and visualized.
NetFlow Optimizer consists of the following components.
Input Processing - designed for parsing and validating of various flow formats, templates, and flow options.
Logic Modules - this is where processing logic and streaming analytics are implemented. A Module may have a number of in-memory databases and watch lists. All Modules are configurable.
Conversion Modules - responsible for translating output from Logic Modules into different format, e.g. syslog, json, etc.
Output processing - responsible for sending data out or writing data to disk.
SNMP Polling and Traps
NFO allows you to configure periodic SNMP Polling of hundreds of devices as well as receive SNMP Traps (you can upload your own MIBs if they are not included with the product). All received SNMP messages are converted into syslog or JSON format and sent to your visualization platform or IT Ops monitoring system.
Reverse-DNS Lookup (FQDN)
NFO enriches flow data with the domain names associated with source and destination IP addresses in NetFlow records. FQDN data is cached for configurable expiration time in NFO to reduce DNS requests and minimize latency.
With Repeater service NFO is able to retransmit flow data it receives to other destinations capabile of receiving binary NetFlow, sFlow, IPFIX, etc. Repeter service can be configured to send flow data from certain devices to certain destinations.
Original Flow Data (OFD)
Original Flow Data conversion service enables you to convert all incoming flow records (NetFlow, IPFIX, sFlow, JFlow, etc) one-to-one to syslog or JSON format. All standard NetFlow v9, sFlow, and IPFIX elements are preconfigured in this service, and you can add the mapping of NetFlow v9 or IPFIX proprietary enterprise fields if needed.
Typically a separate output destination is configured for this service to store full fidelity data in cheap storage for forensics ro compliance.
NetFlow Recorder enables you to look back in time for security issues. You can set a rolling period of time, and store *flows in memory or on disk. Then when you press Play button all recorded flow records are sent out to your SIEM in syslog or JSON format to gain complete visibility of past network traffic.
NFO Controller keeps NFO Server alive. It also contains Configuration database and keeps track of all configuration changes. When Modules enrichment data is updated, NFO Controller determines what’s changed and updates corresponding Modules watch lists running in NFO Server.
REST API is a layer between NFO Controller and external systems. API is used by NFO GUI layer, by External Data Feeder, and by external systems that need to monitor and/or configure NFO.
Another function of the NFO Controller is to provide an interface to TCP destinations, such as:
- Azure Blob Storage
- Azure Log Analytics Workspace
- Amazon OpenSearch
External Data Feeder for NFO (EDFN)
EDFN is a component which serves as a knowledge base of information outside of the NetFlow domain. Its task is to provide NetFlow Optimizer with information generally unavailable in the data streams supplied by NetFlow/IPFIX exporters. It enables automatic updates of security threat lists, Geo IP information (integration with MaxMind or IP2Location), VM names (integration with VMware vCenter), User identity (integration with Active Directory and other identity systems).
Another function of EDFN is to enable ingestion of cloud flow logs, such as Amazon Web Services (AWS), Microsoft Azure, and Google Public Cloud (GCP).
EDFN is comprised of a Platform and a collection of Agents each of which is designed to obtain information of a certain kind. The Platform provides a common interface for the Agents’ configuration and data exchange and serves as a conduit for delivering information collected by the Agents to the NetFlow Optimizer.
EDFN is packaged with NFO, and If NFO is installed on a server with access to the internet, EDFN is installed with it automatically. However, if NFO is installed on a server without internet access, a separate EDFN installation on a server with internet access or in your cloud, is required, in which case it must be downloaded separately from NetFlow Logic’s web site – www.netflowlogic.com/download/).