Installation

Pre-installation Steps

NetFlow data is sent to Splunk from NFO in syslog or JSON formats. It could be ingested directly on UDP input port, or received through Splunk forwarders, or rsyslog / syslog-ng and Splunk forwarders. You can also use Splunk for Syslog Connect. For more information on Splunk for Syslog Connect and NFO filter configuration, see <link>

Configure Splunk Data inputs accordingly per your accepted best practices.

NetFlow Analytics for Splunk App (netflow) (https://splunkbase.splunk.com/app/489/) relies on *flow data processed by NetFlow Optimizer™ (NFO) and enables you to analyze it using Splunk® Enterprise or Splunk® Cloud. To download a free trial of NetFlow Optimizer, please visit https://www.netflowlogic.com/download/ and register to receive the FREE trial key. Please see NetFlow Optimizer Installation and NetFlow Optimizer Administration Guide and follow instructions for your platform.

Install the App on your Splunk Search Heads.

This App requires the Technology Add-On for NetFlow (TA-netflow) (https://splunkbase.splunk.com/app/1838/). This Add-on collects *flow data processed by NetFlow Optimizer™ (NFO) software by NetFlow Logic, providing Splunk CIM compliant field names, eventtypes and tags for *flow data. The Add-on can be used with the App or with Splunk Enterprise Security.

Install this Add-on on your Splunk Search Heads, Indexers, and Heavy Forwarders.

Several dashboards of the App rely on Force Directed App for Splunk for Topology View. Please make sure it is installed in your Splunk environment: https://splunkbase.splunk.com/app/3767/ to use Topology View.

Installing into Splunk enterprise

Technology Add-on for NetFlow Installation

  1. Install Technology Add-on for NetFlow (TA-netflow).

  2. By default NetFlow Optimizer events will be stored in main index. In case you want to use another index, for example flowintegrator, please create the $SPLUNK_ROOT/etc/apps/TA-netflow/local/indexes.conf file, and add the following lines to it:

    [flowintegrator]
    homePath = $SPLUNK_DB/flowintegrator/nfi_traffic/db
    coldPath = $SPLUNK_DB/flowintegrator/nfi_traffic/colddb
    thawedPath = $SPLUNK_DB/flowintegrator/thaweddb
  3. Set sourcetype = flowintegrator In your Data inputs

  4. Restart Splunk

NetFlow Analytics for Splunk App Installation

  1. Install NetFlow Analytics for Splunk App (netflow).

  2. Create the file if it does not already exist: $SPLUNK_ROOT/etc/apps/netflow/local/macros.conf

  3. and add the following lines to it:

    [netflow_index]
    definition = index=flowintegrator sourcetype=flowintegrator
  4. Restart Splunk for the changes to take effect.

If you are installing the App in Splunk Cloud or use Splunk GUI, and would like to use a different index from the default main or continue using a custom index for your NetFlow events, please perform the following on your searchheads:

In Settings->Advanced search->Search macros find the “netflow_index” macro, click on it and change the value in the Definition field from:

sourcetype=flowintegrator

to:

index=flowintegrator sourcetype=flowintegrator

When configuration is completed, click "Do not show again" checkbox on App Setup dashboard:

If you don't see this checkbox (it is not visible in older browsers) please use the workaround below:

Create the file:

$SPLUNK_ROOT/etc/apps/netflow/local/app.conf

and add the following lines to it:

[install]
is_configured = 1

Restart Splunk for the changes to take effect.

Configure Universal Forwarder Input

Create or modify %SPLUNK_HOME%/etc/system/local/inputs.conf file as follows. In general there are two options, either to listen directly for netflow events on a specific port or optionally to monitor files created by syslog-ng or rsyslog.

Receiving Syslogs Directly from NFO (UDP port 10514)

Add the following lines to inputs.conf file and modify it for your netflow index, if necessary:

[udp://10514]
sourcetype = flowintegrator
index = flowintegrator

Configuring Universal Forwarder with syslog-ng or rsyslog

In this scenario syslog-ng or rsyslog are configured to listen to syslogs sent by NFO on a UDP port 10514. Syslog-ng or rsyslog are usually writing the logs into configurable directories. In this example we assume that those are written to /var/log/netflow.

Add the following lines to inputs.conf file and modify it for your netflow index, if necessary:

[monitor:///var/log/netflow]
sourcetype = flowintegrator
index = flowintegrator

It is very important to set sourcetype=flowintegrator and to point it to the index where Netflow Analytics for Splunk App and Add-on are expecting it.

Configure Universal Forwarder Output (Target Indexers)

During the installation of the Universal Forwarders a Receiving Indexer can be configured, as it can be seen here:

It is an optional step during the installation. If it was not configured or if load balancing is required, additional Receiving Indexers can be added later by adding to the %SPLUNK_HOME%/etc/system/local/outputs.conf file:

[tcpout]
defaultGroup = default-autolb-group
[tcpout:default-autolb-group]
server = 10.1.0.100:9997,10.1.0.101:9997

More info about load balancing: http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Setuploadbalancingd#How_load_balancing_works

Installing into a Splunk Cloud Deployment

You must be a Splunk Cloud administrator to install and manage apps in your Splunk Cloud deployment. The procedure for installing apps and add-ons for use with your Splunk Cloud instance depends on the type of your Splunk Cloud deployment and the version of Splunk Cloud that you are running. Please visit Splunk Cloud User Manual for details: https://docs.splunk.com/Documentation/SplunkCloud/8.0.2007/User/SelfServiceAppInstall.

As NetFlow Optimizer sends data out over UDP, use a Universal Forwarder or a Heavy Forwarder or Splunk Connect for Syslog to listen to the NFO source and forward the data to your Splunk Cloud deployment. Please visit Splunk Cloud Admin Manual for details: https://docs.splunk.com/Documentation/SplunkCloud/8.1.2103/Admin/IntroGDI

Installing with Splunk Connect for Syslog

You can use Splunk Connect for Syslog (SC4S) as a forwarder between NFO and Splunk Enterprise or Splunk Cloud. This section describes how to install and configure SC4S and configure HTTP Event Collector (HEC).

Splunk HEC configuration

  1. On Splunk add HEC data input: https://splunk-connect-for-syslog.readthedocs.io/en/latest/gettingstarted/#configure-the-splunk-http-event-collector

    a. Navigate to Settings > Data inputs > HTTP Event Collector > + Add new

    b. Enter HEC name

    c. On Input settings page don't select allowed indexes

    d. Save HEC input

  2. If HEC data input is disable, enable it or enable all tokens in the Global Settings

SC4S Installation and Configuration

  1. Install Docker Engine. For example, instruction for centos: https://docs.docker.com/engine/install/centos/

  2. Configure UDP receive buffer size and enable packet forwarding for IPv4. Edit /etc/sysctl.conf: net.core.rmem_default = 17039360 net.core.rmem_max = 17039360 net.ipv4.ip_forward=1

  3. Save and apply to the kernel: sysctl -p

  4. Configure sc4s environment /opt/sc4s/env_file:

    SPLUNK_HEC_URL=https://<splunk_host>:8088 SPLUNK_HEC_TOKEN=<hec-token-value> #Uncomment the following line if using untrusted SSL certificates SC4S_DEST_SPLUNK_HEC_TLS_VERIFY=no

  5. Configure filter to distinguish NFO syslogs. Put following content into /opt/sc4s/local/config/app_parsers/app-nfo.conf file:

    #
    # Copyright (C) 2021 NetFlow Logic
    # All rights reserved.
    #
    block parser nfo-parser() {
    channel {
    rewrite {
    # set defaults these values can be overidden at run time by splunk_metadata.csv
    r_set_splunk_dest_default(
    index("flowintegrator")
    source("sc4s:nfo")
    sourcetype("flowintegrator")
    vendor_product("nfo")
    template("t_msg_only")
    );
    # add nfo_hostname fields
    set("${HOST}", value("fields.nfo_hostname"));
    # remove nfc_id from the message
    subst('nfc_id=(\d+) ', '', value("MESSAGE") flags(store-matches));
    # add nfc_id field
    set("$1", value("fields.nfc_id") condition("$1" ne ""));
    };
    };
    };
    application nfo[sc4s-syslog] {
    filter {
    "${PROGRAM}" eq "NFO";
    };
    parser { nfo-parser(); };
    };

  6. Start SC4S: systemctl start sc4s

  7. Configure NFO output: <sc4s_host>:514

Useful Commands

  1. start/stop/restart SC4S: systemctl start sc4s systemctl stop sc4s systemctl restart sc4s

  2. To check docker logs: docker logs SC4S

  3. To validate SC4S status on Splunk side search following: index=* sourcetype=sc4s:events index=* sourcetype=sc4s:events "starting up"

  4. To find NFO syslogs: index="flowintegrator" source="sc4s:nfo"