Version: Next

Exporting Logs

When troubleshooting complex issues or working with NetFlow Logic Support, you will need to access specific log files or generate a diagnostic bundle. NFO provides both a Web UI utility for quick access and standard filesystem paths for deep analysis.

1. Viewing Logs in the Web UI

The easiest way to check system health without CLI access is through the NFO interface.

Navigate to Help > Logs.
Select a log from the dropdown menu:

server.log: Core Engine events, module processing, and flow ingestion/output errors.
nfi-updater.log: EDFN enrichment events, threat feed updates, and Cloud Flow Log ingestion status.
catalina.out: Web UI errors, API authentication issues, and Controller logic.

Click Tail to watch logs in real-time. This is particularly useful while reproducing an issue or verifying a configuration change.

2. Generating a Diagnostic Bundle

A diagnostic bundle is a compressed package containing logs, configuration files, and system metadata. Always attach this bundle to your support tickets.

Navigate to Help > Support.
Click Download Logs.
The system will gather all relevant files and download a compressed archive (e.g., nfo-logs-[timestamp].zip or .tar.gz) to your browser.

[!IMPORTANT] The diagnostic bundle is sanitized of sensitive password information, but it does contain your configuration logic and network metadata (IP addresses/hostnames).

3. Filesystem Log Locations

If the Web UI is inaccessible, you can find the logs at these default installation paths:

Component	Linux Path	Windows Path
NFO Engine	`/opt/flowintegrator/logs/bak/server.log`	`C:\NetFlow\rest\logs\server.log`
NFO Controller	`/opt/flowintegrator/tomcat/logs/catalina.out`	`C:\NetFlow\tomcat\logs\catalina.log`
EDFN Agent	`/opt/nfi-updater/logs/nfi-updater.log`	`C:\NetFlow\updater\logs\nfi-updater.log`

4. Log Rotation & Retention

NFO manages log size automatically to prevent disk exhaustion. These settings can be tuned if you need to keep longer historical data for compliance.

Configuration File

Log settings are managed in the server.cfg file (Linux: /opt/flowintegrator/server/etc/server.cfg).

LOG_DIR: Defines the directory where logs are stored.
Rotation: By default, NFO rotates logs when they reach 10MB. It retains the 10 most recent rotated files (e.g., server.log.1 through server.log.10).

5. Exporting NFO and EDFN Logs

You can send audit logs and other statistics to external log monitoring systems via Syslog. This allows for centralized logging, real-time monitoring, and advanced analysis. There are two ways to do this:

Using syslog-ng
Configuring destination in log4j

syslog-ng Configuration

This section explains how to use syslog-ng 3.35+ to forward NFO and EDFN logs in RFC 5424 format to a remote Splunk indexer.

1. Prerequisites

Component	Version / Notes
OS	RHEL / CentOS 8‑stream or similar
syslog-ng	≥ 3.35 (multiline support in `wildcard-file()`)
Splunk	Indexer or Heavy Forwarder listening on UDP port

Ensure SELinux and firewall rules allow outgoing traffic on the chosen port.

2. Directory Layout

/opt/nfi-updater/logs/          # *.log files from the updater agent
/opt/flowintegrator/logs/       # *.log files from the flow integrator
/etc/syslog-ng/conf.d/
    └─ nfi-updater-forward.conf # configuration shown below

3. The syslog-ng Configuration Explained

File: /etc/syslog-ng/conf.d/nfi‑updater‑forward.conf

source s_nfi {
    wildcard-file(
        base-dir("/opt/nfi-updater/logs")
        filename-pattern("*.log")
        program_override("edfn")
        max-files(500)
        follow-freq(10)
        flags(validate-utf8)
    );
};

source s_nfo {
    wildcard-file(
        base-dir("/opt/flowintegrator/logs")
        filename-pattern("*.log")
        program_override("nfo")
        max-files(500)
        follow-freq(10)
        flags(validate-utf8)
    );
};

template t_rfc5424_with_file {
    template("<${PRI}>1 ${ISODATE} ${HOST} ${PROGRAM} ${PID} - [meta@47450 file=\"${FILE_NAME}\" app=\"${PROGRAM}\"] ${MSG}\n");
    template-escape(no);
};

destination d_remote {
    network("<DESTINATION_IP>"
            port(<UDP_PORT>)
            transport("udp")
            template(t_rfc5424_with_file));
};

log { source(s_nfi); destination(d_remote); };
log { source(s_nfo); destination(d_remote); };

3.1 Sources (`s_nfi`, `s_nfo`)

wildcard-file() – modern file source that tails new log files in real time, including those created after rotation.
program_override() – forces the PROGRAM field to edfn or nfo, letting Splunk identify the origin without regex.
flags(validate-utf8) – discards invalid UTF‑8 sequences to keep the pipeline healthy.
follow-freq(10) – polls the directory every 10 seconds. Increase this value on quieter systems to reduce CPU usage.
max-files(500) – maximum open files per source. Raise if log rotation leaves many generations.

3.2 Template (`t_rfc5424_with_file`)

Outputs fully‑formed RFC 5424 with a structured‑data element that adds the file name and the overridden program tag:

<meta@47450 file="catalina.0.log" app="nfo">

Splunk’s built‑in syslog sourcetype recognises the standard header; you can then extract file and app via KV_MODE = auto or TRANSFORMS- rules.

3.3 Destination (`d_remote`)

IP – replace <DESTINATION_IP> with your Splunk indexer’s address.
UDP PORT – lightweight but unreliable. For guaranteed delivery, switch to transport("tcp") or transport("tls") with certificates.

4. Deployment Steps

Install syslog-ng via package manager.
Place nfi‑updater‑forward.conf under /etc/syslog-ng/conf.d/.
Test config: syslog-ng -s (syntax check).
Reload: systemctl restart syslog-ng.
On Splunk, create a new UDP input on port 10514 and choose sourcetype=syslog or the custom type above.
Verify events in Search & Reporting (index=* app=nfo | head 20).

log4j Configuration

NFO Release 2.11.1 and earlier

In releases prior to NFO 2.11.1, please modify the ${nfo_home}/tomcat/bin/setenv.sh file by adding the following properties to CATALINA_OPTS: -Dlog4j.configurationFile=log4j2.xml,../../etc/nfo-log4j2.xml

JAVA_HOME=../../java/jre
CATALINA_OPTS="$CATALINA_OPTS -XX:ErrorFile=$CATALINA_BASE/../logs/hs_err_pid%p.log \
    -Xss1280k -Xms128m -Xmx3g -Dfile.encoding=UTF-8 \
    -Dcom.sun.net.ssl.checkRevocation=true -Dcom.sun.security.enableCRLDP=true \
    -Djava.library.path=../webapps/ROOT/WEB-INF/native \
    -Dlog4j.configurationFile=log4j2.xml,../../etc/nfo-log4j2.xml \
    --add-opens=java.base/java.lang=ALL-UNNAMED \
    --add-opens=java.base/java.io=ALL-UNNAMED \
    --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED \
    --add-opens=java.base/java.nio=ALL-UNNAMED \
    --add-opens=java.base/java.net=ALL-UNNAMED \
    --add-opens=java.base/java.util=ALL-UNNAMED \
    --add-opens=java.sql/java.sql=ALL-UNNAMED"
CATALINA_PID=catalina.pid
CATALINA_OUT="$CATALINA_BASE/../logs/catalina.out"
UMASK="0022"
export LD_LIBRARY_PATH="$CATALINA_BASE/../server/bin:$LD_LIBRARY_PATH"

Sending Logs via Syslog

This section describes how to configure NetFlow Optimizer to send server audit logs, controller audit logs, and statistics to a Syslog server.

Configuration Steps

Create or Modify nfo-log4j2.xml:

Navigate to the NetFlow Optimizer home directory (${nfo_home}/etc/).
Create or open the nfo-log4j2.xml file.
Add or replace the contents of the file with the following XML configuration:

<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
    <Appenders>
        <Syslog name="syslog" format="RFC5424" host="localhost" port="514"
                protocol="UDP" appName="tomcat_nfo" facility="LOCAL0"
                includeMDC="true" newLineEscape="\n " />
    </Appenders>
    <Loggers>
        <!-- NFO controller audit logs -->
        <Logger name="com.netflowlogic.nf2sl.service.impl.ApplicationLogService"
                additivity="true" level="info">
            <AppenderRef ref="syslog" />
        </Logger>
        <!-- NFO server audit logs -->
        <Logger name="nfoServerLogs" additivity="false" level="info">
            <AppenderRef ref="syslog" />
        </Logger>
        <!-- Statistics: mem, cpu, udp stat, NFO server stat -->
        <Logger name="com.netflowlogic.nf2sl.service.scheduling.ResourceUsageTask"
                additivity="false" level="info">
            <AppenderRef ref="syslog" />
        </Logger>
    </Loggers>
</Configuration>

Configure Syslog Server Details:
- Modify the host and port attributes in the <Syslog> appender to match the address and port of your Syslog server.
- Choose between UDP or TCP for the protocol attribute, based on your Syslog server's configuration.
- The appName attribute is set to tomcat_nfo, which helps identify the source of the logs in your Syslog server.
- The facility attribute is set to LOCAL0, which can be adjusted to match your Syslog server's configuration.
Include or Exclude Usernames:
- The includeMDC="true" attribute in the <Syslog> appender controls whether usernames are included in the structured data elements of the Syslog messages.
- If you want to include usernames, leave includeMDC="true".
- If you want to exclude usernames, change it to includeMDC="false".
Restart NFO Service:
- Restart the tomcat_nfo.service to apply the changes.

Considerations

Ensure that your Syslog server is configured to receive messages on the specified host and port.
Verify network connectivity between the NFO machine and the Syslog server.
Adjust the facility and other Syslog parameters to match your Syslog server's requirements.
If using TCP, ensure that your syslog server is configured to listen for TCP connections.

Sending Logs via Splunk HEC

This section describes how to configure NetFlow Optimizer to send server audit logs, controller audit logs, and statistics to a Splunk using Splunk HEC.

Configuration Steps

Create or Modify nfo-log4j2.xml:

Navigate to the NetFlow Optimizer home directory (${nfo_home}/etc/).
Create or open the nfo-log4j2.xml file.
Add or replace the contents of the file with the following XML configuration:

<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
    <Appenders>
        <Http name="hec"
            url="https://host:8088/services/collector/raw"
            verifyHostname="false">
            <Property name="Authorization" value="Splunk 00000000-0000-0000-0000-000000000000" />
            <!--<Property name="x-splunk-request-channel" value="00000000-0000-0000-0000-ffffffffffff" />-->
            <Rfc5424Layout appName="tomcat_nfo" facility="LOCAL0" includeMDC="true" escapeNL="\n " />
            <Ssl>
                <TrustStore location="/opt/flowintegrator/etc/splunk.p12"
                            type="pkcs12" password="password"  />
            </Ssl>
        </Http>
    </Appenders>
    <Loggers>
        <!-- NFO controller audit logs -->
        <Logger name="com.netflowlogic.nf2sl.service.impl.ApplicationLogService"
                additivity="true" level="info">
            <AppenderRef ref="hec" />
        </Logger>
        <!-- NFO server audit logs -->
        <Logger name="nfoServerLogs" additivity="false" level="info">
            <AppenderRef ref="hec" />
        </Logger>
        <!-- Statistics: mem, cpu, udp stat, NFO server stat -->
        <Logger name="com.netflowlogic.nf2sl.service.scheduling.ResourceUsageTask"
                additivity="false" level="info">
            <AppenderRef ref="hec" />
        </Logger>
    </Loggers>
</Configuration>

Configure <Http> appender:
- Modify the url attribute by changing scheme, host, and port.
- Change Authorization property replacing 00000000-...-000000000000 to an actual Splunk HEC token.
- Uncomment x-splunk-request-channel property and enter random UUID if indexer acknowledgement is enabled.
- Remove or comment <Ssl> element when HTTP scheme is utilized instead of HTTPS.
- Configure <Ssl> element (HTTPS scheme only):
  - Create PKCS12 trusstore with Splunk trusted certificate. If you have PEM-file with the Splunk certificates chain, use the following command to convert it to PKCS12 format:
```
keytool -import -alias splunk -file splunk.pem -keystore splunk.p12 -deststoretype pkcs12
```
  - Configure <TrustStore> element attributes.
- The appName attribute is set to tomcat_nfo, which helps identify the source of the logs in your Syslog server.
- The facility attribute is set to LOCAL0, which can be adjusted to match your Syslog server's configuration.
Include or Exclude Usernames:
- The includeMDC="true" attribute in the <Syslog> appender controls whether usernames are included in the structured data elements of the Syslog messages.
- If you want to include usernames, leave includeMDC="true".
- If you want to exclude usernames, change it to includeMDC="false".
Restart NFO Service:
- Restart the tomcat_nfo.service to apply the changes.

Additional Configuration

For Network appenders configuration details and additional parameters visit:

Network Appenders

Exporting Logs

1. Viewing Logs in the Web UI​

2. Generating a Diagnostic Bundle​

3. Filesystem Log Locations​

4. Log Rotation & Retention​

Configuration File​

5. Exporting NFO and EDFN Logs​

syslog-ng Configuration​

1. Prerequisites​

2. Directory Layout​

3. The syslog-ng Configuration Explained​

3.1 Sources (s_nfi, s_nfo)​

3.2 Template (t_rfc5424_with_file)​

3.3 Destination (d_remote)​

4. Deployment Steps​

log4j Configuration​

NFO Release 2.11.1 and earlier​

Sending Logs via Syslog​

Configuration Steps​

Considerations​

Sending Logs via Splunk HEC​

Configuration Steps​

Additional Configuration​

1. Viewing Logs in the Web UI

2. Generating a Diagnostic Bundle

3. Filesystem Log Locations

4. Log Rotation & Retention

Configuration File

5. Exporting NFO and EDFN Logs

syslog-ng Configuration

1. Prerequisites

2. Directory Layout

3. The syslog-ng Configuration Explained

3.1 Sources (`s_nfi`, `s_nfo`)

3.2 Template (`t_rfc5424_with_file`)

3.3 Destination (`d_remote`)

4. Deployment Steps

log4j Configuration

NFO Release 2.11.1 and earlier

Sending Logs via Syslog

Configuration Steps

Considerations

Sending Logs via Splunk HEC

Configuration Steps

Additional Configuration