Exporting NFO and EDFN Logs
You can send audit logs and other statistics to external log monitoring systems via Syslog. This allows for centralized logging, real-time monitoring, and advanced analysis. There are two ways to do this:
- Using syslog-ng
- Configuring destination in log4j
syslog-ng Configuration
This section explains how to use syslog-ng 3.35+ to forward NFO and EDFN logs in RFC 5424 format to a remote Splunk indexer.
1. Prerequisites
Component | Version / Notes |
---|---|
OS | RHEL / CentOS 8‑stream or similar |
syslog-ng | ≥ 3.35 (multiline support in wildcard-file() ) |
Splunk | Indexer or Heavy Forwarder listening on UDP port |
Ensure SELinux and firewall rules allow outgoing traffic on the chosen port.
2. Directory Layout
/opt/nfi-updater/logs/ # *.log files from the updater agent
/opt/flowintegrator/logs/ # *.log files from the flow integrator
/etc/syslog-ng/conf.d/
└─ nfi-updater-forward.conf # configuration shown below
3. The syslog-ng Configuration Explained
File: /etc/syslog-ng/conf.d/nfi‑updater‑forward.conf
source s_nfi {
wildcard-file(
base-dir("/opt/nfi-updater/logs")
filename-pattern("*.log")
program_override("edfn")
max-files(500)
follow-freq(10)
flags(validate-utf8)
);
};
source s_nfo {
wildcard-file(
base-dir("/opt/flowintegrator/logs")
filename-pattern("*.log")
program_override("nfo")
max-files(500)
follow-freq(10)
flags(validate-utf8)
);
};
template t_rfc5424_with_file {
template("<${PRI}>1 ${ISODATE} ${HOST} ${PROGRAM} ${PID} - [meta@47450 file=\"${FILE_NAME}\" app=\"${PROGRAM}\"] ${MSG}\n");
template-escape(no);
};
destination d_remote {
network("<DESTINATION_IP>"
port(<UDP_PORT>)
transport("udp")
template(t_rfc5424_with_file));
};
log { source(s_nfi); destination(d_remote); };
log { source(s_nfo); destination(d_remote); };
3.1 Sources (s_nfi
, s_nfo
)
wildcard-file()
– modern file source that tails new log files in real time, including those created after rotation.program_override()
– forces the PROGRAM field toedfn
ornfo
, letting Splunk identify the origin without regex.flags(validate-utf8)
– discards invalid UTF‑8 sequences to keep the pipeline healthy.follow-freq(10)
– polls the directory every 10 seconds. Increase this value on quieter systems to reduce CPU usage.max-files(500)
– maximum open files per source. Raise if log rotation leaves many generations.
3.2 Template (t_rfc5424_with_file
)
Outputs fully‑formed RFC 5424 with a structured‑data element that adds the file name and the overridden program tag:
<meta@47450 file="catalina.0.log" app="nfo">
Splunk’s built‑in syslog
sourcetype recognises the standard header; you can then extract file
and app
via KV_MODE = auto
or TRANSFORMS-
rules.
3.3 Destination (d_remote
)
- IP – replace
<DESTINATION_IP>
with your Splunk indexer’s address. - UDP PORT – lightweight but unreliable. For guaranteed delivery, switch to
transport("tcp")
ortransport("tls")
with certificates.
4. Deployment Steps
- Install syslog-ng via package manager.
- Place
nfi‑updater‑forward.conf
under/etc/syslog-ng/conf.d/
. - Test config:
syslog-ng -s
(syntax check). - Reload:
systemctl restart syslog-ng
. - On Splunk, create a new UDP input on port 10514 and choose
sourcetype=syslog
or the custom type above. - Verify events in Search & Reporting (
index=* app=nfo | head 20
).
log4j Configuration
NFO Release 2.11.1 and earlier
In releases prior to NFO 2.11.1, please modify the ${nfo_home}/tomcat/bin/setenv.sh
file by adding the following properties to CATALINA_OPTS: -Dlog4j.configurationFile=log4j2.xml,../../etc/nfo-log4j2.xml
JAVA_HOME=../../java/jre
CATALINA_OPTS="$CATALINA_OPTS -XX:ErrorFile=$CATALINA_BASE/../logs/hs_err_pid%p.log \
-Xss1280k -Xms128m -Xmx3g -Dfile.encoding=UTF-8 \
-Dcom.sun.net.ssl.checkRevocation=true -Dcom.sun.security.enableCRLDP=true \
-Djava.library.path=../webapps/ROOT/WEB-INF/native \
-Dlog4j.configurationFile=log4j2.xml,../../etc/nfo-log4j2.xml \
--add-opens=java.base/java.lang=ALL-UNNAMED \
--add-opens=java.base/java.io=ALL-UNNAMED \
--add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED \
--add-opens=java.base/java.nio=ALL-UNNAMED \
--add-opens=java.base/java.net=ALL-UNNAMED \
--add-opens=java.base/java.util=ALL-UNNAMED \
--add-opens=java.sql/java.sql=ALL-UNNAMED"
CATALINA_PID=catalina.pid
CATALINA_OUT="$CATALINA_BASE/../logs/catalina.out"
UMASK="0022"
export LD_LIBRARY_PATH="$CATALINA_BASE/../server/bin:$LD_LIBRARY_PATH"
Sending Logs via Syslog
This section describes how to configure NetFlow Optimizer to send server audit logs, controller audit logs, and statistics to a Syslog server.
Configuration Steps
-
Create or Modify
nfo-log4j2.xml
:- Navigate to the NetFlow Optimizer home directory (
${nfo_home}/etc/
). - Create or open the
nfo-log4j2.xml
file. - Add or replace the contents of the file with the following XML configuration:
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
<Appenders>
<Syslog name="syslog" format="RFC5424" host="localhost" port="514"
protocol="UDP" appName="tomcat_nfo" facility="LOCAL0"
includeMDC="true" newLineEscape="\n " />
</Appenders>
<Loggers>
<!-- NFO controller audit logs -->
<Logger name="com.netflowlogic.nf2sl.service.impl.ApplicationLogService"
additivity="true" level="info">
<AppenderRef ref="syslog" />
</Logger>
<!-- NFO server audit logs -->
<Logger name="nfoServerLogs" additivity="false" level="info">
<AppenderRef ref="syslog" />
</Logger>
<!-- Statistics: mem, cpu, udp stat, NFO server stat -->
<Logger name="com.netflowlogic.nf2sl.service.scheduling.ResourceUsageTask"
additivity="false" level="info">
<AppenderRef ref="syslog" />
</Logger>
</Loggers>
</Configuration> - Navigate to the NetFlow Optimizer home directory (
-
Configure Syslog Server Details:
- Modify the
host
andport
attributes in the<Syslog>
appender to match the address and port of your Syslog server. - Choose between
UDP
orTCP
for theprotocol
attribute, based on your Syslog server's configuration. - The
appName
attribute is set totomcat_nfo
, which helps identify the source of the logs in your Syslog server. - The
facility
attribute is set toLOCAL0
, which can be adjusted to match your Syslog server's configuration.
- Modify the
-
Include or Exclude Usernames:
- The
includeMDC="true"
attribute in the<Syslog>
appender controls whether usernames are included in the structured data elements of the Syslog messages. - If you want to include usernames, leave
includeMDC="true"
. - If you want to exclude usernames, change it to
includeMDC="false"
.
- The
-
Restart NFO Service:
- Restart the
tomcat_nfo.service
to apply the changes.
- Restart the
Considerations
- Ensure that your Syslog server is configured to receive messages on the specified host and port.
- Verify network connectivity between the NFO machine and the Syslog server.
- Adjust the
facility
and other Syslog parameters to match your Syslog server's requirements. - If using TCP, ensure that your syslog server is configured to listen for TCP connections.
Sending Logs via Splunk HEC
This section describes how to configure NetFlow Optimizer to send server audit logs, controller audit logs, and statistics to a Splunk using Splunk HEC.
Configuration Steps
-
Create or Modify
nfo-log4j2.xml
:- Navigate to the NetFlow Optimizer home directory (
${nfo_home}/etc/
). - Create or open the
nfo-log4j2.xml
file. - Add or replace the contents of the file with the following XML configuration:
<?xml version="1.0" encoding="UTF-8" ?>
<Configuration>
<Appenders>
<Http name="hec"
url="https://host:8088/services/collector/raw"
verifyHostname="false">
<Property name="Authorization" value="Splunk 00000000-0000-0000-0000-000000000000" />
<!--<Property name="x-splunk-request-channel" value="00000000-0000-0000-0000-ffffffffffff" />-->
<Rfc5424Layout appName="tomcat_nfo" facility="LOCAL0" includeMDC="true" escapeNL="\n " />
<Ssl>
<TrustStore location="/opt/flowintegrator/etc/splunk.p12"
type="pkcs12" password="password" />
</Ssl>
</Http>
</Appenders>
<Loggers>
<!-- NFO controller audit logs -->
<Logger name="com.netflowlogic.nf2sl.service.impl.ApplicationLogService"
additivity="true" level="info">
<AppenderRef ref="hec" />
</Logger>
<!-- NFO server audit logs -->
<Logger name="nfoServerLogs" additivity="false" level="info">
<AppenderRef ref="hec" />
</Logger>
<!-- Statistics: mem, cpu, udp stat, NFO server stat -->
<Logger name="com.netflowlogic.nf2sl.service.scheduling.ResourceUsageTask"
additivity="false" level="info">
<AppenderRef ref="hec" />
</Logger>
</Loggers>
</Configuration> - Navigate to the NetFlow Optimizer home directory (
-
Configure
<Http>
appender:- Modify the
url
attribute by changing scheme, host, and port. - Change Authorization property replacing 00000000-...-000000000000 to an actual Splunk HEC token.
- Uncomment
x-splunk-request-channel
property and enter random UUID if indexer acknowledgement is enabled. - Remove or comment
<Ssl>
element when HTTP scheme is utilized instead of HTTPS. - Configure
<Ssl>
element (HTTPS scheme only):- Create PKCS12 trusstore with Splunk trusted certificate. If you have PEM-file with the Splunk certificates chain, use the following command to convert it to PKCS12 format:
keytool -import -alias splunk -file splunk.pem -keystore splunk.p12 -deststoretype pkcs12
- Configure
<TrustStore>
element attributes.
- The
appName
attribute is set totomcat_nfo
, which helps identify the source of the logs in your Syslog server. - The
facility
attribute is set toLOCAL0
, which can be adjusted to match your Syslog server's configuration.
- Modify the
-
Include or Exclude Usernames:
- The
includeMDC="true"
attribute in the<Syslog>
appender controls whether usernames are included in the structured data elements of the Syslog messages. - If you want to include usernames, leave
includeMDC="true"
. - If you want to exclude usernames, change it to
includeMDC="false"
.
- The
-
Restart NFO Service:
- Restart the
tomcat_nfo.service
to apply the changes.
- Restart the
Additional Configuration
For Network appenders configuration details and additional parameters visit: