NFO Server and Controller Logs

Understanding NFO Logs

Understanding these logs is essential for troubleshooting issues effectively. The following tables provide a brief overview of the types of information you can find in each log category:

NFO Logs

NFO logs are located in $NFO_HOME/logs. The table below summarizes the log files and their descriptions:

Log File Name	Description	Source
nfc_server.<timestamp>.log	Contains information about the NFO core process, enabled NFO Modules, and detected errors.	NFO Server
server.<timestamp>.log	Contains information about when the NFO core process was started, as well as redirected process standard output.	NFO Server
process.log<.#>	Contains information about CPU and memory usage, as well as statistics about incoming and outgoing traffic (NetFlow and SNMP).	NFO Server
nf2sl.log<.#>	Contains information about the NFO controller component (including keep alive), user login and configuration activities, and detected errors.	NFO Controller
localhost_access_log.<date>.txt	Contains information about requests to the GUI, including IP address and access time.	Tomcat
catalina.#.log	Contains information about the Tomcat process and detected errors.	Tomcat
catalina.out	Contains Tomcat process redirected standard output.	Tomcat
nfo-output.log<.#>	Contains information about various outputs over TCP, such as Kafka, Amazon OpenSearch, Azure Log Analytics Workspace, etc.	NFO Controller
templates.log	Contains information about received NFv9/IPFIX templates.	NFO Controller
nfo_audit.log	Contains information about unresponsive devices placed on the "Skip SNMP polling" list.	NFO Controller

Log Levels

NFO logging levels are (from least to most verbose):

• Error
• Debug
• Verbose
• Flood

Default level is Error.

You can change the log level by navigating to Tracing and Configuration and selecting a Tracing verbosity level you need. Press Save button. Do not restart NFO.

By analyzing these logs and understanding the functionalities of each NFO component, you can effectively diagnose and resolve issues related to data flow, processing, and overall NFO health.

sever.log

Problem	What to look for	Proposed action
NFO Server stopped by OS (for ex. Out of memory killer)	\.\.\/\.\.\/server\/bin\/\/flowintegrator\.sh: line 90: [0-9]+ Killed .*	Start NFO Server, Make sure “Keep alive” is enabled
NFO Server process failed	\.\.\/\.\.\/server\/bin\/\/flowintegrator\.sh: line 90: [0-9]+ Segmentation fault .*	Start NFO Server, Make sure “Keep alive” is enabled. Send logs to support
NFO Server can’t start	ON CONFIG: local configuration failed: 22	Check error messages on GUI (Status page), correct configuration, Send logs to support

nfc_server.<timestamp>.log

Problem	What to look for	Proposed action
NFO Server needs more memory to perform this operation	out of memory	Add RAM to VM or physical memory to host

catalina.#.log

Problem	What to look for	Proposed action
NFO Controller does not start (socket open not permitted / filesystem failures / wrong configuration)	[org.apache.catalina.core.StandardContext startInternal] Context [] startup failed due to previous errors	Try to restart Controller, Send logs to support

nf2sl.log

Problem	What to look for	Proposed action
NFO Controller: Java heap space too small	java.lang.OutOfMemoryError	Edit -Xmx param in tomcat/bin/setenv.sh
Java call stack failure (for ex. WL not loaded)	java.lang.StackOverflowError	Shouldn’t happened, Send logs to support
NFO Server can’t start	ON CONFIG: [AbstractExceptionMapper] NetFlow Integrator server not started	Check error messages on GUI (Status page), correct configuration, Send logs to support
NFO starts with errors, it works incorrectly (for ex. Service loading error)	[CRITICAL]	Check error messages on GUI (Status page), Send logs to support
NFO Server can potentially work incorrect or not work (for ex. Controller stops Server or about to stop due to license expiration)	[ALERT]	Check the license, If license is current, Send logs to support
All types of errors, also some EDFN errors	[ERROR]	Check error messages on GUI (Status page)
All types of warnings, also some EDFN warnings	[WARNING]	Check error messages on GUI (Status page)
Someone (not EDFN) use ‘updater’ credentials over REST API	[NfiResource] Unexpected updater User-Agent:	Check nf2sl.log and access logs: logs/localhost_access_log.*.txt

External NFO Monitoring

You can send audit logs and other statistics to external log monitoring systems via Syslog. This allows for centralized logging, real-time monitoring, and advanced analysis. The following logs will be sent using the procedure below:

• process.log
• messages from Status page
• nfo_audit.log
• nf2sl.log

Sending Logs via Syslog

This section describes how to configure NetFlow Optimizer to send server audit logs, controller audit logs, and statistics to a Syslog server.

Configuration Steps

1. Create or Modify nfo-log4j2.xml:

   • Navigate to the NetFlow Optimizer home directory (${nfo_home}/etc/).
   • Create or open the nfo-log4j2.xml file.
   • Add or replace the contents of the file with the following XML configuration:

   <?xml version="1.0" encoding="UTF-8" ?>
   <Configuration>
       <Appenders>
           <Syslog name="syslog" format="RFC5424" host="localhost" port="514"
                   protocol="UDP" appName="tomcat_nfo" facility="LOCAL0"
                   includeMDC="true" newLineEscape="\n " />
       </Appenders>
       <Loggers>
           <Logger name="com.netflowlogic.nf2sl.service.impl.ApplicationLogService"
                   additivity="true" level="info">
               <AppenderRef ref="syslog" />
           </Logger>
           <Logger name="nfoServerLogs" additivity="false" level="info">
               <AppenderRef ref="syslog" />
           </Logger>
           <Logger name="com.netflowlogic.nf2sl.service.scheduling.ResourceUsageTask"
                   additivity="false" level="info">
               <AppenderRef ref="syslog" />
           </Logger>
       </Loggers>
   </Configuration>

2. Configure Syslog Server Details:

   • Modify the host and port attributes in the <Syslog> appender to match the address and port of your Syslog server.
   • Choose between UDP or TCP for the protocol attribute, based on your Syslog server's configuration.
   • The appName attribute is set to tomcat_nfo, which helps identify the source of the logs in your Syslog server.
   • The facility attribute is set to LOCAL0, which can be adjusted to match your Syslog server's configuration.

3. Include or Exclude Usernames:

   • The includeMDC="true" attribute in the <Syslog> appender controls whether usernames are included in the structured data elements of the Syslog messages.
   • If you want to include usernames, leave includeMDC="true".
   • If you want to exclude usernames, change it to includeMDC="false".

4. Restart NFO Service:

   • Restart the tomcat_nfo.service to apply the changes.

Additional Configuration

For Syslog appender configuration details and additional parameters visit:

• Syslog Appender

Considerations

• Ensure that your Syslog server is configured to receive messages on the specified host and port.
• Verify network connectivity between the NFO machine and the Syslog server.
• Adjust the facility and other Syslog parameters to match your Syslog server's requirements.
• If using TCP, ensure that your syslog server is configured to listen for TCP connections.