Skip to main content
Version: Next

Deployment & Configuration

To integrate NFO with DataSet, you will install the Scalyr Agent to act as a syslog listener. This method is superior to basic file monitoring as it allows DataSet to handle the NFO JSON stream with high concurrency.

1. Install the Scalyr Agent​

  1. Obtain API Key: Visit DataSet API Keys and generate a Log Write Access key.
  2. Install Agent: Run the following on your NFO host (or a separate collector VM):
wget -q https://www.scalyr.com/scalyr-repo/stable/latest/install-scalyr-agent-2.sh
sudo bash ./install-scalyr-agent-2.sh --set-api-key "YOUR_API_KEY"

2. Configure the Agent for NFO​

You must configure a Syslog Monitor within the agent to receive the NFO stream and assign it the NFO parser name.

  1. Edit /etc/scalyr-agent-2/agent.json.
  2. Add the following to the monitors section:
monitors: [
  {
    "module": "scalyr_agent.builtin_monitors.syslog_monitor",
    "protocols": "udp:515",
    "accept_remote_connections": true,
    "message_log": "nfo.log",
    "parser": "NFO"
  }
]

  1. Restart the agent: sudo scalyr-agent-2 restart

3. Configure the NFO Parser​

NFO logs are structured JSON. You must define an NFO parser in the DataSet UI so the fields are searchable as discrete attributes.

  1. In the DataSet UI, navigate to Configuration > Parsers.
  2. Create a new parser named NFO and paste the following configuration:
{
  "formats": [
    {
      "format": "${parse=json}$",
      "rewrites": [
        { "input": "time", "output": "timestamp", "match": ".*", "replace": "$0" }
      ]
    }
  ]
}

  1. Save the parser. This ensures every field in the NFO JSON (like src_ip, bytes, and app_name) is automatically indexed.

4. Configure NFO Output​

Set NFO to stream the data to the agent you just configured.

  1. In the NFO GUI, go to Data Outputs and click (+).
  2. Type: Select JSON (UDP).
  3. Address: localhost (if the agent is on the same machine) or the Agent's IP.
  4. Port: 515 (to match the protocols setting in agent.json).

5. Verification​

Once the data is flowing, you can use the Inspect Log Line tool to verify that the NFO parser is breaking down the JSON correctly.