syslog

NiFi for Syslog

Let’s build with NiFi a flow similar to what we build with Logstash to store syslog messages into an ElasticSearch index.

 

Receving the messages

We start with the ListenSyslog processor of NiFi that can be configured to listen on any UDP or TCP ports for syslog. When listening on TCP, you must specify the maximum number of concurrent TCP connections. This parameters will be dependant of the number of systems sending syslog message simultaneously to your listener.

Setting up Logstash, for Syslog

Configuration to get Syslog messages

This is the configuraton of this Logstash instance. We will use the syslog input model to listen for syslog messages from all our hosts.
We will start the Logstash on server "logstash-runner", then we will configure Rsyslog.

input {
  syslog {
    port => "10514"
    add_field => {
      "source" => "syslog"
    }
  }
}
filter {
  if [source] == "syslog" {
    grok {