Thursday, April 21, 2016

Send JSON from rsyslog to Kibana

Leave a Comment

I'm using rsyslog to watch over my syslogs and send them over to Logstash+Kibana.

My syslogs messages are logged as JSON. They can look something like this:

{"foo":"bar", "timegenerated": 43843274834} 

rsyslog configuration as so:

module(load="omelasticsearch")  #define a template to print all fields of the message template(name="messageToES" type="list" option.json="on") {   property(name="msg") } *.* action(type="omelasticsearch"        server="localserverhere"        serverport="80"        template="messageToES") 

The Kibana is fine, since if I run a CURL command to it, it receives the record. The code as below:

curl -XPOST myserver/test/bar -d '{"test": "baz", "timegenerated":1447145221519}' 

When I run rsyslogs and point it to a dummy server, I can see the incoming requests with the valid json. However, when I point it back to my logstash server, it doesn't show up in logstash or kibana.

Does anyone know how to send syslogs as json into Kibana/logstash?

1 Answers

Answers 1

I've never used it, but it looks like you are missing things from your config file. The docs have a pretty thorough example:

module(load="omelasticsearch") template(name="testTemplate"      type="list"      option.json="on") {        constant(value="{")          constant(value="\"timestamp\":\"")      property(name="timereported" dateFormat="rfc3339")          constant(value="\",\"message\":\"")     property(name="msg")          constant(value="\",\"host\":\"")        property(name="hostname")          constant(value="\",\"severity\":\"")    property(name="syslogseverity-text")          constant(value="\",\"facility\":\"")    property(name="syslogfacility-text")          constant(value="\",\"syslogtag\":\"")   property(name="syslogtag")        constant(value="\"}")      } action(type="omelasticsearch"    server="myserver.local"    serverport="9200"    template="testTemplate"    searchIndex="test-index"    searchType="test-type"    bulkmode="on"    queue.type="linkedlist"    queue.size="5000"    queue.dequeuebatchsize="300"    action.resumeretrycount="-1") 

Based on what you are trying to do, it looks like you need to plug in localserverhere where it shows myserver.local. It also looks like you have ES accepting stuff on port 80, so you'd put in 80 instead of 9200.

If You Enjoyed This, Take 5 Seconds To Share It

0 comments:

Post a Comment