Splunking Kafka with Kafka Connect

For this post I’m going to walk through how to use Kafka Connect (and the kafka-connect-splunk connector) along with a Splunk Heavy Forwarder to stream data from a Kafka cluster to one or many Splunk instances.  If you’re facing the prospect of building a data pipeline to consume data from a Kafka instance and forward it on to Splunk, your list of options is fairly short.

Continue reading

Advertisements

A Note on JSON Formatting

If you want to use rsyslog to reformat syslog data in to JSON format before sending off to an output, you will need to use a template.  The template that comes up the most for this purpose is this one:

Continue reading