• Services & Support
  • Devo.com
  • Contact
    • Contact Us
    • Request a Demo
    • Partner Inquiry
  • Log In
    • USA Devo
    • EU Devo
  • Getting started
    • Concepts
    • Devo video tutorials
  • Architecture
    • Deployment models
  • Administration
    • Users administration
      • Add a new user
    • User roles
      • Create custom roles
    • Credentials
  • User interface
    • Panels
    • Intro to Administration
      • Data Management
    • Intro to Applications
    • Notifications
    • Preferences
      • SAML
        • Google as an identity provider
        • Okta as an identity provider
        • OneLogin as an identity provider
      • Multi-factor authentication
    • Social Intelligence
  • System configuration
    • Installation of software packages
      • Ubuntu
      • Debian
      • CentOS V5-6
        • How to monitor files in CentOS using wildcards
      • Fedora
      • RHEL
    • Relays
      • The In-house Relay
      • In-house Relay rules
      • In-house Relay configuration
      • In-house Relay installation using a virtual machine
      • In-house Relay installation using the software package
      • Installing the USA relay on an Ubuntu 16 VM server
      • CentOS relay installation
      • Relay installation in any Linux distribution
      • High-availability relay
        • Helpful tips and commands
      • In-house Relay troubleshooting
    • Sending the data
      • Sending from Unix-based operating systems
        • Agent configuration for Unix systems
        • File monitoring via rsyslog
        • File monitoring via syslog-ng
        • Secure sending via rsyslog
        • Secure sending via syslog-ng
        • Sending via rsyslog
        • Sending via syslog-ng
        • Sending via traditional syslog
        • Syslog & SELinux configuration
      • Sending from Windows operating systems
        • Devo Agent for Windows events sending
          • Activating WMI
        • MagicLog
          • MagicLog installation
          • MagicLog configuration
        • ProxyServerContainer
          • ProxyServerContainer installation
          • ProxyServerContainer configuration
        • MonitorService
        • Windows Snare agent
        • Integration between Windows Azure monitoring and Devo
      • Sending from MacOS X
      • Sending from your Java application
        • JDK java.util.logging
        • Scoja client library
      • Data upload
      • Sending logs from AWS S3
      • Sending the data using Logstash
        • Sending directly to Devo
          • Configuration file examples
            • Querying databases
            • Read data from keyboard
            • Reading a file
            • Read data from Kafka
        • Sending from an In-house Relay
      • Sending logs using OPSEC LEA
      • HTTP sending
  • Supported technologies
    • Introduction to tags
    • List of supported technologies
      • Antivirus
        • av.mcafee
      • Box (PC/Server)
        • box.iptables
        • box.stat
        • box.unix
        • box.vmware
        • box.win
      • Firewall
        • firewall.checkpoint
        • firewall.cisco
        • firewall.fortinet
        • firewall.huawei
        • firewall.juniper
        • firewall.meraki
        • firewall.paloalto
        • firewall.pfsense
        • firewall.sonicwall
        • firewall.sophos
        • firewall.stonegate
        • firewall.windows
      • Network
        • netstat.netflow
        • switch.cisco
      • Proxy
        • proxy.bluecoat
        • proxy.squid
      • Web
        • web.apache
        • web.apache.mod-security
        • web.iis
        • web.jboss
        • web.nginx
        • web.tomcat
      • Other technologies
        • db.mysql
        • dns.bind
        • edr.cylance
        • my.app
        • social.salesforce
        • test.drop
        • test.keep
        • uba.varonis
        • unknown.unknown
  • Data Search
    • Running a search
    • LINQ
      • Performing operations using LINQ
      • LINQ query examples
    • Viewing the data tables
    • Viewing column info
    • Running queries (tutorials)
    • Last queries
    • Query management
    • Lookup management
      • Dynamic lookups
    • Favorite queries
    • Sharing queries
    • Table toolbar features
      • Time interval history
      • Edit the column layout
      • View selected events
      • Column operations
      • Toggle query editor
      • Toggle search tree
      • New alert definition
      • Aggregations
      • Group
      • Filters
      • Create column
      • Download
    • Additional tools
      • Dashboard data source
      • Charts
        • Affinity chord diagram
        • Availability timeline
        • Bipartite chord diagram
        • Bubble chart
        • Chart aggregation
        • Custom date chart aggregation
        • Flame graph
        • Flat world map by coordinates
        • Flat world map by country
        • Google animated heat map
        • Google area map
        • Google heat map
        • Graph diagram
          • Creating a graph diagram
          • Graph diagram menu
        • Histogram
        • Pie chart
        • Pie layered chart
        • Punch card
        • Sankey diagram
        • Scatter plot
        • Time heatmap
        • Voronoi treemap
      • Graphical correlation
        • Cross-Search Graph Diagram
        • Cross-Search Table Join
        • Cross-Search Sankey Diagram
        • Cross-Search Line Chart
      • Query Info
      • Custom tables
    • Aliased finder
    • Custom finder
      • Creating a custom finder
      • Assigning a custom finder to a role
      • Edition mode
    • Data reinjection
    • Available operations
      • Examples
      • Aggregation operations
      • Filtering operations
        • Order group (filter)
        • String group (filter)
        • General group (filter)
        • Name group (filter)
        • Network group (filter)
        • Logic group (filter)
        • Web group (filter)
      • Create column operations
        • Order group
        • Arithmetic group
        • String group
        • General group
        • Date group
        • Name group
        • Network group
        • Geolocation group
        • Logic group
        • Flow group
        • Web group
        • Mathematical group
        • Conversion group
        • Cryptography group
        • Packet group
          • Ethernet operations
          • IPv4 operations
          • TCP operations
          • UDP operations
    • Best practices for data search
  • Alerts management
    • How to set up a delivery method
      • Setting up an Email type delivery method
      • Setting up an HTTP-JSON type delivery method
      • Setting up a Service Desk type delivery method
      • Setting up a Jira type delivery method
      • Setting up a Pushover type delivery method
      • Setting up a PagerDuty type delivery method
    • Alert policies
    • Alerts dashboard
    • Alerts and correlation libraries
    • How to?
      • Create a new alert
      • Set up an inactivity alert
      • Modify an existing alert
      • Deactivate an alert
      • Create an alert over an alert
      • Create an alert annotation
      • Create a post-filter
      • Edit a query alert
      • Quick filter saved alerts
  • Dashboards
    • Setup a data source
    • Create a new dashboard
    • Working with dashboard widgets
      • Availability timeline widget
      • Chord diagram widget
      • Circle world map widget
      • Color key value widget
      • Color world map widget
      • Column chart widget
      • Comparative chart widget
      • Funnel widget
      • Gauge meter widget
      • Google heatmap widget
      • Heat calendar widget
      • Line chart widget
        • Customize your line chart
      • Monitoring widget
      • Pie chart widget
      • Punch card widget
      • Sectored pie chart widget
      • Table widget
      • Time heatmap widget
      • Tree diagram widget
      • Voronoi tree widget
    • Configuring and sharing dashboards
  • API
    • REST API v2
      • Authorization methods
      • How to query with API v2
        • Forwarding query responses to HDFS
        • Forwarding query responses to Kafka
        • Forwarding query responses to S3
        • Send requests with Postman
      • Job requests
    • Provisioning API
    • OData API feeds management
      • Connecting with Excel
      • Connecting with Tableau
      • Connecting with Power BI
  • Use cases
    • Uploading, analyzing and visualizing data
      • Uploading data
      • Finding data
      • Structuring data
      • Converting data types
      • Performing data analysis
      • Creating a chart
      • Creating a dashboard data source
      • Adding a widget to a dashboard
    • Installing, configuring and uploading data with a Devo relay
      • Installing a Devo relay
      • Configuring and activating the Devo relay
      • Devo relay rules overview
      • Sending data to the Devo cloud through a relay
    • Monitor intranet traffic to dangerous websites
    • Kaspersky integration
    • Monitoring Oracle databases
      • Audit log
      • Alert log
      • Listener log
      • Incident log
      • Trace log
      • Inventory events
      • Performance
PREVIOUS
Reading a file
NEXT
Sending from an In-house Relay

System configuration / Sending the data / Sending the data using Logstash / Sending directly to Devo / Configuration file examples / Read data from Kafka

Download as PDF

Read data from Kafka

Logstash provides an input plugin that reads events from an Apache Kafka topic. 

  • Execute the following line to install Kafka:

    bin/logstash-plugin install logstash-input-kafka
  • Click here to download a configuration file example.

    input {
            kafka {
                    group_id => "test-kafka"
                    topics => ["testlogtrust"]
                    bootstrap_servers => "localhost:9092"
            }
    }
    output {
            syslog {
                    facility => "local7"
                    severity => "informational"
                    host => "us.elb.relay.logtrust.net"
                    port => "443"
                    appname => "my.app.kafka.topic"         #THIS IS WHERE YOU DEFINE THE TAG
                    protocol => "ssl-tcp"
                    ssl_cert => "/home/logtrust/davidcifl.crt"
                    ssl_key => "/home/logtrust/davidcifl.key"
                    ssl_cacert => "/home/logtrust/chainl.crt"
            }
            stdout{codec => rubydebug }
    }
Download as PDF

Did you find what you were looking for?

If not, please let us know what you need. Your feedback will help us to improve.

PREVIOUS
Reading a file
NEXT
Sending from an In-house Relay

Export

See what Devo can do for you. Request a demo!
Discover what's new (Release notes)
  • Services & Support
  • Devo.com
  • Contact
    • Contact Us
    • Request a Demo
    • Partner Inquiry
  • Log In
    • USA Devo
    • EU Devo
  • +1 888 6830910 (USA)
  • +34 900 838 880 (Spain)
Copyright © 2019 Legal Terms Privacy Policy Cookies Policy

Powered by Confluence and Scroll Viewport