Sending data to Devo / Event sources / Unix-like machines / Third-party syslog tools configuration / rsyslog / Obsolete legacy format / Monitoring files using rsyslog (Obsolete legacy format)
Monitoring files using rsyslog (Obsolete legacy format)
The instructions for setting up connections to a Devo Relay or Cloud require the creation of the 00-devo.conf and 49-devo.conf files and cover the sending of all system log events to the Devo endpoint.
This article explains how to use an additional /etc/rsyslog.d/*.conf file to send events from other logs on the host to the same Devo endpoint. You may create multiple .conf files depending on the variety of log files you need to send. Each file can include up to 100 different input files to monitor.
Create the configuration file
For each log file that you want to monitor on the host, you will include a dedicated set of input parameters in the configuration file. These parameters are part of the imfile module loaded in 00-devo.conf. The sample below contains placeholder values:
/etc/rsyslog.d/45-filemonitor.conf file monitoring example
$template myFileMonitorTemplate,"<%PRI%>%timegenerated% %HOSTNAME% %syslogtag% %msg%"
# File access
$InputFileName /path/to/file.log # the file being monitored
$InputFileTag my.devo.tag: # tag to assign to messages from the InputFileName
$InputFileStateFile stat-file1-myFileMonitor # defines name of state files
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
if $syslogtag contains 'my.devo.tag' and $syslogfacility-text == 'local7' then @@DEVO-RELAY:PORT;myFileMonitorTemplate
:syslogtag, contains, "my.devo.tag" ~These are the parameters we recommend using to configure each input file.
| 1 | $InputFileName | The path and filename of the file to monitor. |
|---|---|---|
| 2 | $InputFileTag | The tag to assign to messages that come from the monitored file. |
| 3 | $InputFileStateFile | The name of the state file that will be saved in the $WorkingDirectory (specified in 00-devo.conf). The state file keeps a history of the messages that were more recently processed. In the case of a rsyslog crash or reboot, this file will be checked in order to know where to resume message processing. |
| 4 | $InputFileSeverity / $InputFileFacility | The syslog severity and facility to assign to messages from the input file. |
| 5 | $InputFilePollInterval | The interval in seconds between checking the file for new messages. |
| 6 | $InputFilePersistStateInterval | The number of events after which the processing status should be updated. This should be set to 1. |
| 7 | If $syslogtag contains 'my.devo.tag' … | This line instructs rsyslog to send all incoming events with the ‘my.devo.tag’ tag via TCP to ‘DEVO-RELAY:PORT’ using the log format defined in ‘myFileMonitorTemplate’ in the configuration file header. |
| 8 | :syslogtag, contains, 'my.devo.tag.' ~ | This line removes messages with the my.devo.tag from the rsyslog processing chain so that they will not be re-processed by any other configuration files. |
The following configuration file shows how to monitor multiple Apache server log files and send them to a Devo Relay. Note that this file is not set up with SSL/TLS security.
File /etc/rsyslog.d/45-apache.conf monitoring example
$template apache,"<%PRI%>%timegenerated% %HOSTNAME% %syslogtag% %msg%"
# Define the input of access.log
$InputFileName /var/log/apache2/access.log
$InputFileTag web.apache.access-combined.pro.webFoobar.www1:
$InputFileStateFile stat-file1-ApacheAccess
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
# Define the input of ssl_access.log
$InputFileName /var/log/apache2/ssl_access.log
$InputFileTag web.apache.access-combined.pro.webFoobar-ssl.www1:
$InputFileStateFile stat-file2-ApacheAccess
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
# Define the input of error.log
$InputFileName /var/log/apache2/error.log
$InputFileTag web.apache.error.pro.webFoobar.www1:
$InputFileStateFile stat-file1-ApacheError
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
# Define the input of ssl_error.log
$InputFileName /var/log/apache2/ssl_error.log
$InputFileTag web.apache.error.pro.webFoobar-ssl.www1:
$InputFileStateFile stat-file2-ApacheError
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 1
$InputFilePersistStateInterval 1
$InputRunFileMonitor
if $syslogtag contains 'web.apache.' and $syslogfacility-text == 'local7' then @@DEVO-RELAY:PORT;apache
:syslogtag, contains, "web.apache." ~Ensure the required permissions are enabled
The user running rsyslog, usually syslog, must have permissions over all directories and files that you want to monitor. Here's an example of how you might change the owner:
chown :syslog /var/log/apache2 /var/log/apache2/*.log
If the input files are part of a logrotate policy and logrotate create mode is being used, ensure that syslog will still have permissions over the new file. For example, in Apache:
/etc/logrotate.d/apache2 configuration file extract
/var/log/apache2/*.log {
...
create 640 root syslog
...Restart rsyslog
Close the configuration file, then restart rsyslog.
/etc/init.d/rsyslog restart
Once restarted, go to Data Search in the Devo web application and look for the table that contains the events with the tag you just set up.