Logfile Cache

Accessing logfiles can be time and memory consuming, especially when having remote sites connected to Thruk. Livestatus caches logfile data used for queries. This increases the size of the naemon process, slows down the monitoring and increases the overall load on your monitoring host. Therefor Thruk may use a logfile cache to increase performance of logfile access and reports.

The MongoDB logcache is deprecated and got removed from Thruk in 2016. Use the MySQL logcache instead.

For maximum performance, the Logfile Cache Database should be installed on the same server as Thruk itself (or at least in the same network).

Logfile Cache Architecture

Logfile Cache Architecture

Configuration

Starting with OMD-Labs 2.40 you can simply run omd config set MYSQL on; omd config set THRUK_LOGCACHE on to enable the logcache.

You will need to specify your mysql connection configuration in your thruk_local.conf. Make sure the user and database exists. The tables will then be created automatically.

thruk_local.conf:

  logcache = mysql://username:password@localhost:3306/thruk_logs
create mysql user and tables
#> mysql -u root -p
mysql> CREATE USER 'thruk'@'localhost' IDENTIFIED BY '****';
mysql> CREATE DATABASE IF NOT EXISTS `thruk_logs`;
mysql> GRANT ALL PRIVILEGES ON `thruk_logs`.* TO 'thruk'@'localhost';

Actions

Import

It’s a good idea to import the logfiles after enabling the logfilecache:

Initial import:

    %> thruk -a logcacheimport --local
    ...
    OK - imported 20374 log items from 1 site successfully in 4.94s (4122/s)

Be careful, the 'logcacheimport' deletes all current cached data, so only do that once.

Depending on the size, this may take a while.

Update

Logfile Cache Update

    %> thruk -a logcacheupdate
    OK - imported 3 log items from 3 sites successfully in 0.13s (23/s)

The logcacheupdate imports the delta since the last update. That’s pretty fast and done automatically whenever you access logfiles via Thruk. However, if you access the eventlog page only once in a while, it would be a good idea to update the logcache periodically by a cronjob.

Import from Logfile Archive Files

The MySQL logcache supports import from files too. Duplicates are not imported twice, so you can do that safely as often as you like.

    %> thruk -a logcacheupdate --local /var/log/naemon/archive/naemon-09-07-2012-00.log
    OK - imported 2 log items from 1 site successfully in 0.01s (247/s)

Authentication Update

The cache needs authentication information which has to be updated whenever contacts have changed.

Update logfile caches authentication information:

    %> thruk -a logcacheauthupdate
    OK - updated 71 log items from 3 sites successfully in 0.25s (286/s)

Cleanup

You may want to remove old logfile entries from time to time. This is possible with the logcacheclean command. Optionally you can define the number of days which should be kept in the cache.

    %> thruk -a logcacheclean=730
    OK - removed 18046 log items from 1 site successfully in 0.08s (218816/s)

Statistics

There is also a command to check the current usage statistics of your logfile cache:

    %> thruk -a logcachestats
    Backend              Index Size      Data Size       Items
    Devel                 14.4 MB        136.4 MB       238104

Example Cronjobs

Using the logfile cache needs some extra maintainance to work smoothly, the following cronjobs should cover that. When not using OMD, you will have to adjust the paths.

  # update logcache data
  */5 * * * * $OMD_ROOT/bin/thruk -a logcacheupdate     >>$OMD_ROOT/var/log/thruk.log 2>&1

  # update authentication data in our logcache
  0   2 * * * $OMD_ROOT/bin/thruk -a logcacheauthupdate >>$OMD_ROOT/var/log/thruk.log 2>&1

  # optimize and repair tables
  0  20 * * * $OMD_ROOT/bin/thruk -a logcacheoptimize   >>$OMD_ROOT/var/log/thruk.log 2>&1

  # clean logfiles from cache after two years
  0   1 * * * $OMD_ROOT/bin/thruk -a logcacheclean=730  >>$OMD_ROOT/var/log/thruk.log 2>&1

Icinga 2

When using Icinga 2 it is highly recommended to use the logcache with a file import because the Livestatus API returns wrong results for the logs table in Icinga 2.

Therefor we recommend the following settings (OMD-Labs):

etc/thruk/thruk_local.d/logcache.conf:

  logcache = mysql://root@tmp/run/mysqld/mysqld.sock/thruk_logs
  logcache_delta_updates = 0
  report_update_logcache = 0

This disables on-demand updates which won’t work.

Then use a slightly changed crontab which imports the logfiles directly from the compat logfiles.

etc/cron.d/thruk_logcache

  # update logcache data
  * * * * * $OMD_ROOT/bin/thruk -a logcacheupdate var/icinga2/log/icinga2/compat/icinga.log >>$OMD_ROOT/var/log/thruk.log 2>&1
  5 0 * * * $OMD_ROOT/bin/thruk -a logcacheupdate $(ls -1tr var/icinga2/log/icinga2/compat/archives/*.log) >>$OMD_ROOT/var/log/thruk.log 2>&1

  # update authentication data in our logcache
  0   2 * * * $OMD_ROOT/bin/thruk -a logcacheauthupdate >>$OMD_ROOT/var/log/thruk.log 2>&1

  # optimize and repair tables
  0  20 * * * $OMD_ROOT/bin/thruk -a logcacheoptimize   >>$OMD_ROOT/var/log/thruk.log 2>&1

  # clean logfiles from cache after two years
  0   1 * * * $OMD_ROOT/bin/thruk -a logcacheclean=730  >>$OMD_ROOT/var/log/thruk.log 2>&1
Edit page on GitHub