In this article, we will see how to reduce disk I/O usage upon updating performance and status charts (.rrd files). To do that, RRDCacheD is mutualizing data writing instead of doing so one at a time.
Installation
RRDCacheD is automatically installed with the RRDtool RPM. If it is not the case, you can install it using the following command:
yum install rrdtool-cached
Configuration
Create the RRDCacheD user and create its directory:
mkdir -p /var/rrdtool/rrdcached/
useradd rrdcached -d '/var/rrdtool/rrdcached' -G centreon-broker,centreon -m
chown -R rrdcached: /var/rrdtool
chmod 775 -R /var/rrdtoolAdd the users below to the following groups:
usermod -a -G rrdcached centreon-broker
usermod -a -G rrdcached apache
usermod -a -G centreon rrdcached
usermod -a -G centreon-broker rrdcachedCreate and edit the file /etc/systemd/system/rrdcached.service:
cat > /etc/systemd/system/rrdcached.service <<EOF
eUnit]
Description=Data caching daemon for rrdtool
eService]
Type=forking
User=rrdcached
PIDFile=/var/rrdtool/rrdcached/rrdcached.pid
ExecStart=/usr/bin/rrdcached -m 775 -s rrdcached -l unix:/var/rrdtool/rrdcached/rrdcached.sock -p /var/rrdtool/rrdcached/rrdcached.pid -b /var/rrdtool/rrdcached -w 3600 -z 3600 -f 7200
/Install]
WantedBy=default.target
EOF
The order of the options is very important. If the -m 755 is placed after the -l unix: /var/rrdtool/rrdcached/rrdcached.sock then the socket will be created with the wrong rights.
Options are as follows:
| Option | Description |
|---|---|
| -w | Data are written every x seconds on disk (for instance, 3600s represent 1h) |
| -z | Should be less than -w option. RRDCacheD uses a range value from n0:-z] to do not write in RRDs at the same time. |
| -f | Timeout in cache before write data to disk. |
The option's values must fit to the need/constraints of your platform.
Apply the changes :
systemctl daemon-reloadEnable RRDCacheD to be started on startup:
systemctl enable rrdcachedRestart Apache and start RRDCacheD:
systemctl restart httpd24-httpd
systemctl start rrdcached
RRDCacheD configuration in Centreon web interface
Go to the Configuration>Pollers>Broker configuration menu and edit the configuration by enabling RRDCacheD and adding /var/rrdtool/rrdcached/rrdcached.sock to the “RRDCacheD listening socket/port” input box:
The Socket path must be /var/rrdtool/rrdcached/rrdcached.sock to be consistent with the configuration mentioned in the configuration file.
Go to the Configuration>Pollers menu. Export your configuration to restart centreon-broker:
systemctl restart cbd
Is it working?
Check that the pid, socket, and rights are in /var/rrdtool/rrdcached/:
iroot@central ~]$ ll /var/rrdtool/rrdcached/
total 4
-rw-r--r-- 1 rrdcached rrdcached 4 Feb 21 10:24 rrdcached.pid
srwxrwxr-x 1 rrdcached rrdcached 0 Feb 21 10:24 rrdcached.sock
The implementation of RRDCacheD implies that charts are not updated in real time, so it is possible to see a small blank on some charts. This means that the data is still in the daemon’s cache: this is a normal behavior.
If the daemon crashes for any reason, data will be lost and there will be no way to get them back except by rebuilding the graphs with Centreon-Broker.
In case of any issue, errors will be logged in the Broker RRD logs file in /var/log/centreon-broker/:
# RRDCached is stopped
error: RRD: error while getting response from rrdcached: QLocalSocket: Remote closed
# Error in the socket name/configuration (check the information in /etc/sysconfig/rrdcached or /etc/systemd/system/rrdcached.service)
error: RRD: could not connect to local socket '/var/rrdtool/rrdcached/rrdcached.sock: QLocalSocket::connectToServer: Invalid name
# Error in the acces of the socket (problably a problem of rights, groups, mode...)
error: RRD: could not connect to local socket '/var/rrdtool/rrdcached/rrdcached.sock: QLocalSocket::connectToServer: Socket access errorIf the RRDCacheD process crashes, all the data from the cache would be lost!
Migration
After migrating a platform on which rrdcache was operational, there are a couple of things to check to confirm good operation of rrdcache and avoid issues in the future.
- Check the umask of the centreon-broker user on the new platform :
-
su -c 'umask' -l centreon-brokerIf the result is 0002, you can skip the rest of the steps.
-
- Make sure that the rights of the folders listed below are 755 :
- /var/lib/centreon/metrics
- /var/lib/centreon/status
- Make sure that the .rrd files inside the previously mentioned folders are 664.