The following procedure describes how to configure coredumpctl to retrieve core dumps if a segfault happens.
Generate a core dump during a segfault
Activate core dump generation
Before retrieving a core dump, you must configure the system for it to generate core dumps at segfault.
The first step consists in installing gdb.
On Centos7:
yum install devtoolset-9-gdb
ln -s /opt/rh/devtoolset-9/root/bin/{gdb,gcore,gdb-add-index,gstack,pstack} /usr/bin/
For other distributions, use your OS software manager, for examples:
yum install gdb
or
apt install gdb
Once this step done, you have to change several things in your configuration:
- Uncomment "DumpCore=yes" in "/etc/systemd/system.conf"
- Replace "DefaultLimitCORE=" by "DefaultLimitCORE=infinity" in "/etc/systemd/system.conf"
- Add these lines in "/etc/sysctl.conf":
kernel.core_pattern = |/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e
fs.suid_dumpable=2 - Add "LimitCORE=infinity" in "/usr/lib/systemd/system/cbd.service" just after the line "User"
- Add "LimitCORE=infinity" in "/usr/lib/systemd/system/centengine.service" just after the line "User"
Restart systemd to commit the changes:
sysctl -p
systemctl daemon-reexec
Retrieve a coredump
After activating core dumps generation, you must wait for a segfault to occur. You will see them in the system's log /var/log/messages
.
- List the core dumps:
coredumpctl list
- Retrieve the core dump:
coredumpctl -o filename.core dump /usr/sbin/centengine
This command will store the last coredump of /usr/sbin/centengine
to a file named filename.core
.
You can do the same with /usr/sbin/cbd
.
Generate a coredump manually
Sometimes, it can be interesting to generate a coredump manually. For example if centreon-engine looks blocked but not crashed.
The procedure for that is to install gdb on the server.
On Centos7: (the command for the installation and the one to update the path)
yum install devtoolset-9-gdb
source /opt/rh/devtoolset-9/enable
On other distributions, use the package manager, for example:
yum install gdb
or apt install gdb
Then the generation is made in two steps:
-
Get the PID of the program you want a coredump ; here is an example with centengine:
ps ax | grep centengine
2272 ? Ssl 33:43 /usr/sbin/centengine /etc/centreon-engine/centengine.cfg -
Generate the coredump:
gcore 2272
where 2272 is the PID you get at the step 1.
In the current directory, you should find a new file
core.2272
that is the coredump.
-
You can now send the coredump to the support of Centreon for more investigation.