Solved

Proc-centengine service, return values > 1

  • 23 November 2021
  • 19 replies
  • 845 views

Userlevel 1
Badge +6

Hello everybody,

 

I’m back in the company where I did install a Centreon. As nobody touched it since I left, it need some ‘refreshing’

 

I have an issue with the service mesuring the number of process for centengine. The value is changing all the time

When I check it manually (with ps) I only see one process.

[root@s-sup ~]# ps -aux | grep centengineroot 2271 0.0 0.0 112680 980 pts/0 S+ 14:28 0:00 grep --color=auto centenginecentreo+ 14933 0.9 0.1 739368 28612 ? Ssl nov.19 43:57 /usr/sbin/centengine /etc/centreon-engine/centengine.cfg[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XIZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1'CRITICAL: Number of current processes running: 9 | 'nbproc'=9;;1:1;0;

 

Is my plugin wrong or do I have up to 14 centengine process ?

I’m currently running Centreon 2.8.32 (have to update it, but i got some ‘issues’ each time i do it for now^^)

Linux snmp plugin is up to date (i did update all my plugin pack)

[root@s-sup ~]# rpm -qa | grep centreon-plugin-Operatingsystems-Linux-Snmpcentreon-plugin-Operatingsystems-Linux-Snmp-20210915-070550.el7.centos.noarch

 

Thank you for your feedbacks ;)

icon

Best answer by sims24 29 November 2021, 12:01

View original

19 replies

Userlevel 6
Badge +18

Hi @Benjamin_b, you should have only one centengine process.

 

May be an issue with an old version of centengine

Userlevel 1
Badge +6

Well I’ll wait until I update to last version to see if it correct the issue(s). if I manage to do the update without errors hehe

 

Thank you Laurent

Userlevel 4
Badge +13

Hi @Benjamin_b 
 

Can you test to query the OIDs used by this plugin to know if a process is running or not?

First, can you query the list of processes and get the index value?

snmpwalk -v2c -c public 127.0.0.1 .1.3.6.1.2.1.25.4.2.1.2 | grep -i centengine
HOST-RESOURCES-MIB::hrSWRunName.22867 = STRING: "centengine"

Here the index is 22867.

Now, can you query the status replacing the index value?

snmpwalk -v2c -c public 127.0.0.1 .1.3.6.1.2.1.25.4.2.1.7.22867
HOST-RESOURCES-MIB::hrSWRunStatus.22867 = INTEGER: runnable(2)

 

Userlevel 1
Badge +6

Hi, thank for your answer @sduret 

 

Here is the status :

 

HOST-RESOURCES-MIB::hrSWRunStatus.21900 = INTEGER: runnable(2)

(at this moment plugin says 8 running process)

Userlevel 4
Badge +13

@Benjamin_b 
Is it possible to send me this snmpalk output? Maybe it’s possible to specify on the path and arguments to filter on the good process

snmpwalk -v2c -c public 127.0.0.1 .1.3.6.1.2.1.25.4.2.1 > /tmp/walk-process.txt

If it’s not possible to send the file, can you send the output of this command?

snmpwalk -v2c -c public 127.0.0.1 .1.3.6.1.2.1.25.4.2.1 | grep -i engine

 

Userlevel 1
Badge +6

@sduret 

 

Here is the result of

snmpwalk -v2c -c XYZ 127.0.0.1 .1.3.6.1.2.1.25.4.2.1 |grep -i engine > /tmp/walk-process.txt
cat /tmp/walk-process.txt

HOST-RESOURCES-MIB::hrSWRunName.21900 = STRING: "centengine"
HOST-RESOURCES-MIB::hrSWRunPath.21900 = STRING: "/usr/sbin/centengine"
HOST-RESOURCES-MIB::hrSWRunParameters.21900 = STRING: "/etc/centreon-engine/centengine.cfg"

Thx

Userlevel 4
Badge +13

@Benjamin_b It’s weird because you’ve only one centengine process.
Maybe it’s the same PID…

Can you execute these two commands and send the result?

/usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='public' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug

/usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='public' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug --process-status='running|runnable'

 

Userlevel 1
Badge +6

@sduret 

There the results :

OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
Process '21900' [status: runnable] [name: centengine]
OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
Process '21900' [status: runnable] [name: centengine]

At this moment the plugin says :

CRITICAL: Number of current processes running: 9

Userlevel 6
Badge +19

Most likely, your service definition doesn’t include proper macro configuration. Compare the command provided by @sduret with the one displayed here:

 

 

Userlevel 1
Badge +6

Thanks for your answer @sims24 

 

Here is my macro configuration for this service :

 

I see no real differences between his command, mine and yours. (except for the --debug and --process-status)

 

Userlevel 6
Badge +19

Could you try to add --process-status='running|runnable' in the EXTRAOPTIONS macro please?

It very weird, no reasons to get such difference between UI and CLI execution. What about the CLI in the UI, looks line the one on my screenshot?

Userlevel 1
Badge +6

/usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='xyz'
    --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1'

 

Here is the command as shown in the UI.

Userlevel 1
Badge +6
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XTZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1'
OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug --process-status='running|runnable'
OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
Process '21900' [status: runnable] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug
OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
Process '21900' [status: runnable] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug
CRITICAL: Number of current processes running: 8 | 'nbproc'=8;;1:1;0;
Process '5679' [status: running] [name: centengine]
Process '5684' [status: running] [name: centengine]
Process '5675' [status: running] [name: centengine]
Process '21900' [status: runnable] [name: centengine]
Process '5686' [status: running] [name: centengine]
Process '5685' [status: running] [name: centengine]
Process '5683' [status: running] [name: centengine]
Process '5687' [status: running] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug --process-status='running|runnable'
CRITICAL: Number of current processes running: 8 | 'nbproc'=8;;1:1;0;
Process '5679' [status: running] [name: centengine]
Process '5684' [status: running] [name: centengine]
Process '5675' [status: running] [name: centengine]
Process '21900' [status: runnable] [name: centengine]
Process '5686' [status: running] [name: centengine]
Process '5685' [status: running] [name: centengine]
Process '5683' [status: running] [name: centengine]
Process '5687' [status: running] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1'
CRITICAL: Number of current processes running: 8 | 'nbproc'=8;;1:1;0;
[root@s-sup ~]#

I typed his two commands and mine with 5 minutes delay

Userlevel 6
Badge +19

Ok, an idea came to my mind during the weekend. Do you monitor centengine on Pollers from this server? 

I guess that while the command is running, other centengine process checks are running as well and are erroneously checked as an engine process. 

 

Using --process-path='/usr/sbin/' will avoid this. You can use the PROCESSPATH macro to set the value ‘/usr/sbin/’

 


 

Userlevel 1
Badge +6

Sims24, I’m not sure to fully understand your question.

I have only one server, no remote pollers.

I’ll update the command with your suggestion.

 

Thx

Userlevel 6
Badge +19
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XTZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1'
OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug --process-status='running|runnable'
OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
Process '21900' [status: runnable] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug
OK: Number of current processes running: 1 | 'nbproc'=1;;1:1;0;
Process '21900' [status: runnable] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug
CRITICAL: Number of current processes running: 8 | 'nbproc'=8;;1:1;0;
Process '5679' [status: running] [name: centengine]
Process '5684' [status: running] [name: centengine]
Process '5675' [status: running] [name: centengine]
Process '21900' [status: runnable] [name: centengine]
Process '5686' [status: running] [name: centengine]
Process '5685' [status: running] [name: centengine]
Process '5683' [status: running] [name: centengine]
Process '5687' [status: running] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1' --debug --process-status='running|runnable'
CRITICAL: Number of current processes running: 8 | 'nbproc'=8;;1:1;0;
Process '5679' [status: running] [name: centengine]
Process '5684' [status: running] [name: centengine]
Process '5675' [status: running] [name: centengine]
Process '21900' [status: runnable] [name: centengine]
Process '5686' [status: running] [name: centengine]
Process '5685' [status: running] [name: centengine]
Process '5683' [status: running] [name: centengine]
Process '5687' [status: running] [name: centengine]
[root@s-sup ~]# /usr/lib/centreon/plugins/centreon_linux_snmp.pl --plugin=os::linux::snmp::plugin --mode=processcount --hostname=127.0.0.1 --snmp-version='2c' --snmp-community='XYZ' --process-name='centengine' --process-path='' --process-args='' --regexp-name --regexp-path --regexp-args --warning='' --critical='1:1'
CRITICAL: Number of current processes running: 8 | 'nbproc'=8;;1:1;0;
[root@s-sup ~]#

I typed his two commands and mine with 5 minutes delay

Nevermind, it was inaccurate anyway! 
What happened is that your CLI checks (above) got added to the SNMP process table. Without additional filtering on the process path, it erroneously matches `/usr/lib/centreon/plugins/[….] --process-name=’centengine’ [...]`. 

The SNMP process table gets refreshed every minute. That explains that you could feel like the process check result might be random (sometimes 1, sometimes more than 1)

 

Userlevel 1
Badge +6

I wanted to click on Like and I clicked on ‘Best answer’. Woopsie ^^

Anyway I did the change when I answered you and until now no more errors :)

Userlevel 1
Badge +6

Well, no errors in 24h since the change.

I think it‘s time to say : Issue resolved.

 

Thank you @sims24 

 

Now, time to move to another challenge > updating my 2.8.32 Centreon hehe

Userlevel 6
Badge +19

@Benjamin_b happy to hear that! We will set this value as a default in the template, so thank you for confirming that it works as expected. 

Oh, I will probably write a post soon about Centreon experts best practices to upgrade old centreon platforms. But, I’m not sure it will be ready before christmas; :upside_down:

Reply