Skip to main content

Hi,
We recently upgraded our centreon infra from 24.04.12 to 24.10.7.
Our mariadb server (On a standalone redhat serveur) is version 10.11.10 

Just after the upgrade we noticed that the CPU_LOAD on the database serveur is increasing pemanently and regulary … 
Now 100% CPU is reached … an the top report : load average: 11.56, 9.75, 9.59
(Usually it is below 1,5)

We have 4vcpu on the redhat server.
After enabling the lonq_query logging I noticed that I have between 1 an 14 similar requests running in PARALLEL :

SELECT contact.*, cp.password AS contact_passwd, t.topology_url,
t.topology_url_opt, t.is_react, t.topology_id, tz.timezone_name, t.topology_page as default_page
FROM `centreon`.contact
LEFT JOIN `centreon`.contact as template
    ON contact.contact_template_id = template.contact_id
LEFT JOIN `centreon`.contact_password cp
    ON cp.contact_id = contact.contact_id
LEFT JOIN `centreon`.timezone tz
    ON tz.timezone_id = contact.contact_location
LEFT JOIN `centreon`.topology t
    ON t.topology_page = COALESCE(contact.default_page, template.default_page)
INNER JOIN `centreon`.security_authentication_tokens sat
    ON sat.user_id = contact.contact_id
INNER JOIN `centreon`.security_token st
    ON st.id = sat.provider_token_id
WHERE (sat.token = 'XXXX' OR st.token = '4YYYY')
    AND sat.is_revoked = 0
ORDER BY cp.creation_date DESC LIMIT 1;


The time execution is around 2 seconds for each SELECT.
This SELECT is always here …. (betwwn 1 and 14 instances)

Not sure if it is the root cause of the CPU usage …..

Centron is still running well , but I am afrais as it may crash …


I restarted everything : gorgoned and cbd , php-fpm abd httpd on the Frontend.
I also restarted the mariadb linux server….. as soon as the database is started th CPU LOAD is very High.

Any Idea ?
 


NOTE :
On the frontend if i do a “netstat -anp”, I can see 19 sockets established with the database server :
EX:
tcp        0      0 YYYY:56186       XXXX:3306       ESTABLISHED 1568178/cbd
tcp        0      0 YYYY:58352       XXXX:3306       ESTABLISHED 257494/php-fpm: poo

The netstat also shows permanently between 700 and 900 TIME_WAIT sockets: 
EX: 
tcp        0      0 YYYY:37282       XXXX:3306       TIME_WAIT   -

Just noticed that the CPU_LOAD issue on teh datbase is due to this php script on the Central :
/usr/share/centreon/src/Centreon/Infrastructure/Contact/ContactRepositoryRDB.php

If I rename (temporaly) this file : the CPU load immediately decreasea to a normal value.
if I set back the name … it immediately increases again …

And I get this log :
Trying to access array offset on value of type null {"exception":"iobject] (ErrorException(code: 0): Warning: Trying to access array offset on value of type null at /usr/share/centreon/src/Centreon/Infrastructure/Contact/ContactRepositoryRDB.php:317)"

Th bloc around line 317 is :

315                 }
316                 if (strlen((string) $topologyPage) === 3) {
317                     $lvl2Name = $topologies$topologyPage]a'name'];
318                     $topologyPage = (int) substr((string) $topologyPage, 0, 1);
319                 }


…. but still don’t know how to fix it ...


Reply