Solved

Centreon 22.04.6 Charts are not generating


Userlevel 1
Badge +6

Hi, 

Yesterday I upgraded from version 21.10.11 to version 22.04.6
I noticed that no charts have been generated since yesterday. What could be the reason?
Thanks for any help

 

 

icon

Best answer by Laurent 4 October 2022, 17:49

View original

22 replies

Userlevel 6
Badge +18

Hi @centreon_s did you restart cbd process since you upgraded?

Do you have errors in /var/log/centreon-broker/*.log files?

Regards,

Userlevel 1
Badge +6

Thank you for your response

Yes I restartet all services

Currently, all services (cbd.service centengine.service centreon.service centreontrapd.service gorgoned.service) are active

 

 

last part of files 

ls -ltr /var/log/centreon-broker/

-rw-rw-r-- 1 centreon-broker centreon-broker    1417 08-30 03:19 central-broker-master.log-20220830.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 152 09-04 03:08 watchdog.log-20220904.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 1108 09-04 03:08 central-rrd-master.log-20220904.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 631 09-04 03:08 central-module-master.log-20220904.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 1194 09-04 03:08 central-broker-master.log-20220904.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 426 09-11 03:21 central-rrd-master.log-20220911.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 566 09-11 03:21 central-broker-master.log-20220911.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 317 09-20 03:50 central-rrd-master.log-20220920.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 337 09-20 03:50 central-broker-master.log-20220920.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 187 09-23 03:42 watchdog.log-20220923.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 838 09-23 03:42 central-module-master.log-20220923.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 1960 09-25 03:24 central-rrd-master.log-20220925.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 2664 09-25 03:24 central-broker-master.log-20220925.gz
-rw-rw-r-- 1 centreon-broker centreon-broker 9070 09-30 03:07 central-module-master.log-20220930
-rw-rw-r-- 1 centreon-broker centreon-broker 1498 09-30 03:07 watchdog.log-20220930
-rw-rw-r-- 1 centreon-broker centreon-broker 26467 10-02 03:50 central-broker-master.log-20221002
-rw-rw-r-- 1 centreon-broker centreon-broker 20549 10-02 03:50 central-rrd-master.log-20221002
-rw-rw-r-- 1 centreon-broker centreon-broker 1508 10-04 11:12 watchdog.log
-rw-rw-r-- 1 centreon-broker centreon-broker 26634 10-04 11:53 central-rrd-master.log
-rw-rw-r-- 1 centreon-broker centreon-broker 34144 10-04 11:53 central-broker-master.log
-rw-rw-r-- 1 centreon-broker centreon-broker 17685 10-04 11:53 central-module-master.log

 

$ tail -f /var/log/centreon-broker/central-rrd-master.log
[2022-10-04T11:50:52.829+02:00] [core] [info] main: configuration update requested
[2022-10-04T11:50:52.830+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/50-tcp.so' which is already loaded
[2022-10-04T11:50:52.830+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/70-rrd.so' which is already loaded
[2022-10-04T11:50:52.830+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/15-stats.so' which is already loaded
[2022-10-04T11:50:52.830+02:00] [core] [info] multiplexing: engine started
[2022-10-04T11:53:22.694+02:00] [core] [info] main: configuration update requested
[2022-10-04T11:53:22.695+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/50-tcp.so' which is already loaded
[2022-10-04T11:53:22.695+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/70-rrd.so' which is already loaded
[2022-10-04T11:53:22.695+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/15-stats.so' which is already loaded
[2022-10-04T11:53:22.695+02:00] [core] [info] multiplexing: engine started

 

$ tail -f /var/log/centreon-broker/central-broker-master.log
[2022-10-04T11:53:22.497+02:00] [core] [info] feeder: thread of client 'central-broker-master-input-1' will exit
[2022-10-04T11:53:22.694+02:00] [core] [info] main: configuration update requested
[2022-10-04T11:53:22.695+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/50-tcp.so' which is already loaded
[2022-10-04T11:53:22.695+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/20-unified_sql.so' which is already loaded
[2022-10-04T11:53:22.695+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/15-stats.so' which is already loaded
[2022-10-04T11:53:22.695+02:00] [core] [info] multiplexing: engine started
[2022-10-04T11:53:22.845+02:00] [core] [info] New incoming connection 'central-broker-master-input-2'
[2022-10-04T11:53:22.845+02:00] [core] [info] multiplexing: 'central-broker-master-input-2' starts with 0 in queue and the queue file is disable
[2022-10-04T11:53:22.845+02:00] [core] [info] removing 'central-broker-master-input-1' from acceptor 'central-broker-master-input'
[2022-10-04T11:53:22.845+02:00] [core] [info] Destroying muxer central-broker-master-input-1: number of events in the queue: 0

 

 

$ tail -f /var/log/centreon-broker/central-module-master.log
[2022-10-04T11:53:22.779+02:00] [core] [info] NEB: module for Centreon Broker 22.04.1
[2022-10-04T11:53:22.779+02:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/60-tls.so'
[2022-10-04T11:53:22.835+02:00] [core] [info] protocols: registering protocol ('TLS' (layers 5-5)
[2022-10-04T11:53:22.835+02:00] [core] [info] protocols: registering protocol ('TCP' (layers 1-4)
[2022-10-04T11:53:22.835+02:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/15-stats.so'
[2022-10-04T11:53:22.842+02:00] [core] [info] multiplexing: 'central-module-master-output' starts with 0 in queue and the queue file is disable
[2022-10-04T11:53:22.842+02:00] [core] [info] multiplexing: engine started
[2022-10-04T11:53:22.897+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/50-tcp.so' which is already loaded
[2022-10-04T11:53:22.897+02:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/15-stats.so' which is already loaded
[2022-10-04T11:53:22.897+02:00] [core] [info] multiplexing: engine started

 

 

$ tail -f /var/log/centreon-broker/watchdog.log
[2022-10-03T16:53:40.967+02:00] [cbwd] [info] Stopping process 'central-broker-master' (PID 17963)
[2022-10-03T16:53:40.967+02:00] [cbwd] [info] Process 'central-broker-master' (PID 17963) stopped gracefully
[2022-10-03T17:12:05.387+02:00] [cbwd] [info] Starting progress 'central-broker-master'
[2022-10-03T17:12:05.388+02:00] [cbwd] [info] Process 'central-broker-master' started (PID 29935)
[2022-10-03T17:12:05.388+02:00] [cbwd] [info] Starting progress 'central-rrd-master'
[2022-10-03T17:12:05.388+02:00] [cbwd] [info] Process 'central-rrd-master' started (PID 29936)
[2022-10-04T11:12:38.650+02:00] [cbwd] [info] Starting progress 'central-broker-master'
[2022-10-04T11:12:38.650+02:00] [cbwd] [info] Process 'central-broker-master' started (PID 13634)
[2022-10-04T11:12:38.650+02:00] [cbwd] [info] Starting progress 'central-rrd-master'
[2022-10-04T11:12:38.650+02:00] [cbwd] [info] Process 'central-rrd-master' started (PID 13635)

 

Userlevel 6
Badge +18

Do you use RRDCacheD?

Files are updated in /var/lib/centreon/metrics/ directory?

Userlevel 1
Badge +6

All files stopped at the time of yesterday's update. I'm just showing one last part

-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7803.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7802.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7801.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7800.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7799.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7798.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7797.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7796.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7795.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7794.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7793.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7792.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7791.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7790.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7789.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker 450088 10-03 18:19 7788.rrd

 

 

Userlevel 1
Badge +6

Do you use RRDCacheD?

I do not know what's going on?

Userlevel 6
Badge +18

the restart of cbd (indicated in /var/log/centreon-broker/watchdog.log) was because you generate and export configuration?

 

can you give me the result of this command?

cat /var/lib/centreon-broker/central-rrd-master-stats.json

*maybe adapt the name of the file regarding your configuration

Userlevel 1
Badge +6

the restart of cbd (indicated in /var/log/centreon-broker/watchdog.log) was because you generate and export configuration?

 

Today I was reloading the configuration to poler but also restarting the centeron services in cli

 

$ cat /var/lib/centreon-broker/central-rrd-master-stats.json |jq .
{
"asio_version": "1.22.1",
"endpoint central-rrd-master-input": {
"event_processing_speed": 0,
"last_connection_attempt": -1,
"last_connection_success": -1,
"last_event_at": 1664874758,
"memory_file_path": "/var/lib/centreon-broker//central-rrd-master.memory.central-rrd-master-input",
"name": "central-rrd-master-input",
"one_peer_retention_mode": false,
"peers": "0: ",
"queue_file_path": "/var/lib/centreon-broker//central-rrd-master.queue.central-rrd-master-input",
"queued_events": 0,
"read_filters": "all",
"state": "listening",
"write_filters": ""
},
"endpoint central-rrd-master-output": {
"event_processing_speed": 5.645571049511658e-05,
"failover": null,
"last_connection_attempt": 1664874758,
"last_connection_success": 1664874758,
"last_event_at": 1664877202,
"memory_file_path": "/var/lib/centreon-broker//central-rrd-master.memory.central-rrd-master-output",
"name": "central-rrd-master-output",
"queue_file_enabled": false,
"queue_file_path": "/var/lib/centreon-broker//central-rrd-master.queue.central-rrd-master-output",
"queued_events": 0,
"read_filters": "all",
"state": "connected",
"status": "reading event from multiplexing engine",
"unacknowledged_events": 0,
"write_filters": "all"
},
"module/usr/share/centreon/lib/centreon-broker/10-neb.so": {
"name": "/usr/share/centreon/lib/centreon-broker/10-neb.so",
"size": "1545352B",
"state": "loaded"
},
"module/usr/share/centreon/lib/centreon-broker/15-stats.so": {
"name": "/usr/share/centreon/lib/centreon-broker/15-stats.so",
"size": "704680B",
"state": "loaded"
},
"module/usr/share/centreon/lib/centreon-broker/50-tcp.so": {
"name": "/usr/share/centreon/lib/centreon-broker/50-tcp.so",
"size": "900240B",
"state": "loaded"
},
"module/usr/share/centreon/lib/centreon-broker/60-tls.so": {
"name": "/usr/share/centreon/lib/centreon-broker/60-tls.so",
"size": "618688B",
"state": "loaded"
},
"module/usr/share/centreon/lib/centreon-broker/70-rrd.so": {
"name": "/usr/share/centreon/lib/centreon-broker/70-rrd.so",
"size": "1024088B",
"state": "loaded"
},
"mysql manager": {
"delay since last check": "0"
},
"now": "1664894885",
"pid": 13635,
"thread_pool": {
"latency": "",
"size": 4
},
"version": "22.04.1"
}

 

Userlevel 6
Badge +18

and can you give me the same but for central-broker-master-stats.json

It look like the RRD broker doesn’t receive data from RRD master

Userlevel 6
Badge +18

Can you check also in “Configuration > Pollers > Broker configuration”, editing “central-broker-master” if you have an output to forward event to “centreon-broker-rrd”:

 

Userlevel 1
Badge +6

 

 

 

 

 

Userlevel 1
Badge +6

I added an additional output like yours and reloaded the services. The charts are starting to appear.
In the version before the update from 10/21/11 to 04/22/6, could I have this file, but it was lost after the update?

 

-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7801.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7800.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7799.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7798.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7797.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7796.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7795.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7794.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7793.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7792.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7791.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7790.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7789.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7788.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6097.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6098.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6099.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6100.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6101.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6102.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6103.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6104.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6105.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 5121.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 5122.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 5123.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 5124.rrd

 

 

Userlevel 1
Badge +6

I added an additional output like yours and reloaded the services. The charts are starting to appear.
In the version before the update from 20.10.11 to 22,04.6 could I have this file, but it was lost after the update?

 

-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7795.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7794.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7793.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7792.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7791.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7790.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7789.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-03 18:19 7788.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6097.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6098.rrd
-rw-rw-r-- 1 centreon-broker centreon-broker  450088 10-04 18:31 6099.rrd

 

 

Userlevel 6
Badge +18

Thank you for your feedback.

Do you have a dump of “centreon” configuration database before update?

It look like an output disappear during update process.

Sorry for the inconvenience

Userlevel 1
Badge +6

Unfortunately, I do not have a backup. I opted for virtual machine snapchots.

 

Thank you for your help

Badge

Hi,

 

I have the same problem since 6days. (2022/10/01)

I make the update this day from 22.04.5 to 22.04.6 but still have rrd file not updating.

 

Thank you for help

Badge
{
"asio_version":"1.22.1",
"endpoint central-rrd-master-input":{
"event_processing_speed":0.0,
"last_connection_attempt":-1.0,
"last_connection_success":-1.0,
"last_event_at":1665070948.0,
"memory_file_path":"/var/lib/centreon-broker//central-rrd-master.memory.central-rrd-master-input",
"name":"central-rrd-master-input",
"one_peer_retention_mode":false,
"peers":"0: ",
"queue_file_path":"/var/lib/centreon-broker//central-rrd-master.queue.central-rrd-master-input",
"queued_events":0,
"read_filters":"all",
"state":"listening",
"write_filters":""
},
"endpoint central-rrd-master-output":{
"event_processing_speed":0.0007692307692307692,
"failover":null,
"last_connection_attempt":1665070948.0,*
"last_connection_success":1665070948.0,
"last_event_at":1665070948.0,
"memory_file_path":"/var/lib/centreon-broker//central-rrd-master.memory.central-rrd-master-output",
"name":"central-rrd-master-output",
"queue_file_enabled":false,
"queue_file_path":"/var/lib/centreon-broker//central-rrd-master.queue.central-rrd-master-output",
"queued_events":0,
"read_filters":"all",
"state":"connected",
"status":"reading event from multiplexing engine",
"unacknowledged_events":0,
"write_filters":"all"
},
"module/usr/share/centreon/lib/centreon-broker/10-neb.so":{
"name":"/usr/share/centreon/lib/centreon-broker/10-neb.so",
"size":"1545352B","state":"loaded"
},
"module/usr/share/centreon/lib/centreon-broker/15-stats.so":{
"name":"/usr/share/centreon/lib/centreon-broker/15-stats.so",
"size":"704680B","state":"loaded"
},
"module/usr/share/centreon/lib/centreon-broker/50-tcp.so":{
"name":"/usr/share/centreon/lib/centreon-broker/50-tcp.so",
"size":"900240B",
"state":"loaded"
},
"module/usr/share/centreon/lib/centreon-broker/60-tls.so":{
"name":"/usr/share/centreon/lib/centreon-broker/60-tls.so",
"size":"618688B",
"state":"loaded"
},
"module/usr/share/centreon/lib/centreon-broker/70-rrd.so":{
"name":"/usr/share/centreon/lib/centreon-broker/70-rrd.so",
"size":"1024088B",
"state":"loaded"},
"mysql manager":{
"delay since last check":"0"
},
"now":"1665072218",
"pid":38051,
"thread_pool":{
"latency":"",
"size":8
},
"version":"22.04.1"
}

voici la sortie de cat /var/lib/centreon-broker/central-rrd-master-stats.json

Userlevel 6
Badge +18

@mgu_flag you central-rrd-master doesn’t have connected peer. This indicate that the central-broker-master it doesn’t corrected to it.

Check my answer and create missing “output” for central-broker-master configuration, then generate and export configuration.

Badge

Bonjour Laurent,

J’ai ajouté les output manquant cbd se lançait avec erreur du coup je rollback la config et c’est de nouveau OK

[error] Error during cbd exit: config parser: unified_sql output is incompatible with storage/sql outputs

j’ai ajouté comme output 

 

Alors je conviens que storage est déprécié (j’ai recopié une config sur un centreon 21.04 d’ailleur CBD me l’indique 

J’avais également ajouté IPV4 recopié également de la config d’un centreon 21.04

Une idée ?

Merci beaucoup

Badge

Merci Laurent !

En ajoutant que l’output IPV4 les fichiers rrd se remettent à fonctionner.

Great !!!

Badge +2

Same problem.

Merci pour la conf rrd.

Badge +6

hello

Badge +6

Since the centreon 21.04 upgrade to 22.04.7
Since the upgrade we have several error
and despite the deletion of the brocker files and restart cbd
systemctl stop cbd && rm -f /var/lib/centreon-broker/{*cache*,*new*,*unprocess*} && systemctl start cbd
errors still persist

Reply