Question

Need help: Can't detect poller/remote server after reboot

  • 29 October 2023
  • 1 reply
  • 134 views
F
Badge 
[2023-10-29T20:09:28.462+08:00] [sql] [error] storage: rebuilder: Unable to connect to the database: mysql_connection: error while starting connection
[2023-10-29T20:14:28.462+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-10-29T20:14:28.462+08:00] [sql] [error] storage: rebuilder: Unable to connect to the database: mysql_connection: error while starting connection
[2023-10-29T20:17:01.596+08:00] [tcp] [error] Error while reading on socket: End of file
[2023-10-29T20:17:01.696+08:00] [core] [error] feeder 'central-broker-master-input-1' error:Attempt to read data from peer SCRUBBED:45690 on a closing socket
[2023-10-29T20:17:01.696+08:00] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-10-29T20:17:01.696+08:00] [core] [info] feeder: queue files of client 'central-broker-master-input-1' removed
[2023-10-29T20:17:01.696+08:00] [core] [info] feeder: thread of client 'central-broker-master-input-1' will exit
[2023-10-29T20:17:02.209+08:00] [core] [info] removing 'central-broker-master-input-1' from acceptor 'central-broker-master-input'
[2023-10-29T20:17:02.210+08:00] [core] [info] Destroying muxer central-broker-master-input-1: number of events in the queue: 0
[2023-10-29T20:17:02.381+08:00] [core] [info] main: termination request received by process
[2023-10-29T20:17:05.155+08:00] [tcp] [error] Error while reading on socket: End of file
[2023-10-29T20:17:05.210+08:00] [core] [info] processing acceptor 'central-broker-master-input' finished
[2023-10-29T20:17:05.211+08:00] [core] [error] failover: global error: Attempt to read data from peer localhost:5670 on a closing socket
[2023-10-29T20:17:05.212+08:00] [core] [info] BBDO: unable to send stop message to peer, it is already stopped: Attempt to read data from peer localhost:5670 on a closing socket
[2023-10-29T20:17:05.212+08:00] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-10-29T20:17:05.212+08:00] [tcp] [error] TCP: Connection refused to localhost:5670
[2023-10-29T20:17:05.213+08:00] [core] [error] failover: global error: failover: 'centreon-broker-master-rrd' cannot connect endpoint.
[2023-10-29T20:17:05.305+08:00] [core] [info] bbdo stream stopped with 975 events acknowledged
[2023-10-29T20:17:05.355+08:00] [tcp] [error] Error while reading on socket: Operation aborted.
[2023-10-29T20:17:05.355+08:00] [core] [info] feeder: queue files of client 'central-broker-master-input-3' removed
[2023-10-29T20:17:05.357+08:00] [core] [info] feeder: thread of client 'central-broker-master-input-3' will exit
[2023-10-29T20:17:05.357+08:00] [core] [info] Destroying muxer central-broker-master-input-3: number of events in the queue: 0
[2023-10-29T20:17:05.357+08:00] [core] [info] multiplexing: stopping engine
[2023-10-29T20:17:05.357+08:00] [core] [info] multiplexing: sending events to muxers for the last time
[2023-10-29T20:17:06.213+08:00] [core] [info] Destroying muxer centreon-broker-master-rrd: number of events in the queue: 0
[2023-10-29T20:17:06.512+08:00] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-10-29T20:17:06.513+08:00] [tcp] [error] Error while reading on socket: Operation aborted.
[2023-10-29T20:17:06.513+08:00] [core] [info] Destroying muxer forward-to-master: number of events in the queue: 0
[2023-10-29T20:17:06.567+08:00] [core] [info] sql stream stopped with 0 ackowledged events
[2023-10-29T20:17:06.567+08:00] [core] [info] Destroying muxer central-broker-master-sql: number of events in the queue: 100000
[2023-10-29T20:17:08.145+08:00] [core] [info] storage stream stopped with 0 acknowledged events
[2023-10-29T20:17:08.145+08:00] [core] [info] Destroying muxer central-broker-master-perfdata: number of events in the queue: 100002
[2023-10-29T20:17:10.449+08:00] [core] [info] modules: closing '/usr/share/centreon/lib/centreon-broker/80-sql.so'
[2023-10-29T20:17:10.449+08:00] [core] [info] protocols: unregistering protocol 'SQL'
[2023-10-29T20:17:10.449+08:00] [core] [info] modules: closing '/usr/share/centreon/lib/centreon-broker/60-tls.so'
[2023-10-29T20:17:10.449+08:00] [core] [info] protocols: unregistering protocol 'TLS'
[2023-10-29T20:17:10.449+08:00] [core] [info] modules: closing '/usr/share/centreon/lib/centreon-broker/50-tcp.so'
[2023-10-29T20:17:10.449+08:00] [core] [info] protocols: unregistering protocol 'TCP'
[2023-10-29T20:17:10.450+08:00] [core] [info] modules: closing '/usr/share/centreon/lib/centreon-broker/20-storage.so'
[2023-10-29T20:17:10.450+08:00] [core] [info] protocols: unregistering protocol 'storage'
[2023-10-29T20:17:10.450+08:00] [core] [info] modules: closing '/usr/share/centreon/lib/centreon-broker/15-stats.so'
[2023-10-29T20:17:10.506+08:00] [core] [info] modules: closing '/usr/share/centreon/lib/centreon-broker/10-neb.so'
[2023-10-29T20:17:10.506+08:00] [core] [info] protocols: unregistering protocol 'BBDO'
[2023-10-29T20:17:10.506+08:00] [core] [info] protocols: unregistering protocol 'compression'
[2023-10-29T20:17:10.506+08:00] [core] [info] protocols: unregistering protocol 'file'
[2023-10-29T20:17:10.506+08:00] [core] [info] protocols: destruction (0 protocols still registered)
[2023-10-29T20:17:10.506+08:00] [core] [info] pool: the latency check encountered an error: Operation aborted.
[2023-10-29T20:17:10.508+08:00] [core] [info] log finished
[2023-10-29T20:17:32.130+08:00] [core] [info] /var/log/centreon-broker/centreon-remotesvr-SCRUBBED-broker.log : log started
[2023-10-29T20:17:32.130+08:00] [core] [info] Starting the TCP thread pool of 3 threads
[2023-10-29T20:17:32.131+08:00] [core] [info] protocols: registering protocol ('file' (layers 1-3)
[2023-10-29T20:17:32.131+08:00] [core] [info] protocols: registering protocol ('compression' (layers 6-6)
[2023-10-29T20:17:32.131+08:00] [core] [info] protocols: registering protocol ('BBDO' (layers 7-7)
[2023-10-29T20:17:32.131+08:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/50-tcp.so'
[2023-10-29T20:17:32.239+08:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/10-neb.so'
[2023-10-29T20:17:32.273+08:00] [core] [info] NEB: module for Centreon Broker 21.10.5
[2023-10-29T20:17:32.273+08:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/60-tls.so'
[2023-10-29T20:17:32.362+08:00] [core] [info] protocols: registering protocol ('TLS' (layers 5-5)
[2023-10-29T20:17:32.362+08:00] [core] [info] protocols: registering protocol ('TCP' (layers 1-4)
[2023-10-29T20:17:32.362+08:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/80-sql.so'
[2023-10-29T20:17:32.382+08:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/10-neb.so' which is already loaded
[2023-10-29T20:17:32.383+08:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/20-storage.so'
[2023-10-29T20:17:32.402+08:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/10-neb.so' which is already loaded
[2023-10-29T20:17:32.402+08:00] [core] [info] protocols: registering protocol ('storage' (layers 1-7)
[2023-10-29T20:17:32.403+08:00] [core] [info] protocols: registering protocol ('SQL' (layers 1-7)
[2023-10-29T20:17:32.403+08:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/20-storage.so' which is already loaded
[2023-10-29T20:17:32.403+08:00] [core] [info] modules: attempt to load module '/usr/share/centreon/lib/centreon-broker/15-stats.so'
[2023-10-29T20:17:32.407+08:00] [core] [info] Destroying muxer central-broker-master-input: number of events in the queue: 0
[2023-10-29T20:17:33.148+08:00] [core] [info] multiplexing: engine started
[2023-10-29T20:17:33.736+08:00] [sql] [error] mysql_connection: could not store service:  Out of range value for column 'notification_number' at row 1
[2023-10-29T20:17:33.755+08:00] [sql] [error] conflict_manager: error in the main loop: could not store service:  Out of range value for column 'notification_number' at row 1
[2023-10-29T20:17:34.147+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-10-29T20:17:34.147+08:00] [sql] [error] storage: rebuilder: Unable to connect to the database: mysql_connection: error while starting connection
[2023-10-29T20:17:37.407+08:00] [core] [info] Available connections cleaning: Operation aborted.
[2023-10-29T20:17:37.412+08:00] [core] [info] New incoming connection 'central-broker-master-input-1'
[2023-10-29T20:17:45.085+08:00] [core] [info] Available connections cleaning: Operation aborted.
[2023-10-29T20:17:45.094+08:00] [core] [info] New incoming connection 'central-broker-master-input-2'
[error] Error while reading on socket: Operation aborted.
[2023-10-29T17:50:50.843+08:00] [core] [info] Destroying muxer forward-to-master: number of events in the queue: 0
[2023-10-29T17:50:50.904+08:00] [core] [info] sql stream stopped with 0 ackowledged events
[2023-10-29T17:50:50.905+08:00] [core] [info] Destroying muxer central-broker-master-sql: number of events in the queue: 100000
[2023-10-29T17:50:53.468+08:00] [core] [info] storage stream stopped with 0 acknowledged events
[2023-10-29T17:50:53.468+08:00] [core] [info] Destroying muxer central-broker-master-perfdata: number of events in the queue: 100003
[2023-10-29T20:18:01.841+08:00] [tcp] [error] Error while reading on socket: End of file
[2023-10-29T20:18:01.941+08:00] [core] [error] feeder 'central-broker-master-input-2' error:Attempt to read data from peer SCRUBBED:35488 on a closing socket
[2023-10-29T20:18:01.941+08:00] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-10-29T20:18:01.941+08:00] [core] [info] feeder: queue files of client 'central-broker-master-input-2' removed
[2023-10-29T20:18:01.941+08:00] [core] [info] feeder: thread of client 'central-broker-master-input-2' will exit
[2023-10-29T20:18:03.125+08:00] [core] [info] removing 'central-broker-master-input-2' from acceptor 'central-broker-master-input'
[2023-10-29T20:18:03.126+08:00] [core] [info] Destroying muxer central-broker-master-input-2: number of events in the queue: 0
[2023-10-29T20:18:20.585+08:00] [core] [info] New incoming connection 'central-broker-master-input-3'
[2023-10-29T20:22:34.148+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-10-29T20:22:34.148+08:00] [sql] [error] storage: rebuilder: Unable to connect to the database: mysql_connection: error while starting connection
[2023-10-29T20:27:34.148+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-10-29T20:27:34.148+08:00] [sql] [error] storage: rebuilder: Unable to connect to the database: mysql_connection: error while starting connection


After the remote server and poller rebooted, it wasn’t detected in the centreon. I have attached the logs from centreon-remotesvr-SCRUBBED-broker.log, seek your assistance.

 

I have tried this solution to solve the mariadb problem but to no avail:

All the services are running fine as well. Please let me know what other information you need.

1 reply

F
Badge

 Updated Error Logs 07/11/23

From << /var/log/centreon-broker/centreon-remotesvr-axa-broker.log >>

[2023-11-07T12:25:38.507+08:00] [core] [info] multiplexing: engine started
[2023-11-07T12:25:39.343+08:00] [sql] [error] mysql_connection: could not store service:  Out of range value for column 'notification_number' at row 1
[2023-11-07T12:25:39.362+08:00] [sql] [error] conflict_manager: error in the main loop: could not store service:  Out of range value for column 'notification_number' at row 1
[2023-11-07T12:25:39.507+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:25:39.507+08:00] [sql] [error] storage: rebuilder: Unable to connect to the database: mysql_connection: error while starting connection
[2023-11-07T12:25:40.052+08:00] [core] [info] Available connections cleaning: Operation aborted.
[2023-11-07T12:25:40.057+08:00] [core] [info] New incoming connection 'central-broker-master-input-1'
[2023-11-07T12:25:42.232+08:00] [core] [info] Available connections cleaning: Operation aborted.
[2023-11-07T12:25:42.242+08:00] [core] [info] New incoming connection 'central-broker-master-input-2'
[2023-11-07T12:25:51.987+08:00] [tcp] [error] Error while reading on socket: End of file
[2023-11-07T12:25:52.087+08:00] [core] [error] feeder 'central-broker-master-input-1' error:Attempt to read data from peer 127.0.0.1:39874 on a closing socket
[2023-11-07T12:25:52.087+08:00] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-11-07T12:25:52.087+08:00] [core] [info] feeder: queue files of client 'central-broker-master-input-1' removed
[2023-11-07T12:25:52.088+08:00] [core] [info] feeder: thread of client 'central-broker-master-input-1' will exit
[2023-11-07T12:25:52.525+08:00] [core] [info] New incoming connection 'central-broker-master-input-3'
[2023-11-07T12:25:52.526+08:00] [core] [info] removing 'central-broker-master-input-1' from acceptor 'central-broker-master-input'
[2023-11-07T12:25:52.526+08:00] [core] [info] Destroying muxer central-broker-master-input-1: number of events in the queue: 0
[2023-11-07T12:25:57.543+08:00] [core] [info] main: configuration update requested
[2023-11-07T12:25:57.545+08:00] [core] [info] /var/log/centreon-broker/centreon-remotesvr-axa-broker.log : log started
[2023-11-07T12:25:57.545+08:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/50-tcp.so' which is already loaded
[2023-11-07T12:25:57.545+08:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/80-sql.so' which is already loaded
[2023-11-07T12:25:57.545+08:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/20-storage.so' which is already loaded
[2023-11-07T12:25:57.545+08:00] [core] [info] modules: attempt to load '/usr/share/centreon/lib/centreon-broker/15-stats.so' which is already loaded
[2023-11-07T12:25:57.565+08:00] [core] [info] sql stream stopped with 0 ackowledged events
[2023-11-07T12:25:57.565+08:00] [core] [info] Destroying muxer central-broker-master-sql: number of events in the queue: 100000
[2023-11-07T12:25:58.936+08:00] [core] [info] multiplexing: engine started
[2023-11-07T12:25:58.937+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:25:58.937+08:00] [core] [error] failover: global error: mysql_connection: error while starting connection
[2023-11-07T12:25:59.855+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:25:59.855+08:00] [sql] [error] storage: rebuilder: Unable to connect to the database: mysql_connection: error while starting connection
[2023-11-07T12:26:13.939+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:26:13.939+08:00] [core] [error] failover: global error: mysql_connection: error while starting connection
[2023-11-07T12:26:28.941+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:26:28.942+08:00] [core] [error] failover: global error: mysql_connection: error while starting connection
[2023-11-07T12:26:43.944+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:26:57.200+08:00] [tcp] [error] Error while reading on socket: End of file
[2023-11-07T12:26:57.260+08:00] [core] [error] failover: global error: Attempt to read data from peer 10.130.5.132:5669 on a closing socket
[2023-11-07T12:26:57.260+08:00] [core] [info] BBDO: unable to send stop message to peer, it is already stopped: Attempt to read data from peer SCRUBBED:5669 on a closing socket
[2023-11-07T12:26:57.260+08:00] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-11-07T12:26:57.416+08:00] [tcp] [error] TCP: Connection refused to 10.130.5.132:5669
[2023-11-07T12:26:57.416+08:00] [core] [error] failover: global error: failover: 'forward-to-master' cannot connect endpoint.
[2023-11-07T12:26:58.947+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:26:58.947+08:00] [core] [error] failover: global error: mysql_connection: error while starting connection
[2023-11-07T12:27:13.949+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:27:13.949+08:00] [core] [error] failover: global error: mysql_connection: error while starting connection
[2023-11-07T12:27:28.951+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:27:28.951+08:00] [core] [error] failover: global error: mysql_connection: error while starting connection
[2023-11-07T12:27:43.954+08:00] [sql] [error] mysql_connection: error while starting connection
[2023-11-07T12:27:43.954+08:00] [core] [error] failover: global error: mysql_connection: error while starting connection
[2023-11-07T12:27:49.919+08:00] [core] [info] main: termination request received by process
[2023-11-07T12:27:50.563+08:00] [tcp] [error] Error while reading on socket: End of file
[2023-11-07T12:27:50.616+08:00] [core] [error] failover: global error: Attempt to read data from peer localhost:5670 on a closing socket
[2023-11-07T12:27:50.616+08:00] [core] [info] BBDO: unable to send stop message to peer, it is already stopped: Attempt to read data from peer localhost:5670 on a closing socket
[2023-11-07T12:27:50.616+08:00] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-11-07T12:27:50.666+08:00] [tcp] [error] TCP: Connection refused to localhost:5670

 

From << /var/log/centreon-broker/centreon-remotesvr-axa-module.log >>

[2023-11-07 12:25:34.049] [core] [error] failover: global error: Attempt to read data from peer localhost:5669 on a closing socket
[2023-11-07 12:25:34.049] [core] [info] BBDO: unable to send stop message to peer, it is already stopped: Attempt to read data from pee$
[2023-11-07 12:25:34.049] [core] [info] bbdo stream stopped with 0 events acknowledged
[2023-11-07 12:25:37.050] [core] [error] failover: global error: BBDO: invalid protocol header, aborting connection: waiting for messag$
[2023-11-07 12:25:37.051] [core] [error] failover: global error: failover: 'central-module-master-output' cannot connect endpoint.

 

From <</var/log/messages>>;

Nov  7 12:53:50 centreon-remotesvr-axa mariadbd: 2023-11-07 12:53:50 682 [Warning] Access denied for user 'centreon_remote'@'localhost' (using password: YES)

 

From << /var/log/centreon-gorgone/gorgoned.log >>


2023-11-07 12:38:31 - ERROR - MySQL error : cannot connect to database 'mysql:host=localhost:3306;dbname=centreon_remotesvr': Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2 "No such file or directory") (caller: gorgone::class::db:/usr/share/perl5/vendor_perl/gorgone/class/db.pm:353) (try: 0)
2023-11-07 12:38:31 - ERROR - [nodes] -class- cannot find debug configuration
2023-11-07 12:38:32 - INFO - [cron] Adding cron definition 'broker_stats'
2023-11-07 12:38:32 - INFO - [cron] Adding cron definition 'engine_stats'
2023-11-07 12:38:32 - ERROR - MySQL error : cannot connect to database 'mysql:host=localhost:3306;dbname=centreon_remotesvr': Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2 "No such file or directory") (caller: gorgone::class::db:/usr/share/perl5/vendor_perl/gorgone/class/db.pm:353) (try: 0)
2023-11-07 12:38:32 - ERROR - [autodiscovery] -class- host discovery - cannot get platform versions - Login error [code: '503'] [message: 'Service Unavailable']
2023-11-07 12:38:33 - ERROR - MySQL error : cannot connect to database 'mysql:host=localhost:3306;dbname=centreon_remotesvr': Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2 "No such file or directory") (caller: gorgone::class::db:/usr/share/perl5/vendor_perl/gorgone/class/db.pm:353) (try: 1)
2023-11-07 12:38:33 - ERROR - [legacycmd] Cannot get configuration for pollers
2023-11-07 12:38:42 - INFO - [proxy] Node '23' is registered
2023-11-07 12:39:23 - INFO - [zmqclient] Client connected successfully to 'tcp://10.147.78.245:5556'
2023-11-07 12:39:23 - INFO - [proxy] Pong received from '23'
2023-11-07 12:39:35 - ERROR - [autodiscovery] -class- host discovery - cannot get platform versions - Login error [code: '401'] [message: 'Unauthorized']
2023-11-07 12:39:43 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:40:00 - INFO - [cron] Launching job 'broker_stats'
2023-11-07 12:40:00 - INFO - [cron] Launching job 'engine_stats'
2023-11-07 12:40:00 - INFO - [statistics] Collecting Engine statistics from target '23'
2023-11-07 12:40:00 - INFO - [statistics] Collecting Engine statistics from target '17'
2023-11-07 12:40:00 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:40:05 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:40:33 - INFO - [proxy] Pong received from '23'
2023-11-07 12:40:35 - ERROR - [autodiscovery] -class- host discovery - cannot get platform versions - Login error [code: '401'] [message: 'Unauthorized']
2023-11-07 12:40:53 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:41:33 - INFO - [proxy] Pong received from '23'
2023-11-07 12:41:35 - ERROR - [autodiscovery] -class- host discovery - cannot get platform versions - Login error [code: '401'] [message: 'Unauthorized']
2023-11-07 12:41:51 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:41:57 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:42:07 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:42:35 - ERROR - [autodiscovery] -class- host discovery - cannot get platform versions - Login error [code: '401'] [message: 'Unauthorized']
2023-11-07 12:42:47 - INFO - [proxy] Pong received from '23'
2023-11-07 12:43:27 - INFO - [proxy] Received setlogs for '23'
2023-11-07 12:43:35 - ERROR - [autodiscovery] -class- host discovery - cannot get platform versions - Login error [code: '401'] [message: 'Unauthorized']

 

Reply


Terms & ConditionsCookie settings