hello
can you provide the result of the following commands, running them on the MBI server
/usr/share/centreon-bi/etl/centreonbiMonitoring.pl --db-content
and
/usr/share/centreon-bi/tools/diagnostic.sh
(this one has a long output, and creates a log file, you could send that)
and from mysql command, also from the mariadb on the MBI
show databases;
use centreon_storage
show tables;
Edit and I’ll add that
SELECT
TABLE_NAME,
TABLE_ROWS
FROM
`information_schema`.`tables`
WHERE
`table_schema` = 'centreon_storage';
Hi @christophe.niel-ACT sorry for late;
[root@Srv-Centreon-MBI ~]# /usr/share/centreon-bi/etl/centreonbiMonitoring.pl --db-content
[Table mod_bi_hoststateevents: EMPTY] [Table mod_bi_servicestateevents: EMPTY] [mod_bi_time: 2024-08-02 00:00:00] [Table mod_bi_hostavailability: EMPTY] [Table mod_bi_serviceavailability: EMPTY] [Table mod_bi_hgmonthavailability: EMPTY] [Table mod_bi_hgservicemonthavailability: EMPTY] [Table mod_bi_metricdailyvalue: EMPTY] [Table mod_bi_metricmonthcapacity: EMPTY] [Table mod_bi_metrichourlyvalue: EMPTY] [Table mod_bi_metriccentiledailyvalue: EMPTY] [Table mod_bi_metriccentilemonthlyvalue: EMPTY] [Table mod_bi_metriccentileweeklyvalue: EMPTY]
[root@Srv-Centreon-MBI ~]#
[root@Srv-Centreon-MBI ~]#
[root@Srv-Centreon-MBI ~]#
[root@Srv-Centreon-MBI ~]#
[root@Srv-Centreon-MBI ~]# /usr/share/centreon-bi/tools/diagnostic.sh
########################################################################
##########################################################
# #
# Diagnostic generated on: Fri Sep 27 10:56:42 +01 2024 #
# #
##########################################################
#################### Check connection to databases ####################
--> Connection to monitoring databases:
########## Java ##########
[OK] Java 17 installed
[OK] Connection to centreon database on 10.241.37.12 successful
[OK] Connection to centreon_storage database on 10.241.37.12 successful
--> Connection to reporting server:
[OK] Connection to centreon on localhost successful
[OK] Connection to centreon_storage on localhost successful
#################### CBIS deamon ####################
[OK] CBIS is running
#################### ETL configuration ####################
[INFO] Use large memory tweaks option is disabled
[WARNING] ETL logs returning errors, please check /var/log/centreon-bi/centreonBIETL.log for more details
[OK] ETL cron activated
#################### Retention configuration ####################
[OK] No errors found on retention logs
[OK] Purge cron activated
[OK] Purge option is enabled
#################### SSH key exchange ####################
[OK] Exchange key between the reporting server and 10.241.37.12 done
#################### Check MySQL time consistency ####################
[OK] Reporting & monitoring MySQL timezone are the same ( +01 )
[OK] MySQL date difference between Central (Fri Sep 27 10:56:42 +01 2024) & Reporting server(Fri Sep 27 10:56:43 +01 2024) is less than 10 secondes
#################### Time synchronization ####################
[OK] Chrony or NTP service is running
[OK] 0.725854 sec compare to the monitoring database
#################### MariaDB configuration ####################
[INFO] Version = 10.5.20-MariaDB-log
[OK] open_files_limit=32000
[OK] wait_timeout = 28800
[OK] interactive_timeout = 28800
[OK] group_concat_max_len = 1048576
[OK] innodb_file_per_table enabled.
[INFO] Make sure that /var/lib/mysql/ and /tmp are under the same partition
[INFO] myisam_sort_buffer_size = 1073741824
[INFO] table_cache = 2048
[INFO] thread_cache_size = 64
[INFO] key_buffer_size = 2147483648
[INFO] query_cache_size = 8388608
[INFO] query_cache_type = ON
[INFO] query_cache_limit = 1048576
[INFO] innodb_buffer_pool_size = 2147483648
[INFO] innodb_flush_log_at_trx_commit = 2
[INFO] innodb_log_buffer_size = 1073741824
[INFO] innodb_log_file_size = 536870912
[INFO] innodb_log_files_in_group = 1
[INFO] innodb_file_per_table = ON
[INFO] innodb_open_files = 1024
[INFO] tmp_table_size = 536870912
[INFO] max_heap_table_size = 536870912
[INFO] innodb_max_dirty_pages_pct = 30.000000
[INFO] innodb_lock_wait_timeout = 28800
################# Reporting server memory usage ################
[INFO] Total Memory: 31Gi
[INFO] Used Memory: 5.2Gi
[INFO] Free Memory: 16Gi
################ Reporting server storage usage ################
[INFO] Paritition / used at 48%. (free space:17G)
[INFO] Paritition /usr used at 21%. (free space:12G)
[INFO] Paritition /boot used at 35%. (free space:662M)
[INFO] Paritition /var used at 2%. (free space:94G)
[INFO] Paritition /home used at 4%. (free space:975M)
[INFO] Paritition /var/log used at 3%. (free space:9.8G)
[INFO] Paritition /var/lib/mysql used at 9%. (free space:4.2T)
[INFO] Paritition /var/backup used at 100%. (free space:7.3M)
############## Configuration of the data warehouse #############
[INFO] Average number of Hostgroups associated to one Host: 1.1021
[INFO] Average number hostcategories associated to one Host: 1.0032
[INFO] Average number of Service categories associated to one Service:
[INFO] Number of hosts for which reporting statistics are calculated: 4074
[INFO] Number of services for which reporting statistics are calculated:
[INFO] Number of metrics for which reporting statistics are calculated:
[INFO] Average number of metrics by service:
[INFO] Number of time period: 0
#################### Reporting server database consistency ####################
[OK] Column mod_bam_reporting_bv.bv_name is consistent
[OK] Column mod_bam_reporting_ba.ba_name is consistent
[OK] Column mod_bam_reporting_kpi.kpi_name is consistent
[OK] Column mod_bam_reporting_kpi.ba_name is consistent
[OK] Column mod_bam_reporting_kpi.host_name is consistent
[OK] Column mod_bam_reporting_kpi.kpi_ba_name is consistent
[OK] Column mod_bam_reporting_kpi.meta_service_name is consistent
[OK] Column mod_bam_reporting_kpi.boolean_name is consistent
[OK] Column mod_bi_servicemetrics.metric_name is consistent
[OK] Column mod_bi_servicemetrics.metric_unit is consistent
[OK] Column mod_bi_servicemetrics.sc_name is consistent
[OK] Column mod_bi_servicemetrics.host_name is consistent
[OK] Column mod_bi_servicemetrics.hg_name is consistent
[OK] Column mod_bi_servicemetrics.hc_name is consistent
#################### Job configuration ####################
[OK] All views are available
[INFO] Configured jobs: 68 (68 based on standard report designs, 0 based on specific report designs)
[INFO] Scheduled jobs: 0 - Finished jobs: 67 - Running jobs: 0 - Stopped jobs: 0 - Failed jobs: 1
[INFO] Cyclic jobs: 0 ( Daily: 0 - Weekly: 0 - Monthly: 0)
################### Report designs configured ##################
+-----------------------------------------------+----------------+
| report design | job configured |
+-----------------------------------------------+----------------+
| Hostgroup-service-metric-performance-list | 9 |
| Hostgroups-Rationalization-Of-Resources-1 | 8 |
| Hostgroup-Storage-Capacity-List | 7 |
| Hostgroups-Categories-Performance-List | 5 |
| Hostgroups-Incidents-1 | 5 |
| Hostgroup-Capacity-Planning-Linear-Regression | 5 |
| Hostgroup-Traffic-Average-Usage-By-Interface | 4 |
| Hostgroups-Storage-Capacity-1 | 3 |
| Hostgroup-Availability-2 | 2 |
| Hostgroup-Host-Availability-List | 2 |
| VMWare-Cluster-Performances-1 | 2 |
| Hostgroup-Storage-Capacity-2 | 2 |
| Hostgroup-Graphs-v2 | 1 |
| Content-diagnostic-availability | 1 |
| Hostgroup-Service-Availability-List | 1 |
| Hostgroup-service-incident-resolution-2 | 1 |
| Host-Detail-2 | 1 |
| BV-BA-Availabilities-Calendar | 1 |
| Hostgroup-Host-Events-Pareto | 1 |
| BV-BA-Availabilities-1 | 1 |
| Metric-integrity-check | 1 |
| BA-Event-List | 1 |
| Poller-Performances | 1 |
| Hosts-Not-Classified | 1 |
| Hostgroups-Availability-1 | 1 |
| Host-Graphs-v2 | 1 |
+-----------------------------------------------+----------------+
################## Publication rules configured ################
+--------------+-----------------+
| type of rule | number of rules |
+--------------+-----------------+
| SFTP | 1 |
+--------------+-----------------+
############################# Trash ############################
Number of report design in the trash: 0
Number of jobs in the trash: 0
Number of archive reports in the trash: 8
########################## Widgets used #########################
+----------------------------------------------------------+-------+
| widget name | total |
+----------------------------------------------------------+-------+
| Global Health | 31 |
| Service Monitoring | 24 |
| Host Monitoring | 16 |
| Tactical Overview | 2 |
| Open Tickets | 2 |
| Hostgroup Monitoring | 2 |
| Engine-status | 1 |
| MBI - Hostgroup availability by host categories by month | 1 |
| MBI - Service MTBF and MTRS | 1 |
| Live Top 10 CPU Usage | 1 |
| MBI - Metric capacity planning | 1 |
| Live Top 10 Memory Usage | 1 |
+----------------------------------------------------------+-------+
################ Reporting server databases size ###############
+--------------------+-----------------+
| Database | Size (MB) |
+--------------------+-----------------+
| centreon | 42.37500000 |
| centreon_storage | 253355.48972511 |
| information_schema | 0.20312500 |
+--------------------+-----------------+
############## Reporting server TOP 10 table size ##############
+--------+------------------+----------------------------+-----------------+------------+-------------+--------------+--------------+-------+
| ENGINE | table_schema | table_name | Entry_data_size | Rows | Data_Size | Index_Size | Total_size | Ratio |
+--------+------------------+----------------------------+-----------------+------------+-------------+--------------+--------------+-------+
| InnoDB | centreon_storage | data_bin | 41 | 2319549726 | 97312047104 | 164958453760 | 262270500864 | 0.37 |
| InnoDB | centreon_storage | servicestateevents | 59 | 6128290 | 364789760 | 426491904 | 791281664 | 0.46 |
| MyISAM | centreon_storage | mod_bi_tmp_minmaxavgvalue | 26 | 3761767 | 97805942 | 82367488 | 180173430 | 0.54 |
| InnoDB | centreon_storage | mod_bi_tmp_minmaxavgvalue2 | 46 | 3760673 | 174800896 | 93995008 | 268795904 | 0.65 |
| InnoDB | centreon_storage | mod_bi_tmp_minmaxavgvalue1 | 50 | 3759432 | 189497344 | 93995008 | 283492352 | 0.67 |
| InnoDB | centreon_storage | mod_bi_tmp_minmaxavgvalue8 | 50 | 3758780 | 189497344 | 93995008 | 283492352 | 0.67 |
| InnoDB | centreon_storage | mod_bi_tmp_minmaxavgvalue6 | 46 | 3753782 | 175849472 | 93995008 | 269844480 | 0.65 |
| InnoDB | centreon_storage | mod_bi_tmp_minmaxavgvalue7 | 50 | 3750630 | 189497344 | 93995008 | 283492352 | 0.67 |
| InnoDB | centreon_storage | mod_bi_tmp_minmaxavgvalue3 | 50 | 3747696 | 189497344 | 93995008 | 283492352 | 0.67 |
| InnoDB | centreon_storage | mod_bi_tmp_minmaxavgvalue5 | 50 | 3744436 | 188448768 | 93995008 | 282443776 | 0.67 |
+--------+------------------+----------------------------+-----------------+------------+-------------+--------------+--------------+-------+
###################### Software versions #######################
[INFO] Centreon Web: 22.10.19
95d15121-eaf9-4439-8049-f93e6b1afc15
[INFO] Centreon MBI interface: 22.10.6
[INFO] Centreon BAM server: 22.10.4
[INFO] centreon-bi-reporting-server-22.10.6-1.el8.noarch
[INFO] centreon-bi-report-22.10.6-1.el8.noarch
[INFO] centreon-bi-etl-22.10.6-1.el8.noarch
[INFO] centreon-bi-engine-22.10.6-1.el8.noarch
###################################################################
# #
# Log file is availabile on /var/log/centreon-bi/diagnostic.log #
# #
###################################################################
Hope this informations can help you for helping me
[root@Srv-Centreon-MBI ~]# mariadb
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 3840
Server version: 10.5.20-MariaDB-log MariaDB Server
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]>
MariaDB [(none)]>
MariaDB [(none)]> show databases;
+--------------------+
| Database |
+--------------------+
| centreon |
| centreon_storage |
| information_schema |
| mysql |
| performance_schema |
+--------------------+
5 rows in set (0.000 sec)
MariaDB [(none)]>
MariaDB [(none)]> use centreon_storage
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
MariaDB [centreon_storage]>
MariaDB [centreon_storage]> show tables;
+--------------------------------------------+
| Tables_in_centreon_storage |
+--------------------------------------------+
| acknowledgements |
| centreon_acl |
| data_bin |
| downtimes |
| hosts |
| hoststateevents |
| index_data |
| instances |
| metrics |
| mod_bam_reporting_ba |
| mod_bam_reporting_ba_availabilities |
| mod_bam_reporting_ba_events |
| mod_bam_reporting_ba_events_durations |
| mod_bam_reporting_bv |
| mod_bam_reporting_kpi |
| mod_bam_reporting_kpi_events |
| mod_bam_reporting_relations_ba_bv |
| mod_bam_reporting_relations_ba_kpi_events |
| mod_bam_reporting_relations_ba_timeperiods |
| mod_bam_reporting_status |
| mod_bam_reporting_timeperiods |
| mod_bam_reporting_timeperiods_exceptions |
| mod_bam_reporting_timeperiods_exclusions |
| mod_bi_aclgroup |
| mod_bi_aclgroup_hc_relations |
| mod_bi_aclgroup_hg_relations |
| mod_bi_aclgroup_host_relations |
| mod_bi_aclgroup_sc_relations |
| mod_bi_aclgroup_user_relations |
| mod_bi_ba_incidents |
| mod_bi_centiles |
| mod_bi_hgmonthavailability |
| mod_bi_hgservicemonthavailability |
| mod_bi_hostavailability |
| mod_bi_hostcategories |
| mod_bi_hostgroups |
| mod_bi_hosts |
| mod_bi_hoststateevents |
| mod_bi_hoststateevents_tmp |
| mod_bi_liveservice |
| mod_bi_metriccentiledailyvalue |
| mod_bi_metriccentilemonthlyvalue |
| mod_bi_metriccentileweeklyvalue |
| mod_bi_metricdailyvalue |
| mod_bi_metrichourlyvalue |
| mod_bi_metricmonthcapacity |
| mod_bi_serviceavailability |
| mod_bi_servicecategories |
| mod_bi_servicemetrics |
| mod_bi_services |
| mod_bi_servicestateevents |
| mod_bi_servicestateevents_tmp |
| mod_bi_time |
| mod_bi_tmp_minmaxavgvalue |
| mod_bi_tmp_minmaxavgvalue1 |
| mod_bi_tmp_minmaxavgvalue2 |
| mod_bi_tmp_minmaxavgvalue3 |
| mod_bi_tmp_minmaxavgvalue4 |
| mod_bi_tmp_minmaxavgvalue5 |
| mod_bi_tmp_minmaxavgvalue6 |
| mod_bi_tmp_minmaxavgvalue7 |
| mod_bi_tmp_minmaxavgvalue8 |
| mod_bi_tmp_today_hosts |
| mod_bi_tmp_today_servicemetrics |
| mod_bi_tmp_today_services |
| mod_bi_users |
| servicestateevents |
+--------------------------------------------+
67 rows in set (0.001 sec)
MariaDB [centreon_storage]>
MariaDB [centreon_storage]>
MariaDB [centreon_storage]> SELECT
-> TABLE_NAME,
-> TABLE_ROWS
-> FROM
-> `information_schema`.`tables`
-> WHERE
-> `table_schema` = 'centreon_storage';
+--------------------------------------------+------------+
| TABLE_NAME | TABLE_ROWS |
+--------------------------------------------+------------+
| mod_bi_aclgroup | 17 |
| mod_bi_aclgroup_sc_relations | 68 |
| mod_bi_aclgroup_user_relations | 107 |
| mod_bi_aclgroup_host_relations | 20525 |
| mod_bi_aclgroup_hc_relations | 1972 |
| mod_bi_ba_incidents | 0 |
| mod_bi_hostcategories | 115 |
| mod_bi_aclgroup_hg_relations | 483 |
| mod_bi_hostgroups | 118 |
| mod_bi_services | 0 |
| mod_bi_hosts | 4620 |
| mod_bi_time | 8750 |
| mod_bi_servicemetrics | 0 |
| mod_bam_reporting_timeperiods_exceptions | 0 |
| mod_bam_reporting_relations_ba_timeperiods | 0 |
| mod_bi_users | 720 |
| mod_bam_reporting_timeperiods_exclusions | 0 |
| mod_bi_centiles | 0 |
| mod_bi_liveservice | 0 |
| mod_bi_servicecategories | 4 |
| mod_bam_reporting_status | 0 |
| mod_bi_tmp_minmaxavgvalue5 | 3744436 |
| mod_bi_tmp_minmaxavgvalue7 | 3750630 |
| mod_bi_tmp_minmaxavgvalue | 3761767 |
| mod_bi_tmp_minmaxavgvalue4 | 3740524 |
| mod_bi_tmp_minmaxavgvalue2 | 3760673 |
| mod_bi_tmp_today_servicemetrics | 0 |
| servicestateevents | 6128290 |
| mod_bi_tmp_today_services | 0 |
| mod_bi_tmp_minmaxavgvalue3 | 3747696 |
| mod_bi_tmp_minmaxavgvalue1 | 3759432 |
| mod_bi_hgservicemonthavailability | 0 |
| data_bin | 2319549726 |
| mod_bi_tmp_minmaxavgvalue8 | 3758780 |
| mod_bi_hoststateevents | 0 |
| hoststateevents | 205326 |
| instances | 7 |
| mod_bam_reporting_ba_events | 947 |
| mod_bam_reporting_kpi | 270 |
| metrics | 220874 |
| mod_bi_servicestateevents_tmp | 0 |
| mod_bi_hoststateevents_tmp | 0 |
| mod_bam_reporting_relations_ba_kpi_events | 2732 |
| centreon_acl | 228015 |
| mod_bi_hgmonthavailability | 0 |
| mod_bi_metricmonthcapacity | 0 |
| mod_bi_tmp_today_hosts | 4526 |
| mod_bam_reporting_bv | 21 |
| mod_bi_servicestateevents | 0 |
| mod_bam_reporting_relations_ba_bv | 31 |
| mod_bam_reporting_timeperiods | 6 |
| mod_bi_metricdailyvalue | 2 |
| mod_bi_hostavailability | 321 |
| mod_bi_metrichourlyvalue | 2 |
| mod_bam_reporting_ba | 69 |
| index_data | 59784 |
| mod_bi_tmp_minmaxavgvalue6 | 3753782 |
| mod_bam_reporting_ba_availabilities | 24921 |
| hosts | 5385 |
| downtimes | 9677 |
| mod_bam_reporting_ba_events_durations | 930 |
| mod_bi_metriccentilemonthlyvalue | 3 |
| mod_bam_reporting_kpi_events | 2949 |
| acknowledgements | 6595 |
| mod_bi_serviceavailability | 321 |
| mod_bi_metriccentiledailyvalue | 2 |
| mod_bi_metriccentileweeklyvalue | 2 |
+--------------------------------------------+------------+
67 rows in set (0.005 sec)
Hello
from your output your tables are not all empty, and date have stopped being update on August 2nd
you really need to open a ticket on the support page as you have a business product (MBI) and it comes with a support
you will need to have them check your config and what went wrong, maybe it is a login issue, or something else, but you will need to execute commands to fill the tables and that is not something easily explainable here
(and before filling the table, you need to fix the root cause that prevent the daily load)
indeed that's exactly it well in short lots of things that I wouldn't like to say but I found myself with the obligation to manage the problem alone;
in view of this problem we cannot generate the capacity planning reports or anything I have tried everything but it seems to me that there is a problem of sizing and with the ETL;
too bad
I thank you for your responsiveness
Hello,
We had the same issue 2 months ago.
First, be sure the cron are activated
- /etc/cron.d/centreon-bi-engine
- /etc/cron.d/centreon-bi-backup-reporting-server
- /etc/cron.d/centreon-bi-purge
and what is the date of the last iteration and if there are issue in file:
tail -f /var/log/centreon-bi/centreonBIETL.log
after this we rebuilded the datebase with the help of this procedure:
Concepts | Centreon Documentation
Hope it will help you,
Laurent
Hello and thank you for your feedback
the cron are all activated and the requested output is:
[root@Srv-Centreon-MBI ~]# tail -f /var/log/centreon-bi/centreonBIETL.log
[Wed Oct 2 04:37:51 2024] [INFO] Searching for duplicate host/service entries
[Wed Oct 2 04:37:51 2024] [INFO] Getting host properties from Centreon database
[Wed Oct 2 04:37:54 2024] [INFO] Updating host dimension in Centstorage
[Wed Oct 2 04:37:55 2024] [INFO] Getting hostgroup properties from Centreon database
[Wed Oct 2 04:37:55 2024] [INFO] Updating hostgroup dimension in Centstorage
[Wed Oct 2 04:37:55 2024] [INFO] Getting hostcategories properties from Centreon database
[Wed Oct 2 04:37:55 2024] [INFO] Updating hostcategories dimension in Centstorage
[Wed Oct 2 04:37:55 2024] [INFO] Getting servicecategories properties from Centreon database
[Wed Oct 2 04:37:55 2024] [INFO] Updating servicecategories dimension in Centstorage
[Wed Oct 2 04:37:55 2024] [INFO] Getting service properties from Centreon database
something is going wrong honestly I can't seem to find it after months of diagnosis
Hello @LaurentV ,
the problem was finally solved with Centreon support;
It's been a long time since I checked but when I checked yesterday I found an error message appeared and I don't know where it came from in addition to these two tables I don't know why they are no longer up to date,
[root@Srv-Centreon-MBI ~]# /usr/share/centreon-bi/etl/centreonbiMonitoring.pl --db-content
[mod_bi_metricmonthcapacity: 2024-11-01 00:00:00] [mod_bi_metriccentilemonthlyvalue: 2024-11-01 00:00:00]
any idea what command I should run to fill just these two tables without having to redo the whole procedure?
Kind Regards,
Hello,
I don’t know but when I also check, I have:
/usr/share/centreon-bi/etl/centreonbiMonitoring.pl --db-content
[Table mod_bi_metricmonthcapacity: EMPTY]
What do you have with command: /usr/share/centreon-bi/etl/centreonbiMonitoring.pl --jobs
and do you have some issue with:
/usr/share/centreon-bi/etl/centreonbiMonitoring.pl --jobs
Laurent
the --jobs is the command to get the jobs that generates the reports I think, not the jobs that import the data
as I stated above (I think) all issues with the report servers is handled by the support as it is a business product and not easily debugged (and not really documented)
I had an issue with empty table, and no import, this was fixed by removing some part of the gorgoned process that import the data and fallback on the “legacy” cron tab that were importing the data by script as a cron job
unfortunately I don’t know how this was done, someone from the support took control and did the operations.
also I don't think it was the same issue as all my table were empty, not just these two.
Sorry I can’t be more helpful
hello @christophe.niel-ACT
now I have a new problem I don't know where it comes from by bringing you the data with the documentation command it returns me this error message
ERROR - Login error [code: '451'] [message: 'Failed Connection Host']
all the connectivities are good I don't know what it can't connect to?
PS the daily crons work fine without any problem