Question

UNKNOWN: 500 Can't connect to X.X.X.X:8443 (certificate verify failed)

  • 28 September 2023
  • 48 replies
  • 1561 views

Badge +6

Hello,

I got this error on several services using the api plugins

Anyone know how to fix it?

i made a search about this in forum but the only thread with it did not help…

 


48 replies

Badge +2

Hello,

do your probes check from cloud (poller) to onPremise (hosts to check) ?

Regards.

Badge +6

you went back  to “can’t connect” and “connexion refused”, which is not good. replacing centreon-plugins.exe in the script subfolder should not impact NSCLIENT, you don’t even have to restart it.

if you didn’t modify the nsclient.ini this really looks like there is something interfering with the https on tcp8443 on your windows hosts

 

this is a network problem or config problem on the windows side :/ (are you sure about the cause “1” I listed in a previous post above, about multiple service/program using the TCP8443 port?)

 

i am gonna test again. will let you know.

Badge +6

Hello,

do your probes check from cloud (poller) to onPremise (hosts to check) ?

Regards.

hello,

everything on premise.

Best regards,

Badge +2

 

Hi,

Does all yours probes using api with port 8443 on hosts windows  have the same problem ?

Does you set on hosts windows the same nsclient.ini ?

Regards.

Badge +6

 

Hi,

Does all yours probes using api with port 8443 on hosts windows  have the same problem ?

Does you set on hosts windows the same nsclient.ini ?

Regards.

Yes, all probes using the rest api have the same error message. We've downgraded the nsclient.ini by GPO, so all hosts have the same configuration.

Badge +6

you went back  to “can’t connect” and “connexion refused”, which is not good. replacing centreon-plugins.exe in the script subfolder should not impact NSCLIENT, you don’t even have to restart it.

if you didn’t modify the nsclient.ini this really looks like there is something interfering with the https on tcp8443 on your windows hosts

 

this is a network problem or config problem on the windows side :/ (are you sure about the cause “1” I listed in a previous post above, about multiple service/program using the TCP8443 port?)

 

With the nsclient rest api service started on port 8445 I can't see the port when doing netstat on the command line.
Maybe this is the problem

Badge +2

Hi,

Does windows firewall on windows hosts in Inbound Rules, allow “C:\programs files\Centreon NSClient++\nscp.exe” ?

 

Regards.

Badge +6

Hi,

Does windows firewall on windows hosts in Inbound Rules, allow “C:\programs files\Centreon NSClient++\nscp.exe” ?

 

Regards.

it does

 

Badge +2

Hi

If you use https://127.0.0.1:8443 ( on windows Host ) accept insecure connexion, do you obtain Sign in to use NSClient++ banner ?

Regards.

Userlevel 5
Badge +14

you went back  to “can’t connect” and “connexion refused”, which is not good. replacing centreon-plugins.exe in the script subfolder should not impact NSCLIENT, you don’t even have to restart it.

if you didn’t modify the nsclient.ini this really looks like there is something interfering with the https on tcp8443 on your windows hosts

 

this is a network problem or config problem on the windows side :/ (are you sure about the cause “1” I listed in a previous post above, about multiple service/program using the TCP8443 port?)

 

With the nsclient rest api service started on port 8445 I can't see the port when doing netstat on the command line.
Maybe this is the problem

 

there should a log in the nsclient folder, it should say stuff when it can’t bind a port or have major issues.

no idea what is happening

could you paste your section for nsclient restapi (here is mine for tcp port 5443, notice the “s” for ssl over 5443)

; Section for REST API
[/settings/WEB/server]

; ALLOWED HOSTS - A coma separated list of allowed hosts. You can use netmasks (/ syntax) or * to create ranges.
allowed hosts = mypoller

;CACHE ALLOWED HOSTS - If host names (DNS entries) should be cached, improves speed and security somewhat but won’t allow you to have dynamic IPs for your Nagios server.
cache allowed hosts = false

; PORT NUMBER - Port to use for REST API.
port = 5443s

;PASSWORD - Password used to authenticate against server
password = xxxx

; CERTIFICATE - Ssl certificate to use for the ssl server
certificate = ${certificate-path}/certificate.pem

 

Badge +6

you went back  to “can’t connect” and “connexion refused”, which is not good. replacing centreon-plugins.exe in the script subfolder should not impact NSCLIENT, you don’t even have to restart it.

if you didn’t modify the nsclient.ini this really looks like there is something interfering with the https on tcp8443 on your windows hosts

 

this is a network problem or config problem on the windows side :/ (are you sure about the cause “1” I listed in a previous post above, about multiple service/program using the TCP8443 port?)

 

With the nsclient rest api service started on port 8445 I can't see the port when doing netstat on the command line.
Maybe this is the problem

 

there should a log in the nsclient folder, it should say stuff when it can’t bind a port or have major issues.

no idea what is happening

could you paste your section for nsclient restapi (here is mine for tcp port 5443, notice the “s” for ssl over 5443)

; Section for REST API
[/settings/WEB/server]

; ALLOWED HOSTS - A coma separated list of allowed hosts. You can use netmasks (/ syntax) or * to create ranges.
allowed hosts = mypoller

;CACHE ALLOWED HOSTS - If host names (DNS entries) should be cached, improves speed and security somewhat but won’t allow you to have dynamic IPs for your Nagios server.
cache allowed hosts = false

; PORT NUMBER - Port to use for REST API.
port = 5443s

;PASSWORD - Password used to authenticate against server
password = xxxx

; CERTIFICATE - Ssl certificate to use for the ssl server
certificate = ${certificate-path}/certificate.pem

 

; Section for REST API
[/settings/WEB/server]

; ALLOWED HOSTS - A coma separated list of allowed hosts. You can use netmasks (/ syntax) or * to create ranges.
allowed hosts = 10.3.10.50

;CACHE ALLOWED HOSTS - If host names (DNS entries) should be cached, improves speed and security somewhat but won’t allow you to have dynamic IPs for your Nagios server.
cache allowed hosts = true

; PORT NUMBER - Port to use for REST API.
port = 8443s

;PASSWORD - Password used to authenticate against server
password = c.x.x.x.4

; CERTIFICATE - Ssl certificate to use for the ssl server
certificate = ${certificate-path}/certificate.pem

Userlevel 5
Badge +14

well, we went far, and we are running in circles.

 

your nsclient on your windows host seems OK.

you have the right windows firewall rules (nscp.exe allowed)

your centreon central seems to be able to communicate on https tcp8443 with the windows host, but only some times

 

sometimes it works (but cannot run the plugin), sometimes it says 403, which can mean a lot of things… 

I personnally never seen that, and can’t help you more, you need to figure what is happening between your servers and on your network (do you have another FW, EDPR, antivirus, or anything that could interact with the network of the windows host?)

 

please make a clean windows for testing, and just try to run basic nsclient restapi things like cpu/ram/disk before trying to run advanced plugin like wsus and exchange.

be sure to use the same config file and source for nsclient

 

also try what Piere asked above with the opening the url locally on the server, the web page should say NSCLIENT++

Badge +6

well, we went far, and we are running in circles.

 

your nsclient on your windows host seems OK.

you have the right windows firewall rules (nscp.exe allowed)

your centreon central seems to be able to communicate on https tcp8443 with the windows host, but only some times

 

sometimes it works (but cannot run the plugin), sometimes it says 403, which can mean a lot of things… 

I personnally never seen that, and can’t help you more, you need to figure what is happening between your servers and on your network (do you have another FW, EDPR, antivirus, or anything that could interact with the network of the windows host?)

 

please make a clean windows for testing, and just try to run basic nsclient restapi things like cpu/ram/disk before trying to run advanced plugin like wsus and exchange.

be sure to use the same config file and source for nsclient

 

also try what Piere asked above with the opening the url locally on the server, the web page should say NSCLIENT++

In any case, thank you for our exchange, I'm really short of ideas, I don't have the possibility to create VMs just for testing, I'm on the company infrastructure. I'll keep on testing and trying to find leads/solutions if I ever get everything working I'll put the solution here.

Thanks again for everything.

 

Edit : About opening the link on the machine concerned the internet explorer page gives me an error 

 

Userlevel 5
Badge +14

OK

Then your nsclient is not running properly

Be sure the service is running

Please check nsclient.log in the nsclient folder

 

Badge +6
2023-10-10 11:48:12: error:c:\source\0.5.1\modules\WEBServer\WEBServer.cpp:119: Invalid password/token from: 10.3.10.50: cXnXXX
2023-10-10 11:48:26: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 14:42:01: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 14:42:03: error:c:\source\0.5.1\modules\WEBServer\WEBServer.cpp:761: Failed to start server: Failed to set ssl_certificate: Cannot load PEM
2023-10-10 14:43:40: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 14:43:41: error:c:\source\0.5.1\modules\WEBServer\WEBServer.cpp:761: Failed to start server: Failed to set ssl_certificate: Cannot load PEM
2023-10-10 14:56:32: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 14:57:26: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 14:57:56: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 14:59:57: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 14:59:57: error:c:\source\0.5.1\modules\WEBServer\WEBServer.cpp:761: Failed to start server: Failed to set ssl_certificate: Cannot load PEM
2023-10-10 15:06:50: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:07:21: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:07:27: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:07:41: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:08:04: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:08:57: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:09:15: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:09:38: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:10:08: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:14:12: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:14:12: error:c:\source\0.5.1\modules\WEBServer\WEBServer.cpp:761: Failed to start server: Failed to set ssl_certificate: Cannot load PEM
2023-10-10 15:15:59: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-10 15:15:59: error:c:\source\0.5.1\modules\WEBServer\WEBServer.cpp:761: Failed to start server: Failed to set ssl_certificate: Cannot load PEM
2023-10-11 03:00:56: error:c:\source\0.5.1\modules\CheckSystem\pdh_thread.cpp:307: Failed to get network metrics: Failed to fetch network metrics: ConnectServer failed: namespace=root\cimv2, user=:8007045b: Un arrêt système est en cours.


2023-10-11 03:01:08: error:c:\source\0.5.1\modules\CheckSystem\pdh_thread.cpp:307: Failed to get network metrics: Failed to fetch network metrics: ConnectServer failed: namespace=root\cimv2, user=:8007045b: Un arrêt système est en cours.


2023-10-11 03:01:39: debug:c:\source\0.5.1\service\logger\nsclient_logger.cpp:52: Creating logger: threaded-file
2023-10-11 03:01:41: error:c:\source\0.5.1\modules\WEBServer\WEBServer.cpp:761: Failed to start server: Failed to set ssl_certificate: Cannot load PEM

 

Badge +6

OK

Then your nsclient is not running properly

Be sure the service is running

Please check nsclient.log in the nsclient folder

 

sorry I missed the message but you can see above what comes out of the nsclient log file

Userlevel 5
Badge +14

Ok, so now you have a real cause and an explanation on why the service is not working. it doesn’t find the certificate.pem et doesn’t start the web server

 

I have no idea how you got there, it was working before given you got some message saying it work. (if was working before I asked if the plugin was up to date from the last exe on github)

 

you should have a folder called “security” 

with the PEM file

 

this comes with the setup.exe of nsclient you get from centreon here Download Centreon | Open Source IT Infrastructure Monitoring Tool or here https://github.com/centreon/centreon-nsclient-build/releases/tag/20211104134145

 

I would suggest to uninstall everything nsclient related, if there are files remaining after uninstall, remove them manually, in program files), then make a clean install and put your “.ini” with the right configuration

(and you will need to update the centreon_plugins.exe in the script/centreon folder)

Badge +6

Ok, so now you have a real cause and an explanation on why the service is not working. it doesn’t find the certificate.pem et doesn’t start the web server

 

I have no idea how you got there, it was working before given you got some message saying it work. (if was working before I asked if the plugin was up to date from the last exe on github)

 

you should have a folder called “security” 

with the PEM file

 

this comes with the setup.exe of nsclient you get from centreon here Download Centreon | Open Source IT Infrastructure Monitoring Tool or here https://github.com/centreon/centreon-nsclient-build/releases/tag/20211104134145

 

I would suggest to uninstall everything nsclient related, if there are files remaining after uninstall, remove them manually, in program files), then make a clean install and put your “.ini” with the right configuration

(and you will need to update the centreon_plugins.exe in the script/centreon folder)

its already there

 

Userlevel 5
Badge +14

mmmh, you got 2 different modified date between the ca.pem and certificate.pem,

again, I have no idea if this is linked, but all the server I got have the same date for the 2 files (they seems to be generated at first run ever, then no idea how to regenerate/fix that, if they don’t match, the ssl will not work)

 

have you tried a full uninstall/delete all file in program files/reinstall of nsclient ?

 

Badge +6

mmmh, you got 2 different modified date between the ca.pem and certificate.pem,

again, I have no idea if this is linked, but all the server I got have the same date for the 2 files (they seems to be generated at first run ever, then no idea how to regenerate/fix that, if they don’t match, the ssl will not work)

 

have you tried a full uninstall/delete all file in program files/reinstall of nsclient ?

 

i installed this :

Centreon-NSClient-0.5.2.41-20211102-x64.exe

and now :

files are the same size now so now i got the “good message”.  >>> Unknown: 403 Forbidden

Userlevel 5
Badge +14

ok,

and does your local browser with “https://localhost:8443” work?

what does the nsclient.log says ?

 

Badge +6

i got it working thank you all and specially @christophe.niel-ACT 

so to get it working properly :

i downloaded the last nsclient from github

i downloaded also the last centreon-plugins.exe

installed all on the host

nscp web install

nscp web -- password --set XXXXXX

net stop nscp
net start nscp

add in  NSCPRESTAPIEXTRAOPTIONS : --ssl-opt="SSL_verify_mode => SSL_VERIFY_NONE"

modify the password in centreon

after this i had a 500 read timeout

i added in NSCPRESTAPIEXTRAOPTIONS : --timeout=30

and working!

Userlevel 5
Badge +14

nice to know it works

Reply