This started with a ticket to troubleshoot an SSL certificate on a nagios server in a remote datacenter.
CHECK_NRPE: ERROR – COULD NOT COMPLETE SSL HANDSHAKE
I checked the certifficate. It seemed correct and not expired as yet. I went to the nagios services configuration and found the command that was failing.
/usr/lib64/nagios/plugins/check_http -t 30 -I localhost -S -f follow -e ' 401 Authorization Required' -u /nagios/ HTTP CRITICAL - Invalid HTTP response received from host on port 443
Testing this command on other nagios servers they all failed to get a successful test.
Tried:
/usr/lib64/nagios/plugins/check_http -t 30 -I sca1-nms501 -S -f follow -e ' 401 Authorization Required' -u /nagios/ HTTP OK HTTP/1.1 401 Authorization Required - 757 bytes in 0.025 seconds |time=0.025239s;;;0.000000 size=757B;;;0
Hmmmm. So localhost fails, hostname is successful. I picked at the nagios configuration…
The first issue seemed to be that this host was configured to do nagios monitoring on itself. None of the other nagios servers were doing this. To resolve this I set the remote nagios server to monitor the local datacenter nagios server and I confirmed that the local nagios server was already correctly setup to monitor the remote. All good, right?
Not so fast – all of the checks come back failed. All with "CHECK_NRPE: ERROR – COULD NOT COMPLETE SSL HANDSHAKE".
I enabled nrpe debug by changing debug=0 to debug=1 in nrpe.cfg on the local nagios server, the one my remote was trying to reach and failing.
I got:
Aug 31 11:14:13 nms nrpe[9386]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9388]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9390]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9392]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9394]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9398]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9400]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9396]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9402]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9404]: Host 10.30.12.5 is not allowed to talk to us! Aug 31 11:14:13 nms nrpe[9406]: Host 10.30.12.5 is not allowed to talk to us!
Aha! I added the remote nagios server to the allowed_hosts list, restarted nrpe, and the checks turned green.
Methodical but effective today.
—doug