sshd running but no PID file

Posted by dunxd on Server Fault See other posts from Server Fault or by dunxd
Published on 2012-02-27T10:15:28Z Indexed on 2012/04/14 17:31 UTC
Read the original article Hit count: 337

Filed under:
|
|
|

I'm recently started using monit to monitor the status of sshd on my CentOS 5.4 server. This works fine, but every so often monit reports that sshd is no longer running. This isn't true - I am still able to login to the server via ssh, however I note the following:

  • There is no longer any PID file at /var/run/sshd.pid - after a reboot this file exists. Once it is gone, restarting sshd via service sshd restart does not create the PID file.
  • sudo service sshd status reports openssh-daemon is stopped - again, restarting sshd does not change this, but a reboot does.
  • sudo service sshd stop reports failed, presumably because of the missing PID file.

Any idea what is going on?

Update

sudo netstat -lptun gives the following output relating to port 22

tcp        0      0 :::22      :::*    LISTEN      20735/sshd

Killing the process with this PID as suggested by @Henry and then starting sshd via service results in service sshd status recognising the process by PID again. Would still like to understand this better.

RPM verify suggested by a couple of answerers shows this:

sudo rpm -vV openssh openssh-server openssh-clients | grep 'S\.5'
S.5....T  c /etc/pam.d/sshd
S.5....T  c /etc/ssh/sshd_config

/etc/pam.d/sshd has the following contents:

#%PAM-1.0
auth       include      system-auth
account    required     pam_nologin.so
account    include      system-auth
password   include      system-auth
session    optional     pam_keyinit.so force revoke
session    include      system-auth
#session    required     pam_loginuid.so

Should that last line be commented out?

Update Here's the output of @YannickGirouard 's script:

$ sudo ./sshd_test
Searching for the process listening on port 22...

Found the following PID: 21330

Command line for PID 21330: /usr/sbin/sshd

Listing process(es) relating to PID 21330:

UID        PID  PPID  C STIME TTY          TIME CMD
root     21330     1  0 14:04 ?        00:00:00 /usr/sbin/sshd

Listing RPM information about openssh packages:

Name        : openssh                      Relocations: (not relocatable)
Version     : 4.3p2                             Vendor: CentOS
Release     : 72.el5_7.5                    Build Date: Tue 30 Aug 2011 12:34:14 AM BST
Install Date: Sun 06 Nov 2011 12:50:57 AM GMT      Build Host: builder10.centos.org
Group       : Applications/Internet         Source RPM: openssh-4.3p2-72.el5_7.5.src.rpm
Size        : 745390                           License: BSD
Signature   : DSA/SHA1, Fri 02 Sep 2011 01:13:01 AM BST, Key ID a8a447dce8562897
URL         : http://www.openssh.com/portable.html
Summary     : The OpenSSH implementation of SSH protocol versions 1 and 2

------------------------------------------------------

Name        : openssh-clients              Relocations: (not relocatable)
Version     : 4.3p2                             Vendor: CentOS
Release     : 72.el5_7.5                    Build Date: Tue 30 Aug 2011 12:34:14 AM BST
Install Date: Sun 06 Nov 2011 12:51:04 AM GMT      Build Host: builder10.centos.org
Group       : Applications/Internet         Source RPM: openssh-4.3p2-72.el5_7.5.src.rpm
Size        : 871132                           License: BSD
Signature   : DSA/SHA1, Fri 02 Sep 2011 01:13:01 AM BST, Key ID a8a447dce8562897
URL         : http://www.openssh.com/portable.html
Summary     : The OpenSSH client applications

------------------------------------------------------

Name        : openssh-server               Relocations: (not relocatable)
Version     : 4.3p2                             Vendor: CentOS
Release     : 72.el5_7.5                    Build Date: Tue 30 Aug 2011 12:34:14 AM BST
Install Date: Sun 06 Nov 2011 12:51:04 AM GMT      Build Host: builder10.centos.org
Group       : System Environment/Daemons    Source RPM: openssh-4.3p2-72.el5_7.5.src.rpm
Size        : 492478                           License: BSD
Signature   : DSA/SHA1, Fri 02 Sep 2011 01:13:01 AM BST, Key ID a8a447dce8562897
URL         : http://www.openssh.com/portable.html
Summary     : The OpenSSH server daemon

------------------------------------------------------

However, I've since got things working by killing the process and starting afresh, as suggested by @Henry below, so perhaps I am no longer seeing the same thing. Will try again if I am seeing the issue again after next reboot.

Update - 14 March Monit alerted me that sshd had disappeared, and again I am able to ssh onto the server. So now I can run the script

$ sudo ./sshd_test
Searching for the process listening on port 22...

Found the following PID: 2208

Command line for PID 2208: /usr/sbin/sshd

Listing process(es) relating to PID 2208:

UID        PID  PPID  C STIME TTY          TIME CMD
root      2208     1  0 Mar13 ?        00:00:00 /usr/sbin/sshd
root      1885  2208  0 21:50 ?        00:00:00 sshd: dunx [priv]

Listing RPM information about openssh packages:

Name        : openssh                      Relocations: (not relocatable)
Version     : 4.3p2                             Vendor: CentOS
Release     : 72.el5_7.5                    Build Date: Tue 30 Aug 2011 12:34:14 AM BST
Install Date: Sun 06 Nov 2011 12:50:57 AM GMT      Build Host: builder10.centos.org
Group       : Applications/Internet         Source RPM: openssh-4.3p2-72.el5_7.5.src.rpm
Size        : 745390                           License: BSD
Signature   : DSA/SHA1, Fri 02 Sep 2011 01:13:01 AM BST, Key ID a8a447dce8562897
URL         : http://www.openssh.com/portable.html
Summary     : The OpenSSH implementation of SSH protocol versions 1 and 2

------------------------------------------------------

Name        : openssh-clients              Relocations: (not relocatable)
Version     : 4.3p2                             Vendor: CentOS
Release     : 72.el5_7.5                    Build Date: Tue 30 Aug 2011 12:34:14 AM BST
Install Date: Sun 06 Nov 2011 12:51:04 AM GMT      Build Host: builder10.centos.org
Group       : Applications/Internet         Source RPM: openssh-4.3p2-72.el5_7.5.src.rpm
Size        : 871132                           License: BSD
Signature   : DSA/SHA1, Fri 02 Sep 2011 01:13:01 AM BST, Key ID a8a447dce8562897
URL         : http://www.openssh.com/portable.html
Summary     : The OpenSSH client applications

------------------------------------------------------

Name        : openssh-server               Relocations: (not relocatable)
Version     : 4.3p2                             Vendor: CentOS
Release     : 72.el5_7.5                    Build Date: Tue 30 Aug 2011 12:34:14 AM BST
Install Date: Sun 06 Nov 2011 12:51:04 AM GMT      Build Host: builder10.centos.org
Group       : System Environment/Daemons    Source RPM: openssh-4.3p2-72.el5_7.5.src.rpm
Size        : 492478                           License: BSD
Signature   : DSA/SHA1, Fri 02 Sep 2011 01:13:01 AM BST, Key ID a8a447dce8562897
URL         : http://www.openssh.com/portable.html
Summary     : The OpenSSH server daemon

------------------------------------------------------

Again, when I look for /var/run/sshd.pid I don't find it.

$ cat /var/run/sshd.pid
cat: /var/run/sshd.pid: No such file or directory
$ sudo netstat -anp | grep sshd
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      2208/sshd
$ sudo kill 2208
$ sudo service sshd start
Starting sshd:                                             [  OK  ]
$ cat /var/run/sshd.pid
3794
$ sudo service sshd status
openssh-daemon (pid  3794) is running...

Is it possible that sshd is restarting and not creating a pidfile for some reason?

© Server Fault or respective owner

Related posts about centos

Related posts about ssh