I setup RSA authentication on one of my Ubuntu servers, however after every restart, I can't log-in via ssh RSA. In order to log-in with ssh I need to first log-in via console, then the RSA starts working. Why???

Below are my sshd config file as well as an output from the ssh -vv command before console log-in and after. .

Before console log-in:

debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /Users/smorhaim/.ssh/smorhaim (0x7ff8d8c242c0)
debug2: key: /Users/smorhaim/.ssh/id_rsaadmin (0x7ff8d8c24cf0)
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/smorhaim/.ssh/smorhaim
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey
debug1: Offering RSA public key: /Users/smorhaim/.ssh/id_rsaadmin
debug2: we sent a publickey packet, wait for reply
debug1: Authentications that can continue: publickey
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey).

After console log-in:

debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /Users/smorhaim/.ssh/smorhaim (0x7f91c14242c0)
debug2: key: /Users/smorhaim/.ssh/id_rsaadmin (0x7f91c1424ae0)
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /Users/smorhaim/.ssh/smorhaim
debug2: we sent a publickey packet, wait for reply
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug2: input_userauth_pk_ok: fp b1:d5:90:43:be:43:52:a9:7f:05:c7:04:86:57:b3:ff
debug1: Authentication succeeded (publickey).
Authenticated to 10.10.30.151 ([10.10.30.151]:22).

sshd config:

Port 22
Protocol 2
ListenAddress 10.10.30.151

UsePrivilegeSeparation yes
SyslogFacility AUTHPRIV
PermitRootLogin no
PasswordAuthentication no
ChallengeResponseAuthentication no
UsePAM yes
X11Forwarding yes