Tuesday, March 19, 2013

Verifying SSH Key Fingerprint and More

We have two different users on a remote server and wish to set up public key authentication over SSH for both of them.

At beginning, we were able to connect to user bench, but not user aime1.  This has stirred up our investigation into the issue. Yes, SSH does support multiple user authentications over SSH. It turns out the culprit was that we have forgotten to change the access permission of authorized_keys file for the user aime1:

$chmod 600 authorized_keys

In this article, we will show you:
  • How to investigate ssh connection issues?
  • What is going on underneath the ssh session?

The Solution


As described in [1], you need to copy the content of id_rsa.pub from the local server to the authorized_keys file at the remote server (note that .ssh folders are located differently with different users) .  Also, you need to change the authorized_keys' permission to 600.  If you forgot to do latter, SSH won't automatically authenticate local server to remote server when local server offers its RSA public key. Note that we have chosen RSA, instead DSA, as the authentication keys over SSH.

What's Going on Underneath?


To debug any connection issue, you can add -v option as shown below:

$ ssh -v aime1@remoteServer
OpenSSH_6.0p1, OpenSSL 1.0.1c 10 May 2012
debug1: Reading configuration data HKEY_CURRENT_USER/SOFTWARE/Mortice Kern Syste
ms/etc/ssh_config
debug1: Reading configuration data HKEY_LOCAL_MACHINE/SOFTWARE/Mortice Kern Syst
ems/etc/ssh_config
debug1: Connecting to remoteServer [10.133.188.166] port 22.
debug1: Connection established.
debug1: identity file /C=/Documents and Settings/aroot/.ssh/id_rsa type 1
debug1: identity file /C=/Documents and Settings/aroot/.ssh/id_rsa-cert type -1
debug1: identity file /C=/Documents and Settings/aroot/.ssh/id_dsa type -1
debug1: identity file /C=/Documents and Settings/aroot/.ssh/id_dsa-cert type -1
debug1: identity file /C=/Documents and Settings/aroot/.ssh/id_ecdsa type -1
debug1: identity file /C=/Documents and Settings/aroot/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_4.3
debug1: match: OpenSSH_4.3 pat OpenSSH_4*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.0
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024 p="" sent="">debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA 
97:a2:85:f3:f1:28:ab:c2:70:df:58:2f:c3:61:65:9a

debug1: Host 'remoteServer' is known and matches the RSA host key.
debug1: Found key in /C=/Documents and Settings/aroot/.ssh/known_hosts:1
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /C=/Documents and Settings/aroot/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Trying private key: /C=/Documents and Settings/aroot/.ssh/id_dsa
debug1: Trying private key: /C=/Documents and Settings/aroot/.ssh/id_ecdsa
debug1: Next authentication method: password
aime1@remoteServer's password:

As shown above, local server failed to automatically authenticated itself to the remote server (i.e., SSH still prompted user for the password).  In the middle of output, the following three lines are interesting:

debug1: Server host key: RSA 97:a2:85:f3:f1:28:ab:c2:70:df:58:2f:c3:61:65:9a
debug1: Host 'remoteServer' is known and matches the RSA host key.
debug1: Found key in /C=/Documents and Settings/aroot/.ssh/known_hosts:1

What happened is that the fingerprint of remote server's RSA public key was returned and it matched one entry stored in the known_hosts file of the local server.  In public-key cryptography, a public key fingerprint is a short sequence of bytes used to authenticate or look up a longer public key[5]. Fingerprints are created by applying a cryptographic hash function to a public key. Since fingerprints are shorter than the keys they refer to, they can be used to simplify certain key management tasks.

The RSA public key from the remote server (i.e., a Linux box) is stored here:
  • /etc/ssh/ssh_host_rsa_key.pub

To verify that, you can run:

$ssh-keygen -l -f /etc/ssh/ssh_host_rsa_key.pub

2048 97:a2:85:f3:f1:28:ab:c2:70:df:58:2f:c3:61:65:9a /etc/ssh/ssh_host_rsa_key.pub

or you can try:

[aime1@remoteServer ~]$ ssh localhost
he authenticity of host 'localhost (127.0.0.1)' can't be established.
RSA key fingerprint is 97:a2:85:f3:f1:28:ab:c2:70:df:58:2f:c3:61:65:9a.


This remote server's finger print was matched against entries stored in the local server's known_hosts file.  To verity it, you can run:

$ ssh-keygen -l -f known_hosts
2048 97:a2:85:f3:f1:28:ab:c2:70:df:58:2f:c3:61:65:9a remoteServer,10.133.188.166 (RSA)


In the verbose output, it also told us that it matched the first entry on the known hosts list.

Later, the output also told us that something was not quite working:

debug1: Authentications that can continue: publickey,gssapi-with-mic,password
debug1: Trying private key: /C=/Documents and Settings/aroot/.ssh/id_dsa
debug1: Trying private key: /C=/Documents and Settings/aroot/.ssh/id_ecdsa
debug1: Next authentication method: password
!> aime1@remoteServer's password:

SSH on the local server failed to authenticate itself to the remote server with its offered RSA key.  So, it tried the following other ways of authentication in sequence:
  • id_dsa
  • id_ecdsa
  • password

Without much ado, we know what went wrong and have fixed the issue.  Then when we tried it again, it was authenticated automatically.  From the verbose output, we can tell the difference (i.e., it succeeded to authenticate with RSA key):

debug1: Server accepts key: pkalg ssh-rsa blen 277
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).
Authenticated to remoteServer ([10.133.188.166]:22).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
Last login: Mon Mar 18 18:30:26 2013 from localServer


Recap


We need to change the file access permission of authorized_keys as shown below:

Before
-rw-rw-r-- 1 aime1 svrtech  396 Mar 19 08:48 authorized_keys

After
-rw------- 1 aime1 svrtech  396 Mar 19 08:48 authorized_keys


After the change, we can now establish ssh session without being asked for password:

$ ssh aime1@remoteServer
Last login: Mon Mar 18 20:17:51 2013 from localServer
[aime1@remoteServer ~]$

Where Are Server's Authentication Keys Stored?


If your on a Linux box, they are probably stored in the following files:
  • RSA
    • /etc/ssh/ssh_host_rsa_key.pub 
  • DSA 
    • /etc/ssh/ssh_host_dsa_key.pub). 

If you're on a Mac OSX, they are stored in one of the following files:
  • /etc/ssh_host_rsa_key.pub
  • /etc/ssh_host_key.pub
  • /etc/ssh_host_dsa_key.pub

No comments: