SSH Key Management Explained: Best Practices & More

Infrastructure and DevOps administrators face significant barriers in managing Secure Shell (SSH) keys. An organization may have hundreds, if not thousands, of keys to manage. It takes considerable effort to assign, append, delete, and rotate keys diligently while ensuring no client-server communication breaks during the process. A lack of a streamlined SSH key inventory can lead to many keys being unused or unaccounted for.

Unfortunately, even well-considered plans for SSH key management and automated SSH key managers cannot completely eliminate administrative strain. In this article, we’ll explore the complexities of SSH key management. We’ll also show how to authenticate users effectively without having to manage SSH keys for individual users.

Overview of SSH Keys and Authentication

An improvement over the telnet tool due to encryption, the SSH protocol enables access to remote servers and databases. Common use cases include monitoring, maintenance, file transfers, deployment of code from source repositories onto multiple hosts, remote execution of commands on a server from a client, and data collection from multiple machines to run a job. Initially built for *nix environments, users can access SSH directly from the command prompt with iOS, OS X, and Linux machines. Users can also access SSH from Windows machines using PuTTY or an alternative client and from Android devices using an SSH client app such as JuiceSSH.

SSH Authentication

You can use passwords, private/public keys, and certificates for SSH authentication.

Password-based authentication, the simplest option, validates a combination of username and password against a central directory service. Often organizations pair password authentication with another token, such as key-based authentication.

Key-based authentication, the most widespread option, relies on an asymmetric cipher or encryption algorithm that uses a pair of public and private keys (created with the ssh-keygen utility). The public key functions as the encryption key and the private key functions as the decryption key. When setting up server key pairs, only a select list of administrators in any organization should generate the pair of public and private keys. For user authentication, users should generate their own key pair and provide admins with their public key. This ensures the private key only sits on one person's computer.

You can also use key-based authentication in tandem with passwords. At the time of generating the key pair, the ssh-keygen utility gives the option to enter a passphrase. The passphrase serves as an additional authentication layer and encrypts the private key. When authenticating, the user needs to have the private key and enter the passphrase.

To establish a client-server connection using SSH keys:

• The client sends the public key to the server.
• The server verifies if the client’s public key exists in its authorization list.
• If yes, then the server sends a “challenge” back to the client, encrypting the challenge data using the public key.
• The client can decrypt the “challenge” as it’s the only one with access to the private key.
• The client sends the decrypted data back to the server, verifying its authenticity.
• Now, a secure tunnel is established between the client and the server, which is used for further communications.

Certificate-based authentication uses certificates or public keys stamped with a digital signature of the certificate authority (CA). Instead of explicitly adding public keys to the server authorization list, a client presents a certificate to the server, which verifies if it’s issued by a trusted certificate authority. Since CAs create certificates with an expiration date, administrators must reissue the certificates at appropriate time intervals.

Benefits of SSH Keys

SSH keys are:

• simple to generate
• more secure than passwords due to their dual key structure and asymmetric encryption
• well-suited for machine-to-machine interactions across networks

In the case of remote servers, symmetric encryption is not possible because it’s difficult to handshake on a common key without risking it being discovered. The distribution of public keys that a private key later decrypts, solves this problem.

SSH keys favor many everyday use cases. For instance, SSH keys can allow for a trusted connection to be made between two machines for an automated backup job without requiring an interactive login. It is also used for committing code to version control repositories and inside various systems management tools.

Risks and Challenges of SSH Key Management

Key-based authentication is only as effective as the systems in place to manage your keys. Research shows that in some large organizations up to 90% of authorized keys are no longer used, and 10% of those grant privileged access. SSH key management defines strategies for creating, assigning, and ultimately deleting SSH keys when they are no longer needed. Regardless, organizations still face a number of challenges in key management:

1. Key sprawl: With too many keys to manage, administrators don’t know where to begin. Many organizations don’t have a complete inventory of all their keys, making it difficult to assess which keys they need to delete.

2. Manual effort: In most companies, administrators still manage their keys manually. This is both time-consuming and error-prone. For instance, as SSH keys are created with no expiration date, admins may want to rotate their keys regularly to minimize risks. But they may forget to rotate some keys or forget to distribute the new key to some servers, unwittingly breaking the connection between hosts. In a study by Dimensional Research, 40% of organizations said they don’t rotate keys at all or only do so occasionally.

3. No onboarding and offboarding process: Often, enterprises don’t have clear guidelines or checklists on when to provision or remove keys. Therefore, administrators may add SSH keys to the authorization lists of servers without knowing for certain if the reason for granting that access is valid. 
   
   A lack of guidelines can also lead to incomplete access deprovisioning. For example, a client machine has a program running that uses SSH keys for authentication and executes remote commands on a server. If an admin decides to move this program to run from a different client machine but forgets to deprecate the keys of the previous machine, both client machines will inadvertently continue to have access to the server.

4. No policies and procedures: Admins can’t effectively manage SSH keys without clear policy definition. For example, if you want to reuse server machines for different purposes and reinstall, patch, or upgrade them, you could lose the public key store. You need to integrate key management policies with the rest of the organization’s IT decisions.

5. No audit process: Even in companies that use automated SSH key managers, organizations struggle to audit access. With no trail of who added or deleted keys, when, and why, there is no accountability or ownership for key management.
   
   Due to the increasing federal regulations and compliance procedures such as SOC 2, auditing is not just a nice feature but mandatory to meet compliance goals.

6. Risk of misused keys: While attempting to ease the administrative hassle of using SSH keys, administrators may share a common system account rather than using an individual account, which in turn means they’re using the same key. This shared account increases the risk of keys being misused. For example, if one of these administrators leaves the organization, the person can potentially still access a server that uses the shared key.

Best Practices to Move Beyond SSH Key Management

Organizations of all sizes must reckon with the unwieldiness of SSH key management. You can tackle some of these challenges by governing keys according to best practices, like inventorying keys, rotating keys at fixed intervals, and assigning one key per user.

However, these approaches cannot inherently solve all of the issues with the SSH key lifecycle. For example, a key management policy won’t reduce the effort in the manual provisioning process.

The real question is—rather than managing a large number of keys, can you collapse the amount significantly or even eliminate them altogether? Let’s discuss how a control plane, like StrongDM, can help us implement a few best practices:

1. Centralize access management

Routing access requests through a unified control plane eliminate the need to generate SSH keys for every individual user that requires access to a server. With a control plane, you only need to add the public keys of the intermediary machines that act as proxies to the authorization lists on the servers you need to access.

When users want to access a server or database, the control plane's software-defined network controls and manages their access. The proxy software validates the user’s session, and permissions, then route the session to the target server through the most efficient path. The server will only accept trusted connections from the control plane.

2. Define user onboarding policies

If you configure user authentication and authorizations at the level of the control plane, you can easily define and implement onboarding policies. A control plane integrates with and delegates authentication to your existing identity providers, such as Active Directory or Okta. In cases, where there is no existing directory service or single-sign on provider to leverage, you can define accounts natively within the control plane. Once authenticated, the control plane routes users to the destination database or server, allowing them only to perform authorized actions.

3. Define user offboarding policies

Whether termination, department move, end of business need, or something else, admins need to create policies that guide how to revoke user access. A centralized interface eliminates the need for admins to track down and delete private keys hidden away on servers or personal laptops. From the control plane, you can see all the resources your users have access to and revoke access with a few clicks.

4. Define an audit strategy

Successful implementation of an audit strategy helps with compliance, general monitoring, troubleshooting, and assessing security risks or events. StrongDM offers automatic logging of user activity. You can review logs to see administrative activities, queries, live replays of SSH or RDP sessions, and even search within these captured sessions.

5. Plan for remote work and contractor access

Unless rotated or deleted, SSH keys perpetually allow access to remote servers, which poses challenges for limiting access to vendors, contractors, and remote employees. To terminate or limit access, you only need to make corresponding changes in the control plane. The gateway also helps protect a remote session by leasing temporary credentials and terminating them once the session is complete.

Moving from SSH Keys to a Centralized Control Plane

SSH key management can easily overwhelm you. By following the best practices discussed here — simplifying access management, improving onboarding and offboarding, plus proper auditing and logging — and using a centralized proxy software with a control plane, managing user roles and authentication can become simple and hassle-free.

StrongDM provides a platform for managing access to all of your databases, applications, and servers, allowing you to manage user roles and authentication with minimal effort.

Give StrongDM a try today, or schedule a no-BS demo with our team.