How to Secure SSH Access to Your Server

SSH (Secure Shell) is the front door to almost every Linux server. It encrypts your connection, so nobody can read your session in transit, but the protocol cannot protect you from a guessable password, a stolen private key or an over-generous configuration. Every internet-facing SSH port sees a steady stream of automated login attempts. This guide shows how to secure SSH step by step, in an order that will not lock you out of your own server.

Before you secure SSH: keep a session open

Before changing anything SSH-related, open two terminal sessions to the server. Make changes in one, test logging in with a fresh third session, and only close the original once the new login works. If you are on a cloud VM, also know how to reach the provider's web console, which works even if SSH does not.

Step 1: Switch to key-based authentication

An SSH key pair consists of a private key, which stays on your computer, and a public key, which you place on the server. Keys are effectively impossible to brute-force, unlike passwords.

On your own computer (Linux, macOS, or Windows 10/11 with the built-in OpenSSH client):

ssh-keygen -t ed25519 -C "alice@laptop"

Accept the default file location and set a passphrase. The passphrase encrypts the private key on disk, so a stolen laptop does not mean a stolen server. An SSH agent (ssh-agent, or the macOS Keychain or Windows OpenSSH Authentication Agent service) remembers it so you do not retype it constantly.

Copy the public key to the server:

ssh-copy-id -i ~/.ssh/id_ed25519.pub alice@203.0.113.25

On Windows without ssh-copy-id, append the contents of the .pub file to ~/.ssh/authorized_keys on the server. Permissions matter: ~/.ssh should be 700 and authorized_keys 600, or sshd may refuse to use them.

Step 2: Tighten the sshd configuration

Once key login works, change the server settings. On current Ubuntu, Debian and RHEL-family releases, put your settings in a drop-in file that loads before the vendor ones, such as /etc/ssh/sshd_config.d/00-secure.conf:

# Authentication
PermitRootLogin no
PasswordAuthentication no
KbdInteractiveAuthentication no
PubkeyAuthentication yes
PermitEmptyPasswords no
AuthenticationMethods publickey

# Limit who and how
AllowGroups sshusers
MaxAuthTries 3
LoginGraceTime 30
MaxStartups 10:30:60

# Reduce features you do not use
X11Forwarding no
AllowAgentForwarding no
AllowTcpForwarding no
ClientAliveInterval 300
ClientAliveCountMax 2

Create the group and add administrators to it before reloading, or nobody will be allowed in:

sudo groupadd sshusers
sudo usermod -aG sshusers alice
sudo sshd -t && sudo systemctl reload ssh    # 'sshd' on RHEL family

sshd -t checks the configuration for errors before you apply it. If you need port forwarding for tunnels or deployment tools, enable AllowTcpForwarding only for specific users with a Match User block. To see the settings sshd is actually using after all files are combined, run sudo sshd -T.

Step 3: Restrict who can reach port 22

The strongest protection is not exposing SSH to the whole internet. Options, from simplest to strongest:

• Firewall allow-list: permit port 22 only from your office's fixed IP address. With UFW: sudo ufw allow from 198.51.100.7 to any port 22 proto tcp. Find your current public address with our What is my IP tool.
• Cloud security groups: apply the same restriction at the provider level, so traffic is dropped before it reaches the server.
• VPN: expose SSH only on a private network reachable through WireGuard, OpenVPN or a similar VPN.
• Provider access services: tools such as AWS Systems Manager Session Manager, Azure Bastion or Google Cloud IAP let you reach servers without any open inbound SSH port.

Confirm from outside that port 22 is closed to the public with our port checker.

What about changing the SSH port?

Moving SSH from port 22 to, say, 2222 cuts down log noise from basic scanners, but it is not real security; a port scan finds it quickly. It is harmless if you want quieter logs, but do it in addition to the measures above, not instead of them. Remember to update the firewall (and the SELinux port label on RHEL-family systems) before switching.

Step 4: Block repeated failures

Fail2ban reads authentication logs and temporarily bans IP addresses with repeated failures. A minimal jail in /etc/fail2ban/jail.local:

[sshd]
enabled  = true
maxretry = 5
findtime = 10m
bantime  = 1h

With password authentication disabled, failed attempts cannot succeed anyway, but bans reduce noise and resource use.

Step 5: Add a second factor for sensitive servers

For servers holding sensitive data, you can require both a key and a time-based one-time code. The libpam-google-authenticator package (Debian/Ubuntu) or google-authenticator from EPEL provides this. After configuring PAM, set:

KbdInteractiveAuthentication yes
AuthenticationMethods publickey,keyboard-interactive

Hardware security keys are another strong option: OpenSSH supports FIDO2 keys with ssh-keygen -t ed25519-sk, so the private key cannot be used without the physical device. Test second-factor setups carefully with a session still open.

Step 6: Use a jump host for many servers

If you manage several servers, expose only one hardened bastion (jump host) and reach the others through it. OpenSSH makes this simple in ~/.ssh/config:

Host bastion
    HostName bastion.example.com
    User alice

Host app-*
    ProxyJump bastion
    User alice

Then ssh app-01 connects through the bastion automatically, and the application servers need no public SSH access at all.

Step 7: Audit keys and logins regularly

• Review every authorized_keys file monthly and remove keys belonging to former staff or retired laptops.
• Label keys with a comment identifying the person and device so they are easy to audit.
• Check recent logins with last and authentication logs with journalctl -u ssh (or sshd).
• For larger teams, consider SSH certificates signed by a central authority, which expire automatically.

The sshd_config manual documents every option. If you would like SSH access designed and audited across your servers, it is part of our server management service.

Key takeaways

• Use Ed25519 keys protected by a passphrase, then disable password and root login.
• Limit SSH to named groups and restrict port 22 by IP, VPN or a provider access service.
• Add Fail2ban, and a second factor or hardware key for sensitive servers.
• Use a jump host for fleets and audit authorized keys regularly.