SSH Security Best Practices:Keys, Users and Fail2ban

To secure SSH, use key-based authentication and disable password logins, block direct root login, allow only the users who need access, restrict where connections can come from, keep OpenSSH updated, and protect your private keys. With password authentication off, brute-force attacks against SSH cannot succeed, however many attempts are made.

These practices apply to any Linux VPS or dedicated server. Step-by-step key setup is in how to set up SSH key authentication.

1. Use keys instead of passwords

SSH keys are cryptographic key pairs: the private key stays on your computer; the public key goes on the server. They are far stronger than passwords and cannot be guessed.

• Prefer Ed25519 keys (ssh-keygen -t ed25519).
• Protect the private key with a passphrase.
• Use an SSH agent so you type the passphrase once per session.

2. Disable password and root login

In /etc/ssh/sshd_config (or a file in /etc/ssh/sshd_config.d/):

PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
PubkeyAuthentication yes

Log in as a normal user and use sudo for administrative tasks. This leaves an audit trail and means attackers must guess a username as well.

3. Limit who can log in

• Allow only specific users or groups: AllowUsers deploy admin or AllowGroups sshusers.
• Remove accounts that no longer need access, and their keys from ~/.ssh/authorized_keys.
• Give each person their own account and key; do not share keys.

4. Restrict network access

• Allow SSH only from trusted IP addresses (your office, a VPN or a bastion host) in the firewall where practical. See Linux firewall basics.
• Use your provider's network firewall as an extra layer.

Changing the port from 22 reduces automated noise in logs but is not a security control by itself.

5. Add Fail2ban

Fail2ban bans IP addresses after repeated failed attempts. With key-only authentication it mainly reduces log noise and load, and it also protects other services such as mail logins. See brute-force attack prevention.

6. Keep OpenSSH and the server updated

Apply security updates promptly; vulnerabilities in SSH or its libraries are rare but serious when they happen. Enable automatic security updates; see the Linux server hardening checklist.

7. Use sensible session settings

• MaxAuthTries 3 limits attempts per connection.
• LoginGraceTime 30 shortens how long an unauthenticated connection can stay open.
• ClientAliveInterval and ClientAliveCountMax close dead sessions.
• X11Forwarding no unless you need it.

Modern OpenSSH defaults to strong ciphers; avoid re-enabling old algorithms for compatibility with outdated clients.

8. Protect your private keys

• Never copy private keys to servers or share them by email or chat.
• Store them only on devices you control, encrypted with a passphrase.
• Consider hardware-backed keys (security keys supporting ed25519-sk) for high-value access.
• Remove a person's public keys from every server when they leave.

9. Monitor access

• Review authentication logs (journalctl -u ssh or /var/log/auth.log) for unusual logins.
• Alert on logins from unexpected locations or at unusual times.
• Keep logs off the server for forensic value.

Example: a hardened sshd configuration file

Rather than editing the main configuration, many administrators add a drop-in file, for example /etc/ssh/sshd_config.d/50-hardening.conf:

PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no
PubkeyAuthentication yes
AuthenticationMethods publickey
AllowGroups sshusers
MaxAuthTries 3
LoginGraceTime 30
ClientAliveInterval 300
ClientAliveCountMax 2
X11Forwarding no

Then add administrators to the group (sudo usermod -aG sshusers deploy), test with sudo sshd -t, reload, and confirm a new login works before closing your session. Check that your distribution's main configuration includes the sshd_config.d directory; most current releases do.

Auditing who can log in

Periodically list what grants access:

# Users with login shells
grep -E '/(bash|sh|zsh)$' /etc/passwd

# Authorised keys for each user
sudo find /home /root -name authorized_keys -exec sh -c 'echo "== $1"; cat "$1"' _ {} \;

Every key should belong to a person or system you can name. Remove anything unknown, and record who each remaining key belongs to (the comment at the end of each key line helps).

10. Keep a recovery path

Before tightening SSH, make sure you know how to use your provider's web console or rescue mode, so a configuration mistake does not lock you out permanently.

Frequently Asked Questions

Are SSH keys really safer than long passwords?

Yes. They cannot be guessed or brute-forced, and the private key never travels to the server.

Should I disable SSH entirely?

On servers managed only through a control panel or automation, you can restrict SSH heavily, but most servers need it for administration and recovery.

What is a bastion host?

A hardened server that is the only machine allowed to SSH into your other servers, centralising access control and logging.

Related reading

SSH hardening is part of website security basics for server owners. A ServerNeed managed VPS includes security hardening and firewall configuration; you still decide who gets access and keep your own keys safe.

Sources

• OpenSSH manual: sshd_config
• Ubuntu Server documentation: OpenSSH server