SSH Key Management Best Practices: Why You Should Never Share Private Keys

Introduction

Secure Shell (SSH) is the backbone of secure remote server administration. Whether you’re managing a single VPS or a fleet of cloud servers, SSH keys provide the cryptographic foundation for authentication. However, one of the most common mistakes I see system administrators make is sharing private keys among team members.

In this comprehensive guide, I’ll explain why sharing private keys is dangerous, and show you how to properly manage multiple user access to your servers using industry best practices.

The Scenario: Multiple Users Need Server Access

Imagine this common situation: You have two team members who need SSH access to your production server. The quick-and-dirty approach might be to generate one key pair and share the private key with both users. This is a critical security mistake.

Why Sharing Private Keys is Dangerous

1. No Accountability

When multiple people share a private key, you lose all ability to track who did what:

# From auth.log - you see the key, but not the person
Feb 23 08:45:12 server sshd[12345]: Accepted publickey for root from 192.168.1.100 port 54321

Was that John deploying code or an attacker who compromised his laptop? With shared keys, you can’t tell.

2. Compromise Spreads Like Wildfire

Scenario	Impact of Shared Key
User’s laptop stolen	Attacker has immediate access to ALL servers
Malware infection	Key can be exfiltrated and used by attackers
Disgruntled employee	Can’t revoke access without breaking everyone’s login
Phishing attack	Stolen key works for all users

3. Revocation Nightmare

When one user leaves the team or loses their laptop:

With shared keys:

# You have to:
1. Generate a new key pair
2. Distribute new private key to ALL remaining users
3. Update public key on ALL servers
4. Pray no one saved the old key

With individual keys:

# One simple command:
sudo sed -i '/john@laptop/d' /home/*/.ssh/authorized_keys

4. Violates Security Best Practices

Major security standards explicitly forbid shared credentials:

• CIS Benchmarks: “Ensure each user has their own unique key”
• PCI DSS: “Assign a unique ID to each person with computer access”
• ISO 27001: “Users shall be uniquely identifiable”
• NIST 800-53: “Implement authenticated access management for unique users”

The Right Way: Individual SSH Keys

How SSH Key Authentication Actually Works

SSH uses public-key cryptography:

• Public key (id_ed25519.pub): Safe to share, can be placed on servers
• Private key (id_ed25519): NEVER SHARE THIS – it’s your digital identity

Step-by-Step: Setting Up Multiple Users Securely

Step 1: Each User Generates Their Own Key Pair

On their local machine (laptop/desktop), each user runs:

On Linux/macOS:

ssh-keygen -t ed25519 -C "john@company.com"

On Windows (PowerShell):

ssh-keygen -t ed25519 -C "jane@company.com"

This creates two files in ~/.ssh/:

• id_ed25519 – 🔒 PRIVATE – KEEP SECRET
• id_ed25519.pub – ✅ PUBLIC – Safe to share

Step 2: Users Send You ONLY Their Public Keys

Each user sends you the contents of their .pub file:

cat ~/.ssh/id_ed25519.pub
# Output: ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHcwkgZ3UC1jVoFkaxNcTIGeGLFdE0uwgW70O70sV7y5 john@company.com

Step 3: Add Keys to the Server

As the server administrator, add each user’s public key:

For existing user john:

# Switch to john's home directory
sudo su - john

# Create .ssh directory with proper permissions
mkdir -p ~/.ssh
chmod 700 ~/.ssh

# Add john's public key
echo "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHcwkgZ3UC1jVoFkaxNcTIGeGLFdE0uwgW70O70sV7y5 john@company.com" >> ~/.ssh/authorized_keys

# Set correct permissions
chmod 600 ~/.ssh/authorized_keys

For user jane:

sudo su – jane

mkdir -p ~/.ssh

chmod 700 ~/.ssh

echo "ssh-ed25519 AAAAB3NzaC1yc2EAAAADAQABAAABAQC8s3Cp1T3+J6hQk2rP5kZ5wX6jLc7VgYwZx+9nYtQx+jane@company.com" >> ~/.ssh/authorized_keys

chmod 600 ~/.ssh/authorized_keys

Step 4: Quick Method Using ssh-copy-id

If users still have password access (temporarily), they can add their own keys:

# From user's local machine
ssh-copy-id -i ~/.ssh/id_ed25519.pub john@your-server.com -p 1786

Automating User Setup with a Script

Save this as add-ssh-user.sh on your server:

#!/bin/bash
# Script to safely add SSH users with their own keys
# Usage: ./add-ssh-user.sh username "public-key-string"

USERNAME=$1
PUBKEY="$2"

# Check arguments
if [ -z "$USERNAME" ] || [ -z "$PUBKEY" ]; then
    echo "❌ Usage: $0 username \"public-key\""
    exit 1
fi

# Create user if they don't exist
if ! id "$USERNAME" &>/dev/null; then
    echo "📝 Creating user $USERNAME..."
    sudo useradd -m -s /bin/bash "$USERNAME"
    sudo passwd -l "$USERNAME"  # Lock password (SSH key only)
else
    echo "✅ User $USERNAME already exists"
fi

# Setup SSH directory
sudo mkdir -p "/home/$USERNAME/.ssh"
echo "$PUBKEY" | sudo tee -a "/home/$USERNAME/.ssh/authorized_keys" > /dev/null

# Fix permissions
sudo chown -R "$USERNAME:$USERNAME" "/home/$USERNAME/.ssh"
sudo chmod 700 "/home/$USERNAME/.ssh"
sudo chmod 600 "/home/$USERNAME/.ssh/authorized_keys"

echo "✅ SSH key added for $USERNAME"
echo "🔒 User can now login with: ssh $USERNAME@your-server.com -p 1786"

Make it executable and run:

chmod +x add-ssh-user.sh
sudo ./add-ssh-user.sh john "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIHcwkgZ3UC1jVoFkaxNcTIGeGLFdE0uwgW70O70sV7y5 john@company.com"

Best Practices for SSH Key Management

1. Always Use Passphrases

# Generate key with passphrase
ssh-keygen -t ed25519 -C "john@company.com" -f ~/.ssh/id_ed25519
# You'll be prompted for a passphrase

2. Use ssh-agent to Remember Passphrases

# Start ssh-agent
eval "$(ssh-agent -s)"

# Add your key (you'll enter passphrase once)
ssh-add ~/.ssh/id_ed25519

# List added keys
ssh-add -l

3. Regular Auditing

Check who has access:

# List all users with SSH keys
for user in $(ls /home); do
    if [ -f "/home/$user/.ssh/authorized_keys" ]; then
        echo "🔑 $user has keys:"
        cat "/home/$user/.ssh/authorized_keys" | awk '{print "   " $3}'
    fi
done

4. Immediate Revocation

When a user leaves:

# Remove their key (using comment as identifier)
sudo sed -i '/john@company.com/d' /home/john/.ssh/authorized_keys

# Or disable their account entirely
sudo usermod -L john  # Lock password
sudo chmod 600 /home/john/.ssh/authorized_keys  # Prevent new logins

Emergency Scenario: When You REALLY Need Shared Access

In true emergencies (like 3 AM and a user lost their key), use a temporary shared account:

# Create emergency account with strong password
sudo useradd -m -s /bin/bash emergency-access
sudo passwd emergency-access  # Set strong temporary password

# Add note to self
echo "Emergency access granted to John at $(date) for laptop issue" | sudo tee -a /var/log/emergency-access.log

# Revoke immediately after use
sudo userdel -r emergency-access

Monitoring SSH Access

Set up monitoring to track who logs in:

# Watch auth.log in real-time
sudo tail -f /var/log/auth.log | grep "Accepted publickey"

# Create login notification script
sudo nano /usr/local/bin/ssh-notify.sh

#!/bin/bash
echo "SSH Login: User $PAM_USER from $PAM_RHOST at $(date)" | \
    mail -s "SSH Login Alert" admin@company.com

Add to /etc/pam.d/sshd:

session optional pam_exec.so /usr/local/bin/ssh-notify.sh

Summary: Key Takeaways

✅ Do	❌ Don’t
Each user generates their own key pair	Share private keys
Use ed25519 keys (strongest)	Use deprecated RSA keys under 4096 bits
Add passphrases to keys	Leave keys without passphrases
Revoke keys when users leave	Leave former employees with access
Audit access regularly	Assume everything is fine
Use ssh-agent for convenience	Re-enter passphrases constantly
Monitor auth.log	Ignore who’s logging in

Conclusion

SSH key management doesn’t have to be complicated. By following these best practices—giving each user their own key pair, using passphrases, and regularly auditing access—you maintain security, accountability, and flexibility.

Remember: Your private key is your digital identity. Treat it like your house key—you wouldn’t make copies for strangers, and you’d change the locks if one went missing.

The extra few minutes it takes to set up individual keys saves hours of cleanup and potential security breaches down the road. Start implementing these practices today, and your future self (and your security auditor) will thank you.

---

Have questions about SSH key management? Need help securing your servers? Drop a comment below!