The Complete Guide to SSH Key Management on Linux

SSH keys provide secure, passwordless authentication for remote servers and services like GitHub and GitLab. Understanding SSH key management is essential for any developer or system administrator. This guide covers everything from generating keys to advanced management techniques.

Generating SSH Keys

Modern best practices recommend Ed25519 keys for their security and performance. RSA remains widely compatible if needed.

# Generate Ed25519 key (recommended)
ssh-keygen -t ed25519 -C "your_email@example.com"

# Generate RSA key (for compatibility)
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

# Keys are stored in ~/.ssh/
# Private key: id_ed25519
# Public key: id_ed25519.pub

Copying Keys to Servers

# Using ssh-copy-id (easiest)
ssh-copy-id user@server

# Manual method
cat ~/.ssh/id_ed25519.pub | ssh user@server "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

# Set correct permissions on server
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

SSH Config File

The SSH config file simplifies connections by storing host configurations.

# ~/.ssh/config
Host myserver
    HostName 192.168.1.100
    User admin
    Port 22
    IdentityFile ~/.ssh/id_ed25519

Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/github_ed25519

Host production
    HostName prod.example.com
    User deploy
    IdentityFile ~/.ssh/production_key
    ProxyJump bastion

# Now connect with:
ssh myserver

SSH Agent for Key Management

# Start SSH agent
eval "$(ssh-agent -s)"

# Add key to agent
ssh-add ~/.ssh/id_ed25519

# List added keys
ssh-add -l

# Add to shell profile for automatic start
# ~/.bashrc or ~/.zshrc
if [ -z "$SSH_AUTH_SOCK" ]; then
    eval "$(ssh-agent -s)"
    ssh-add
fi

Multiple Keys for Different Services

Use separate keys for different services to limit exposure if one key is compromised.

# Generate service-specific keys
ssh-keygen -t ed25519 -f ~/.ssh/github_ed25519 -C "github"
ssh-keygen -t ed25519 -f ~/.ssh/work_ed25519 -C "work"
ssh-keygen -t ed25519 -f ~/.ssh/personal_ed25519 -C "personal"

Security Best Practices

• Use Passphrases – Protect private keys with strong passphrases
• Correct Permissions – Private keys should be 600, .ssh directory 700
• Rotate Keys – Periodically generate new keys and remove old ones
• Audit authorized_keys – Review which keys have access to your servers
• Disable Password Auth – On servers, disable password authentication in sshd_config

Troubleshooting

# Verbose connection for debugging
ssh -v user@server

# Check key permissions
ls -la ~/.ssh/

# Test key authentication
ssh -T git@github.com

Conclusion

Proper SSH key management is fundamental to secure system administration. Use strong key types, protect private keys with passphrases, and organize keys by purpose for a maintainable and secure setup.