SSH login without password in Linux

Introduction

SSH (Secure Shell) is the backbone of secure remote server management in Linux systems. While traditional password-based authentication works, it has significant drawbacks: passwords can be weak, intercepted, or brute-forced. SSH key-based authentication eliminates these vulnerabilities by using cryptographic key pairs, providing a more secure and convenient way to access remote servers.

Passwordless SSH authentication is essential for system administrators managing multiple servers, automating deployment pipelines, running scheduled tasks across servers, and implementing secure DevOps workflows. Once configured, you can seamlessly connect to remote systems without entering passwords, while maintaining – or even improving – your security posture.

This comprehensive guide walks you through setting up passwordless SSH authentication, understanding the underlying security mechanisms, troubleshooting common issues, and implementing best practices for production environments.

How SSH Key Authentication Works

SSH key authentication uses asymmetric cryptography with two related keys:

• Private Key: Stored securely on your local machine (never shared)
• Public Key: Copied to remote servers you want to access

When you connect to a remote server, SSH uses your private key to prove your identity without transmitting the key itself. The server verifies your identity using the corresponding public key. This cryptographic handshake is virtually impossible to intercept or forge, making it far more secure than password authentication.

Prerequisites

Before setting up passwordless SSH authentication, ensure you have:

• SSH client installed on your local machine (usually pre-installed on Linux)
• SSH server running on the remote host (openssh-server package)
• User account access on both local and remote systems
• Network connectivity between the systems
• Basic understanding of Linux command line

Step-by-Step Setup Guide

Normally, we will login to the remote server using user name and password at the time of authentication.

But here, we can login to the remote server without knowing user name and password of the server using ssh keys. This will be helpful when we need to pull some information or do some task in multiple servers at a time from a particular server. This can be done in 3 simple steps.

In this example, I’m using node1 and node2 servers, will login to node2 without providing any password from node1.

Step 1: Generate SSH Key Pair on Local Machine

Generate the RSA key in node1 as a root user. Accept the default values.

[root@node1 ~]# ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): <Enter>
Enter passphrase (empty for no passphrase): <Enter>
Enter same passphrase again: <Enter>
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
ff:bc:21:ed:c2:03:fa:0c:1d:3e:ab:60:e6:56:fc:ee root@node1

Understanding the options:

• -t rsa: Specifies RSA encryption algorithm (you can also use ed25519 for better security)
• -b 4096: Optional – increases key size to 4096 bits for enhanced security
• Passphrase: Optional extra layer of protection for your private key

Step 2: Copy Public Key to Remote Server

Copy the file /root/.ssh/id_rsa.pub key to node2.

[root@node1 Desktop]# ssh-copy-id -i /root/.ssh/id_rsa.pub root@node2

root@ramesh's password:
Now try logging into the machine, with "ssh 'root@ramesh'", and check in:

.ssh/authorized_keys

to make sure we haven't added extra keys that you weren't expecting.

The ssh-copy-id command automatically:

• Creates ~/.ssh directory on remote server if it doesn’t exist
• Sets correct permissions (700 for directory, 600 for authorized_keys file)
• Appends your public key to ~/.ssh/authorized_keys

Step 3: Test Passwordless SSH Connection

Now you can login to node2 from node1 as root user without providing any password.

[root@node1 Desktop]# ssh node2
Last login: Tue Nov  8 15:39:32 2016 from 192.168.123.1
[root@node2 ~]# uptime
15:42:21 up 5 days, 18:01,  2 users,  load average: 0.00, 0.00, 0.00

If you’re prompted for a password, something went wrong. Continue to the troubleshooting section below.

Manual Key Copy Method

If ssh-copy-id is not available, you can manually copy the key:

# Display your public key
cat ~/.ssh/id_rsa.pub

# SSH to remote server
ssh user@remote-server

# Create .ssh directory if needed
mkdir -p ~/.ssh
chmod 700 ~/.ssh

# Append your public key to authorized_keys
echo "your-public-key-here" >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Security Best Practices

Protect Your Private Key

• Never share your private key – Only public keys should leave your system
• Use passphrases – Add extra protection to private keys
• Set strict permissions: chmod 600 ~/.ssh/id_rsa
• Store keys securely – Consider encrypted file systems or hardware tokens

Server-Side Security

Edit /etc/ssh/sshd_config on your server:

# Disable password authentication
PasswordAuthentication no

# Disable root login (use sudo instead)
PermitRootLogin no

# Enable public key authentication
PubkeyAuthentication yes

# Restart SSH service
systemctl restart sshd

Use Strong Key Types

# Modern Ed25519 keys (recommended)
ssh-keygen -t ed25519 -C "your_email@example.com"

# RSA with 4096 bits
ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Advanced Configuration

SSH Config File for Multiple Servers

Create ~/.ssh/config to manage multiple servers easily:

Host webserver
    HostName 192.168.1.100
    User admin
    IdentityFile ~/.ssh/id_rsa_webserver
    Port 2222

Host dbserver
    HostName db.example.com
    User dbadmin
    IdentityFile ~/.ssh/id_rsa_database

Host *
    ServerAliveInterval 60
    ServerAliveCountMax 3

Now connect simply with: ssh webserver

Managing Multiple SSH Keys

# Generate key for specific purpose
ssh-keygen -t ed25519 -f ~/.ssh/id_ed25519_github -C "github-account"

# Use specific key
ssh -i ~/.ssh/id_ed25519_github git@github.com

# Add key to ssh-agent
eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519_github

Troubleshooting Common Issues

1. Still Prompted for Password

Solution: Check permissions on remote server

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
ls -la ~/.ssh/

2. Permission Denied (publickey)

Solution: Verify public key is in authorized_keys

cat ~/.ssh/authorized_keys | grep "$(cat ~/.ssh/id_rsa.pub)"

3. SSH Key Not Being Used

Solution: Use verbose mode to debug

ssh -vvv user@remote-server
# Look for "Offering public key" messages

4. SELinux Blocking SSH Keys

Solution: Restore SELinux context

restorecon -R -v ~/.ssh

5. Home Directory on NFS

Solution: SSH doesn’t allow keys from world-writable locations

# Move keys to local filesystem
# Or disable StrictModes in sshd_config (not recommended)

6. Wrong Key Being Used

Solution: Specify key explicitly

ssh -i ~/.ssh/specific_key user@remote-server

7. Public Key Authentication Disabled

Solution: Enable in /etc/ssh/sshd_config

PubkeyAuthentication yes
# Restart: systemctl restart sshd

Automating Tasks with Passwordless SSH

Remote Command Execution

# Run single command
ssh server "uptime"

# Run multiple commands
ssh server "cd /var/log && tail -f syslog"

# Copy files with scp
scp file.txt server:/tmp/

# Sync directories with rsync
rsync -avz /local/dir/ server:/remote/dir/

Parallel Execution on Multiple Servers

#!/bin/bash
for server in web1 web2 db1 db2; do
    ssh $server "df -h" &
done
wait

Frequently Asked Questions

What is the difference between password and SSH key authentication?

Password authentication requires you to enter a password each time you connect, which can be intercepted or brute-forced. SSH key authentication uses cryptographic key pairs – your private key proves your identity without transmitting secrets over the network, making it far more secure and convenient.

How do I add a passphrase to an existing SSH key?

Use the ssh-keygen -p command to change or add a passphrase to your existing private key. This adds an extra layer of security without regenerating the key pair.

Can I use the same SSH key for multiple servers?

Yes, you can copy the same public key to multiple servers. However, for better security isolation, consider using separate keys for different environments (development, production) or purposes (GitHub, AWS, internal servers).

How do I disable password authentication completely?

Edit /etc/ssh/sshd_config, set PasswordAuthentication no and ChallengeResponseAuthentication no, then restart SSH service with systemctl restart sshd. Ensure key authentication works first!

What should I do if I lose my private key?

You cannot recover a lost private key. You must generate a new key pair and update the public key on all servers where you had access. This is why backing up your ~/.ssh directory securely is important.

How do I revoke SSH key access for a user?

Remove the user’s public key entry from ~/.ssh/authorized_keys on the server. Each key is on a separate line, so delete the appropriate line and save the file.

Can I use SSH keys with root user?

Yes, but it’s not recommended. Instead, use a regular user with SSH key authentication and configure sudo access for administrative tasks. Set PermitRootLogin no in sshd_config for better security.

How long should my SSH key be?

For RSA keys, use at least 2048 bits (4096 recommended). For Ed25519 keys, the default 256 bits is sufficient and more secure than RSA 2048. Ed25519 is the recommended choice for new keys.

Why does ssh-copy-id ask for a password?

The first time you set up key authentication, ssh-copy-id needs to use password authentication to copy your public key to the remote server. After that, subsequent connections will use key authentication without passwords.

How can I use different keys for different servers automatically?

Create an SSH config file at ~/.ssh/config and specify different IdentityFile for each host. SSH will automatically use the correct key when you connect to each server.

Conclusion

SSH key-based authentication is a fundamental skill for any Linux system administrator or DevOps engineer. It provides superior security compared to passwords while enabling automation and improving productivity. By following this guide, you’ve learned not just how to set up passwordless SSH, but also the security implications, best practices, and troubleshooting techniques.

Key takeaways: Always protect your private keys, use strong key types like Ed25519, disable password authentication on production servers, and regularly audit your authorized_keys files. With proper implementation, SSH keys form a robust foundation for secure remote server management.

Next steps: Consider implementing SSH certificate authorities for large-scale deployments, explore SSH bastion hosts for enhanced security, and integrate SSH key management into your infrastructure-as-code workflows.