How to connect with SSH and set up SSH keys

SSH is how you log in to a Linux VPS and run commands on it. You type into a terminal on your own computer, and the commands run on the server. This guide shows how to connect for the first time, how to create an SSH key and put it on the server, and how to turn off password logins once the key works.

It covers Windows, macOS and Linux, all of which have SSH built in. Setting up keys takes about fifteen minutes and is the single most useful security step you can take on a new server.

What is in this guide

• A few words you will see
• Why keys beat passwords
• Before you start
• 1. Open a terminal
• 2. Connect for the first time
• 3. Create a normal user
• 4. Create an SSH key on your computer
• 5. Put the key on the server
• 6. Test the key
• 7. Turn off password logins
• 8. Make connecting easier
• Keys for several devices
• If you lose your key
• Troubleshooting
• Limits
• When to step up
• What next

A few words you will see

Term	What it means
Terminal	A window where you type commands. Called PowerShell or Terminal on Windows, and Terminal on macOS and Linux.
SSH	Secure Shell: an encrypted connection for logging in to a server.
Key pair	Two linked files. The private key stays on your computer. The public key goes on the server.
Passphrase	A password that protects your private key, so a stolen key file is useless on its own.
authorized_keys	The file on the server that lists the public keys allowed to log in to an account.
Fingerprint	A short code that identifies a server's own key, so you know you reached the right server.
Root	The all-powerful admin account on Linux.

Why keys beat passwords

Bots try to guess SSH passwords on every server on the internet, all day long. A strong password helps, but a key is in a different league: it is far too long to guess, it is never sent to the server, and it only works with the file on your computer.

The public key is safe to share. It is like a lock you hand out. Only your private key opens it. Keep the private key on your devices, protect it with a passphrase, and never send it to anyone, including support.

Before you start

• Your server's IP address and root password, from the RackNerd welcome email.
• A computer running Windows 10 or later, macOS or Linux.
• Access to the VPS control panel, also from the welcome email. Its console is your way back in if something goes wrong.

In the commands below, replace SERVER_IP with your server's address and alex with the username you choose.

1. Open a terminal

System	How to open a terminal
Windows	Search the Start menu for PowerShell or Terminal. The ssh command is built in on current versions.
macOS	Open Terminal from Applications, then Utilities.
Linux	Open your Terminal app. If ssh is missing, install the OpenSSH client from your package manager.

Check SSH is available by typing ssh -V. It should print a version number.

2. Connect for the first time

ssh root@SERVER_IP

The first time, SSH shows the server's fingerprint and asks whether to continue. This is how you know you reached your server and not an impostor. For extra care, open the console in the VPS control panel, log in, and run this to see the server's fingerprint:

ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

If it matches what your terminal shows, type yes. Then enter the root password. Nothing appears as you type the password. That is normal.

Once you are in, change the root password to a long, unique one:

passwd

3. Create a normal user

Working as root makes mistakes more costly, and root is the account bots try first. Create a normal user with permission to use sudo for admin commands:

adduser alex
usermod -aG sudo alex

Choose a strong password when asked. You will still use it for sudo, but not for logging in once keys are set up.

4. Create an SSH key on your computer

Run this on your own computer, not on the server. Open a new terminal window if you are still logged in to the server:

ssh-keygen -t ed25519 -C "alex laptop"

• Press Enter to accept the default file location.
• Enter a passphrase when asked, and store it in your password manager.
• The text after -C is a label, so you can tell your keys apart later. Use the device name.

This creates two files in the .ssh folder in your home folder: id_ed25519, the private key, and id_ed25519.pub, the public key. Ed25519 is a modern key type that is short, fast and strong.

Typing your passphrase every time gets tiresome. macOS remembers it in the Keychain. On Windows and Linux, the SSH agent can hold it for your session. See your system's documentation for turning the agent on, as the steps differ by version.

5. Put the key on the server

On macOS and Linux, one command copies the public key to the right place:

ssh-copy-id alex@SERVER_IP

Enter alex's password when asked.

On Windows, ssh-copy-id is usually not available. Run this in PowerShell instead. It is one long command:

type $env:USERPROFILE\.ssh\id_ed25519.pub | ssh alex@SERVER_IP "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

It reads your public key, logs in once with alex's password, and adds the key to alex's authorized_keys file with the right permissions.

By hand, on any system, you can also open id_ed25519.pub in a text editor, copy its single line, log in to the server as alex, and paste it into ~/.ssh/authorized_keys with nano. Then set the permissions with chmod 700 ~/.ssh and chmod 600 ~/.ssh/authorized_keys.

6. Test the key

ssh alex@SERVER_IP

You should be asked for your key's passphrase, not the server password, or not asked at all if the agent remembers it. Check that sudo works:

sudo whoami

It should print root. Do not continue until key login works.

7. Turn off password logins

With the key working, stop SSH from accepting passwords at all, and stop direct root logins. Create a small settings file on the server:

sudo nano /etc/ssh/sshd_config.d/00-hardening.conf

Put these lines in it:

PasswordAuthentication no
KbdInteractiveAuthentication no
PermitRootLogin no

Save with Ctrl+O and Enter, and exit with Ctrl+X. The 00 at the start of the name matters: SSH uses the first value it reads for each setting, and some server images include their own file in this folder that turns passwords back on. Reading yours first means yours wins.

Check, confirm and restart:

sudo sshd -t
sudo sshd -T | grep -E "passwordauthentication|permitrootlogin"
sudo systemctl restart ssh

The second command should show both set to no. On some systems the service is called sshd rather than ssh.

Keep this session open. In a new terminal, check that ssh alex@SERVER_IP still works. Then check that a password login is refused:

ssh -o PubkeyAuthentication=no alex@SERVER_IP

It should fail with "Permission denied". If anything goes wrong, fix it from the session you kept open, or from the control panel console.

For more ways to protect the server, see a beginner's guide to securing your VPS.

8. Make connecting easier

Instead of typing the user and IP address every time, give the server a short name. On your computer, open or create the file config in your .ssh folder and add:

Host myvps
    HostName SERVER_IP
    User alex
    IdentityFile ~/.ssh/id_ed25519

Now ssh myvps is enough. The same name works with scp for copying files, for example scp backup.zip myvps:, and with editors that open folders over SSH.

If your connection drops when idle, add ServerAliveInterval 60 under the Host line. To keep programs running on the server after you disconnect, see screen and tmux.

Keys for several devices

Create a separate key on each device, such as your laptop and desktop, rather than copying one private key between them. Add each public key to the server, one per line in authorized_keys. Then if a device is lost, you remove just that line and the others keep working.

To see which keys can log in, run cat ~/.ssh/authorized_keys on the server. The label at the end of each line, from the -C option, tells you which device it belongs to.

On a phone, SSH apps can create their own keys. Treat a phone key like any other device key.

If you lose your key

If your computer dies or the key is deleted, you are not locked out for good:

1. Log in through the console in the VPS control panel, using alex's password.
2. Create a new key on your new device, as in step 4.
3. Add the new public key to /home/alex/.ssh/authorized_keys by pasting it in with nano.
4. Remove the old key's line if the old device might be in someone else's hands.

This is why your user still has a strong password: it is your way in through the console. Keep a backup of your private key somewhere safe, such as an encrypted backup, if losing access would be a serious problem.

Troubleshooting

Symptom	Likely cause	Fix
Connection timed out	Wrong IP address, server still starting, or a firewall blocks port 22	Check the IP in the welcome email. Use the console to run sudo ufw allow OpenSSH
Connection refused	SSH is not running, or listens on another port	Use the console to run sudo systemctl status ssh
Permission denied (publickey)	Key not in authorized_keys, wrong user, or wrong permissions	Check the key line is there for that user, then run the chmod commands in step 5
Still asks for the server password	The key is not being offered or accepted	Run ssh -v alex@SERVER_IP and look for lines about your key
Too many authentication failures	Your computer offers several keys and runs out of tries	Add an IdentityFile line for this server in your SSH config, as in step 8
Warning that the host key has changed	The server was reinstalled, or you reached a different machine	If you reinstalled it, run ssh-keygen -R SERVER_IP and connect again. If not, stop and check with support
Password login still works after step 7	Another file sets PasswordAuthentication yes first	Run the sshd -T check and look at files in /etc/ssh/sshd_config.d/
Unprotected private key file warning	The private key file can be read by other users	On macOS and Linux, run chmod 600 ~/.ssh/id_ed25519

Limits

• Keys protect logins, not everything. Your apps, updates and firewall still need attention.
• A stolen device is a risk. A passphrase on your key and a locked computer are what stop someone using it.
• Commands vary slightly. Older systems and some distributions name the SSH service differently, or keep all settings in one file.
• Never share a private key, including with support. They will never need it.

When to step up

SSH works the same on every plan size. Once you are comfortable connecting, the next limit is usually memory: a 1 GB or 2 GB plan suits learning and a few light jobs, while running several apps or Docker containers is easier on 4 GB. When you manage more than one server, the SSH config file from step 8 keeps them all one short name away.

What next

• Finish setting up the server: your first hour on a new RackNerd VPS.
• Block repeated login attempts on other services: installing fail2ban.
• Learn the everyday tools: useful Linux server tools.

To compare what is available now, start with the cheapest RackNerd VPS guide and check the sales page.