Learn SSH: Introduction to SSH on Unix-Based Systems

What SSH is, how to set up an SSH server in an Ubuntu or Rocky Linux virtual machine, and how to connect to it from your terminal with a password.

On this page

Updated November 2026 for Ubuntu Server 24.04 and 26.04 and for Rocky Linux 10. The original post used CentOS 7, which is no longer supported.

In this article series, I will explain how to use a command-line terminal to connect to secure shell (SSH) servers, using the config file for quick access to frequently used servers, and setting up an SSH server on Ubuntu or Rocky Linux.

What Is SSH?

So what is SSH, and why do we use it? SSH (secure shell) lets you log in to another computer and type commands there, over an encrypted connection that nobody in between can read. That is how you run heavy software on a powerful server from your laptop, or deploy a website to a machine in a data center.

Setting Up the SSH Server

To get started, I recommend installing Ubuntu Server or a minimal Rocky Linux in a virtual machine on your computer, for example with VirtualBox. Rocky Linux installs the SSH server by default. On Ubuntu, select Install OpenSSH server in the installer, or install it afterward:

$ sudo apt install openssh-server

Then, open the following file with root privileges using your favorite text editor:

/etc/ssh/sshd_config

For instance, we can use vi to open the file:

$ sudo vi /etc/ssh/sshd_config

This file is the configuration file for the SSH daemon on your virtual machine. Here, we need to make sure the following line is not commented out using # and it is set to yes:

PasswordAuthentication yes

NOTE: Current releases also read settings from files in /etc/ssh/sshd_config.d/, and those files take priority. If your change has no effect, look there for another PasswordAuthentication line.

If we make any changes to this file, we need to restart the SSH service. The service is named ssh on Ubuntu and sshd on Rocky Linux:

$ sudo systemctl restart ssh     # Ubuntu
$ sudo systemctl restart sshd    # Rocky Linux

Connecting via SSH

Now, we need to find our virtual machine’s IP address (host-only adaptor) to connect to it. Run this inside the virtual machine:

$ ip -brief address

Now, we can open up a terminal on our computer (not inside the virtual machine!) and connect to our virtual machine using the username and password we set during server installation:

$ ssh username@virtual_machine_IP_address

In my case, the result looks like this:

$ ssh sina@192.168.56.3
The authenticity of host '192.168.56.3 (192.168.56.3)' can't be established.
ECDSA key fingerprint is SHA256:MyF58hvhnjfyBHdQEl9fkpiGyGMG+b1W2LPfzpUQYu4.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.56.3' (ECDSA) to the list of known hosts.
sina@192.168.56.3's password:
Last login: Mon Aug 12 00:00:00 2019
[sina@localhost ~]$

Now, we are successfully connected to the virtual machine using SSH! We can run any command on the virtual machine using the terminal on our local computer.

Next article in the series: SSH CLI Commands

Written by Sina Fathi-Kazerooni

I write about reinforcement learning, transformers, multi-agent systems, and the tools I build along the way.