SSH config file: how to manage multiple servers cleanly

Quick Answer: The SSH config file at ~/.ssh/config lets you define named aliases for SSH connections. Each entry uses a Host label followed by options like HostName, User, Port, and IdentityFile. After setup, ssh my-server replaces a command like ssh -i ~/.ssh/key -p 2222 ubuntu@203.0.113.10 — no flags, no memorisation.

---

If you manage more than two or three servers, you have probably typed something like this more than once:

ssh -i ~/.ssh/my-project-key -p 2222 -L 8080:localhost:3000 ubuntu@203.0.113.42

That works. It is also exhausting to type, impossible to remember, and impossible to explain to a teammate.

The ~/.ssh/config file exists to solve exactly this problem. It lets you define named aliases for SSH connections with all of their options pre-configured. After setting it up, connecting to that server above becomes:

ssh my-project

Quick-reference: common config directives

| Directive | What it does | |---|---| | HostName | The real IP address or hostname SSH connects to | | User | The login username on the remote server | | Port | SSH port (default 22) | | IdentityFile | Path to the private key file to use | | ProxyJump | Jump through a bastion host automatically | | ForwardAgent | Forward your local SSH agent to the remote host | | ServerAliveInterval | Send keep-alive packets every N seconds | | AddKeysToAgent | Automatically add key to SSH agent on first use | | Compression | Compress the data stream (useful on slow connections) | | StrictHostKeyChecking | How to handle unknown host keys (accept-new, yes, no) |

Where the file lives

The SSH config file lives at ~/.ssh/config on macOS and Linux. On Windows, it is at C:\Users\YourName\.ssh\config.

macOS / Linux:

touch ~/.ssh/config
chmod 600 ~/.ssh/config

Windows (PowerShell):

New-Item -Path "$env:USERPROFILE\.ssh\config" -ItemType File -Force

Then open it with any text editor. On Windows, notepad $env:USERPROFILE\.ssh\config works. On macOS, nano ~/.ssh/config or code ~/.ssh/config if you use VS Code.

The chmod 600 is important on macOS/Linux. SSH will refuse to use a config file that is readable by other users on the system.

Basic syntax

Each entry in the config file starts with a Host keyword followed by the alias you want to use. Indented lines below it set the options for that host.

Host my-project
  HostName 203.0.113.42
  User ubuntu
  Port 2222
  IdentityFile ~/.ssh/my-project-key

Save that, and now ssh my-project does exactly the same thing as the long command above. This also works for scp, rsync, and anything else that uses SSH under the hood:

scp my-project:/var/log/app.log ./
rsync -avz my-project:/var/www/ ./backup/

Managing multiple keys for multiple projects

If you work with multiple clients or projects, each with its own SSH key, the config file keeps things organized:

Host client-a-prod
  HostName 198.51.100.10
  User ec2-user
  IdentityFile ~/.ssh/client-a-key

Host client-a-staging
  HostName 198.51.100.20
  User ec2-user
  IdentityFile ~/.ssh/client-a-key

Host client-b-server
  HostName 203.0.113.50
  User ubuntu
  IdentityFile ~/.ssh/client-b-key
  Port 2222

Now you have named connections for every environment. Switching between them is a two-word command, and there is no risk of accidentally using the wrong key for the wrong client.

Setting defaults for all hosts

The special alias Host * applies settings to every SSH connection. Use it for global settings you want everywhere:

Host *
  ServerAliveInterval 60
  ServerAliveCountMax 3
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_ed25519

Settings in more specific Host blocks override the defaults in Host *. So you can set a default key for most connections and override it for specific hosts that need a different key.

The AddKeysToAgent yes line automatically adds your private key to the SSH agent when you first use it, so you only need to enter your passphrase once per login session.

Jump hosts with ProxyJump

If you use a bastion host to reach private servers, the config file automates the tunneling. Instead of specifying jump hosts on the command line every time, define them in config:

Host bastion
  HostName 203.0.113.10
  User ubuntu
  IdentityFile ~/.ssh/id_ed25519

Host db-server
  HostName 10.0.0.5
  User ubuntu
  IdentityFile ~/.ssh/id_ed25519
  ProxyJump bastion

Host app-server
  HostName 10.0.0.10
  User ubuntu
  IdentityFile ~/.ssh/id_ed25519
  ProxyJump bastion

Now ssh db-server automatically connects through the bastion first, without you having to think about it. The same applies to scp db-server:/path/to/file ./. See our guide on how to set up a bastion host on AWS and GCP for how to configure the underlying network.

Automatic port forwarding

If you frequently need a port forward when working on a specific server, make it automatic:

Host dev-server
  HostName 203.0.113.30
  User ubuntu
  LocalForward 5432 localhost:5432

Every time you run ssh dev-server, SSH also opens a local tunnel from your machine's port 5432 to the remote server's Postgres instance. Your local database client connects to localhost:5432 through the encrypted tunnel without any extra flags. See our SSH port forwarding guide for more patterns.

SSH agent forwarding

Agent forwarding lets you authenticate to a third server using your local key, without copying your private key to the intermediate server.

Host bastion
  HostName 203.0.113.10
  User ubuntu
  ForwardAgent yes

One warning: only use ForwardAgent yes for hosts you trust completely. A compromised server with agent forwarding enabled can use your forwarded agent to authenticate to other servers on your behalf while you are connected.

GitHub and GitLab with separate keys

Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/github-key

Host gitlab.com
  HostName gitlab.com
  User git
  IdentityFile ~/.ssh/gitlab-key

This is how you use different SSH keys for different Git hosting services. See our full guide to using SSH with GitHub, GitLab, and Bitbucket for the complete setup including multiple accounts.

Splitting config into multiple files with Include

If you manage a large number of servers, a single growing config file gets unwieldy. The Include directive lets you split your config into multiple files:

# ~/.ssh/config
Include ~/.ssh/conf.d/*

Host *
  ServerAliveInterval 60
  AddKeysToAgent yes

Then create separate files for each project or client:

mkdir -p ~/.ssh/conf.d
# ~/.ssh/conf.d/client-a
# ~/.ssh/conf.d/personal
# ~/.ssh/conf.d/github

SSH reads Include directives before evaluating Host blocks, so the Host * defaults at the bottom of the main file still apply to all included hosts.

On Windows, use full paths in the Include directive:

Include C:/Users/YourName/.ssh/conf.d/*

Useful patterns for common setups

AWS EC2 with a .pem key:

Host my-ec2
  HostName ec2-54-123-45-67.compute-1.amazonaws.com
  User ec2-user
  IdentityFile ~/.ssh/my-key.pem
  StrictHostKeyChecking accept-new

StrictHostKeyChecking accept-new automatically accepts the host key on first connection, useful for frequently provisioned/destroyed EC2 instances where the host key changes.

Long session with keep-alive and compression:

Host remote-dev
  HostName 203.0.113.40
  User ubuntu
  ServerAliveInterval 30
  Compression yes

Troubleshooting config file issues

SSH is not reading your config file

Check three things:

1. The file is named exactly config with no extension — not config.txt or ssh_config.
2. On macOS/Linux, permissions must be 600: run chmod 600 ~/.ssh/config.
3. On Windows, the file must be in C:\Users\YourName\.ssh\, not your Desktop or Documents.

Tab indentation breaks parsing

SSH config is space-indented. If you paste from certain editors or use tabs, SSH silently ignores the block or throws a parse error. Convert all indentation to spaces.

Symptom: ssh my-alias tries to connect using your default key and username instead of the ones you defined.

Fix: Open the config in a text editor that shows whitespace and replace any tabs with spaces.

Host * overriding your specific host settings

The Host * block should always be at the bottom of your config file. SSH reads the config top to bottom and applies the first matching value for each directive. If Host * is at the top, its IdentityFile gets applied before your specific host's IdentityFile is even read.

Wrong order:

Host *
  IdentityFile ~/.ssh/id_ed25519

Host client-a
  IdentityFile ~/.ssh/client-a-key   # This gets IGNORED

Correct order:

Host client-a
  IdentityFile ~/.ssh/client-a-key   # This is used

Host *
  IdentityFile ~/.ssh/id_ed25519     # Fallback for everything else

Debugging which config values are being applied

To see exactly which config file and options SSH is using:

ssh -vv my-alias 2>&1 | grep -i "config\|identity\|host"

This prints every config file SSH read, every identity file it tried, and which Host block matched.

Complete example config file

Here is a single, copy-pasteable config that combines all the patterns above:

# ─── GitHub and GitLab ────────────────────────────────────────────────────────
Host github.com
  HostName github.com
  User git
  IdentityFile ~/.ssh/github-key

Host gitlab.com
  HostName gitlab.com
  User git
  IdentityFile ~/.ssh/gitlab-key

# ─── Client A (production + staging) ──────────────────────────────────────────
Host client-a-prod
  HostName 198.51.100.10
  User ec2-user
  Port 2222
  IdentityFile ~/.ssh/client-a-key

Host client-a-staging
  HostName 198.51.100.20
  User ec2-user
  Port 2222
  IdentityFile ~/.ssh/client-a-key

# ─── Internal servers via bastion ─────────────────────────────────────────────
Host bastion
  HostName 203.0.113.10
  User ubuntu
  IdentityFile ~/.ssh/id_ed25519
  ForwardAgent yes

Host internal-db
  HostName 10.0.0.5
  User ubuntu
  IdentityFile ~/.ssh/id_ed25519
  ProxyJump bastion
  LocalForward 5432 localhost:5432

# ─── Defaults for everything ──────────────────────────────────────────────────
Host *
  ServerAliveInterval 60
  ServerAliveCountMax 3
  AddKeysToAgent yes
  IdentityFile ~/.ssh/id_ed25519

Copy this, swap in your real hostnames and key paths, and you have a working config for a typical multi-server setup. Every pattern in this file is covered in detail in the sections above.