Paranoid Security: Establishing a Connection the Hard Way

2008-12-06

howto security linux Technology

Recently, I was describing the personal setup I use to connect to my home machine over on watchingback (a group that has gone unfortunately silent). This setup combines port-knocking (with one-time sequences), disk encryption, and passphrase-protected rsa keys. Here’s a basic rundown of how it works from an end-user perspective (i.e., once everything is set up):

First, the user (me) inserts a USB flash drive with an encrypted partition. He mounts up the encrypted disk on a local machine (I’ll call this machine the ‘client’ throughout this article), providing the necessary password, and runs a script called ‘callhome’. He is prompted for his passphrase, and then gets a terminal session on his home machine (we’ll call this one the ‘server’).

Read on for details about this setup, and how to do it.

Warning: what follows is madness. It is overkill taken to an extreme. I am describing a way you can take a very, very simple procedure (connecting remotely to a system), and make it exceedingly complicated, all for the benefit of a little added security. Whether or not this security is worthwhile to you is, of course, your business. In an age where our governments and fellow citizens are increasingly keen on everything from our shopping and reading habits to our credit card numbers, I personally feel that cautiousness is worth the effort.

It is madness. I’m not convinced it isn’t justified madness.

This tutorial assumes you are running Linux, and that you are comfortable with the command-line interface and with networked computing in general (of course, you’re reading this on the Internet, so that’s a good start). All of my examples will be Fedora-centric. If you don’t use Fedora, you’ll need to figure out what the commands are for your distro.

So, how is this complex setup I describe different from just typing “ssh user@server”? Well, first, the callhome script executes a portknocking sequence. Until this sequence is done, ssh is closed on the server. After the sequence, ssh is opened only for the IP address of the client, and only for a small time window. The ssh connection must happen during this window. The script initiates the ssh connection, which helps keep this secure. In addition, each portknocking sequence is valid only once - the USB drive contains a list of all valid sequences, and the script is set up to only use each one once.

Next, ssh on our server is set up to only allow connections with public keys. This means that even if an attacker knew the correct portknocking sequence, he would not be able to login with a password - he must have the private RSA key. The private key is on our USB flash drive, which is encrypted. The key itself is further encrypted with its own passphrase, so you still enter a password to connect home, the work to verify it is simply done on the local machine. The passphrase is never sent across the Internet, even in an encrypted/hashed form.

There are some other nice features, including a ‘panic’ portknocking sequence that will shut down the portknocking server itself, locking down the remote server completely. This panic script is stored on a machine to which I have a shell account. If the USB flash drive is ever lost/stolen, I can get to any machine with an ssh client, log in to the shell account, and kill the knock server. New connections to the server then become impossible.

This setup is useful for more than just a terminal connection home. You can forward X through it and run graphical apps from home (this is typically going to be very slow, however). You can forward any ports you like, so that you can route web traffic through this ssh tunnel and prevent people on your network from watching where you go on the web. Anything you can do with a normal ssh connection can be done here. Later I’ll demonstrate some examples that I use. So, that’s the setup.

Now I will outline exactly how to do it, one step at a time. You might want to grab a snack and use the bathroom - this is going to be a long trip.

Part 1: Dynamic DNS

Before you can call home to your server, it helps to have a name to call it by. However, you can’t use a traditional hostname if your machine is on a broadband network because your IP address may periodically change. Dynamic DNS (or DynDNS) was created to solve this problem. A daemon runs on your server that periodically checks the IP address of the server and sends it to a DynDNS server. This DynDNS server then updates a DNS record whenever your IP address changes. I use DynDNS.com. It’s free and easy. Just choose a hostname for your machine, then install and configure the ddclient software. You can get instructions on configuring ddclient for DynDNS.com here.

Part 2: Configuring SSH

On the server, find your sshd configuration file (on Fedora, this is at /etc/ssh/sshd_config) and ensure the following options are set to these values:

RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile .ssh/authorized_keys PasswordAuthentication no ChallengeResponseAuthentication no

Now, restart your ssh daemon:

service sshd restart

Now, try to ssh into your machine (you can just do ‘ssh user@localhost’). You’ll get denied immediately, without even seeing a password prompt. This is what we want. Next, we create the ssh key that we will use. Run:

ssh-keygen -t rsa -b 4096

When prompted, specify a path other than the default. Your home directory is a good choice - we will be moving id_rsa to the USB flash drive later. Also, make sure you specify a good passphrase - if the USB flash drive is compromised, the strength of this passphrase will buy you time to lock down the server. Now you have 2 files in your home directory, id_rsa and id_rsa.pub. id_rsa is your encrypted, private RSA key. id_rsa.pub is the public key that matches this private key. Copy the contents of id_rsa.pub into ~/.ssh/authorized_keys. This step will allow the private key to connect to the server as this user.

Part 3: portknocking

There’s still one significant security concern: unknown vulnerabilities. OpenSSH is a complex program, and almost certainly still contains a vulnerability or two that haven’t been discovered. To combat getting hit with that latest exploit, we can hide the presence of ssh from the outside world completely. This is the beauty of portknocking. The premise of portknocking is that the ssh port is firewalled off unless a specific sequence of ports are first pinged, in order. This doesn’t add a lot of security by itself; an attacker can simply sniff the portknock sequence, then repeat it to open the same port. Normally, portknocking will only deter attackers who don’t know you have ssh open.

However, the portknocking server we are going to use supports one-time sequences. With this configuration, the correct knock sequence changes after each knock. The server has a list of sequences to use, and we will also keep this list with us on the USB flash drive. Before we begin configuring portknocking, make sure you have firewalled off port 22. There are two possible network setups we will consider:

You have a router between the server and the Internet. This router passes ssh traffic to your server, and the router acts as the firewall that blocks ssh access.

The server is connected directly to the Internet. Local firewall rules on the machine are blocking ssh access.

In the first instance, you need to be able to install a portknocking server on the router; additionally, the firewall rules needed will be more complicated, and will vary based on how your router is configured. My example here assumes the second case: that the server itself is listening to the knocks (i.e. it is directly connected to the Internet). The first case is discussed in Appendix C. Install knockd. Once installed, you’ll need to configure /etc/knockd.conf. For now, I’ll present a basic configuration (we’ll add some more stuff to this later):

[options] logfile = /var/log/knockd.log

[ssh] one_time_sequences = /etc/knockd/ssh seq_timeout = 10 tcpflags = syn start_command = /usr/sbin/iptables -A INPUT -s %IP% -p tcp –dport 22 -j ACCEPT cmd_timeout = 5 stop_command = /usr/sbin/iptables -D INPUT -s %IP% -p tcp –dport 22 -j ACCEPT

In /etc/knockd/ssh, you need to have sequences of numbers to use as one-time sequences. Each entry in the list should be formatted like this:

1,2,3,4,5

There is a space at the beginning of the line; this is helpful because knockd will comment out each line as it uses it by placing a ‘#’ at the beginning of the line. The numbers you generate should ideally be between 1024 and 65535; I generate my numbers with a script similar to the following:

#!/usr/bin/perl $num_keys = 50; @data = d20diceroller --nototals "5d65535[reroll< 1024][repeat $num_keys]";

foreach (@data)
{
next if (/:/);

s/ $//;
s/ /,/g;
s/^/ /;

print;
}

This script uses a program I created, d20diceroller, to generate its random numbers. That tool is part of the d20tools package, and can be found at its sourceforge page. The subversion repository is currently recommended. Now that you have the one-time sequences, you must start the knock daemon. You’ll most likely want to add this to an init script (such as /etc/rc.local):

knockd -i eth0 &

‘eth0’ here should be replaced with whatever the name of your Internet-facing network interface is. Now, portknocking is configured and running. We only need to configure the USB flash drive, and we’re done with the basics.

Part 4: USB Flash drive setup

First, you need a partition on the flash drive that will be dedicated as the encrypted partition. This can be one small partition, or it can be the entire disk. I set aside the last 10 MB of the disk, myself. Use fdisk, parted, or another partitioning tool to get the disk to your liking (remember, re-partitioning can erase everything you have on the drive. Be careful). Once the disk is partitioned, you must create a secure volume, then create a filesystem on that volume. As root, run the command:

cryptsetup create secure /dev/sda1

Where ‘/dev/sda1’ is the device name of the partition that should be the encrypted partition. Enter your desired passphrase when prompted. This should be a different passphrase than you used with the ssh key, ideally. Now, you should have a device file named /dev/mapper/secure. This is the encrypted pseudo-device Linux has created to represent your partition. Create a filesystem on it. I recommend a DOS filesystem because of its portability (an ext3 filesystem will retain the UID/GID and permissions for each file, which can get really confusing when moving from system to system and using users with different UIDs):

mkdosfs -F 16 /dev/mapper/secure

Now mount /dev/mapper/secure. On it, create a directory called .ssh. Copy the id_rsa file you created earlier into this directory, and create a file called ‘config’ that looks like this:

Host your.server.address home User your_user_name UserKnownHostsFile .ssh/known_hosts HostName your.server.address Port 22 IdentityFile .ssh/id_rsa Compression yes

Also, take a copy of the sequences you created earlier (in /etc/knockd/ssh) and copy them to a file called ‘sequences’ in this .ssh directory. You need to modify this sequences file so that the commas are converted to spaces. You can do that with this command:

perl -p -i -e ’s/,/ /’ sequences

Now, create a script in the root of the encrypted partition with these contents:

#!/bin/sh SERVER_NAME=your.server.name WD=$(echo $0 | perl -pe ’s/^(.)/.?$/\1/’) cd $WD chmod 600 $WD/.ssh/id_rsa &> /dev/null sequence=head -n 1 $WD/.ssh/sequences [ -z “$sequence” ] && echo “Error: No more knock sequences” && exit 1 for i in $sequence; do nc -z $SERVER_NAME $i; done sleep 1 ssh -F $WD/.ssh/config home && sed -i ‘1d’ $WD/.ssh/sequences

This script will execute the next portknocking sequence in the list, then automatically ssh into the server. It uses the config file in our local .ssh directory, so the username and key file are already specified. Now, to unmount the USB flash drive’s encrypted partition, you can execute these commands:

umount /dev/mapper/secure cryptsetup remove secure

That’s it! Now, all you need to do use the system is set up the partition as an encrypted volume, mount the encrypted filesystem, run the ‘callhome’ script, and enter your ssh passphrase. Extra-secure connection home, for the truly paranoid. The only upkeep required is to periodically generate a new list of sequences when you run low. This system is a bit more complicated than just using an ssh command, but I discuss how to automate the connection procedure on systems you use a lot in Appendix B.

But wait, there’s more! What happens when The Bad Guys steal our USB flash drive and start frantically trying to decrypt it? Enter the panic knock.

Part 5: Disaster recovery

A scenario, if you will. You’re sitting at your desk, your uber-secure connection home humming along, letting you chat on IRC without your boss being any wiser. You lock your X session and get up to grab some coffee. You get back to your desk, and glance over at your workstation, expecting to see protruding from the front your faithful USB flash drive, fast friend these many years, steadfast companion against the dangers of revealing your personal life’s details to those who would kill for it. But it’s gone. Someone has taken it. A quick survey of your fellow workers (and by “survey” we mean “threaten them with violence so they know you’re serious”) reveals that they don’t have it. No one saw anyone come near your desk.

There is only one explanation: Identity Theft Ninja. Trained in the secluded mountains of Japan from birth, these versatile agents of stealth can smell a USB flash drive that allows a connection to someone’s home server from a league distant. You never had a chance. Hope is not lost, however! Because the drive is encrypted, and the ssh key is further encrypted, you have an advantage. The Identity Theft Ninja have powerful computers for cracking encryption schemes, but it will still take time.

Basically, when you notice your USB drive is missing, you can execute your panic script. The panic script should live on a shell server; something you can get to from any machine. I recommend silenceisdefeat. On the shell server, you simply have a script called ‘panic’. It can look like the following:

#!/bin/sh SERVER_NAME=your.server.name sequence=(“1” “2” “3” “4” “5”)

for i in ${sequence[@]};
do nc -z $SERVER_NAME $i;
done

Most shell servers will not give you execute privileges, but because this is a script, you can simply type ‘sh panic’ to execute it. On the knock server, we have a special action to perform when someone executes that particular sequence. Add this to /etc/knockd.conf:

[shutdown] sequence = 1,2,3,4,5 seq_timeout = 10 tcpflags = syn command = killall knockd

By the way, do not actually use the sequence 1 2 3 4 5. Use a random sequence, but include a number that will never appear in your normal portknocking sequences. The ‘out of phase’ number guarantees you never accidentally shut down the server, and keeping the sequence random guarantees that a portscan or other malicious attack won’t lock you out down your server. It would be a good idea to change this sequence every time you use it, as well, to prevent an attacker from repeating the sequence to frustrate you.

Appendix A: Beyond SSH - forwarding other traffic

You can take advantage of the power of SSH to create an extremely secure tunnel for almost any data; you aren’t limited to running commands on your remote machine. Perhaps you want to browse the web through the encrypted tunnel, so other users on the network can’t see that you’re really shopping on newegg instead of getting work done. In that case, you could add this to your .ssh/config file:

DynamicForward 8137

This creates a SOCKS proxy that you can route traffic through. Simply configure your web browser to use a proxy at localhost, port 8137. If you want to tunnel certain sites through the proxy but not others (and you use Firefox), check out FoxyProxy. Check the command ‘man ssh_config’ for more options you can put in the .ssh/config file.

Appendix B: Mounting your encrypted volume made easy

You need root access to create and mount an encrypted volume. If you use the same few computers all the time (and you have root access on them), you can simplify your life. First, use the ‘visudo’ command and add a line to the end of the sudoers file like this:

your_user ALL=(root) NOPASSWD: /sbin/cryptsetup

This will allow you, as a normal user, to execute ‘cryptsetup’, which lets you create and remove encrypted volumes. Next, add a line like this to /etc/fstab:

/dev/mapper/secure /mnt/secure auto noauto,user,umask=077 0 0

This will allow users to mount /dev/mapper/secure once it is created. The umask guarantees other users on the system can’t see our files, which would compromise the ssh key. Don’t worry, we can still prevent another user on the system from hijacking our mount; that comes next. Now, create two files in /usr/local/bin, called ‘secureon’ and ‘secureoff’. In secureon, put:

#!/bin/sh sudo cryptsetup create secure /dev/sda1 && <br/>mount /mnt/secure

sda1, of course, is whatever the device name of the encrypted partition is. You can use udev or hal to ensure this is always a consistent name. secureoff looks like this:

#!/bin/sh umount /mnt/secure && <br/>sudo cryptsetup remove secure

Make both of these scripts executable (‘chmod 755 /usr/local/bin/secureo*’). Now you can simply run ‘secureon to create and mount the secure volume (you’ll be prompted for the encryption passphrase), and ‘secureoff’ when you’re finished.

Appendix C: Behind a router

The last case we will consider is the complex but extremely common situation where you have one device acting as a router. This changes the iptables rules we need to use with the knockd server.

First, you need to have a router that you can install Linux software on. In other words, your router must be running Linux. If you have a computer acting as your router, this is probably no problem for you. If you have a consumer broadband router, this may be more difficult. You can get Linux firmware for certain models of broadband router, however. Several broadband router distributions exist; I use OpenWRT; it is easy to install new software with OpenWRT, and knockd is available for it.

The exact rules you will need are going to depend on your particular iptables setup, but to forward a port you will need two rules: one in the filter table’s FORWARD chain and one in the nat table’s PREROUTING chain. The approach that I recommend is to add the rule in the FORWARD chain permanently, and use knockd to add and remove the PREROUTING rule. This simplifies the knockd configuration, and allows you to use the FORWARD chain as a handy reference for what forwards are possible.

For example, let’s say you have a machine at 10.10.9.18, and the knock daemon will open SSH to this machine. First, you want to add this firewall rule permanently:

iptables -A FORWARD -p tcp –dport 22 -d 10.10.9.18 -j ACCEPT

Put that in your router’s iptables configuration. If your router is running Fedora, put this line (minus ‘iptables’) in /etc/sysconfig/iptables.

If you’re using OpenWRT, I would suggest using the forwarding_wan chain instead of the FORWARD chain. Also, on OpenWRT you can put this line in /etc/firewall.user.

The start_command and stop_command lines in /etc/knockd.conf will add and remove the PREROUTING rule, like so:

start_command = /usr/sbin/iptables -t nat -A PREROUTING -s %IP% -p tcp –dport 22 -j DNAT –to 10.10.9.18:22 stop_command = /usr/sbin/iptables -t nat -D PREROUTING -s %IP% -p tcp –dport 22 -j DNAT –to 10.10.9.18:22

For OpenWRT, use the prerouting_wan chain instead of the PREROUTING chain.

One great thing you can do with a router is use different knock sequences to forward SSH to different servers. If you have several machines on your network, you can simply add additional sections to knockd.conf (and additional rules in the FORWARD chain). As long as they use different knock sequences, you can overload port 22 to forward to whichever machine you need.