Setting up rdiff-backup on FreeBSD 12.1

My main PC workstation (as opposed to my Mac Pro) is a dual-boot Windows and Linux machine. While backing up the Windows portion is relatively easy via some cheap-ish commercial backup software, I ended up backing up my Linux home directories only very occasionally. Clearly, Something Had To Be Done ™.

I had a look around for Linux backup software. I was familiar with was Timeshift, but at least the Manjaro port can’t back up to a remote machine and was useless as a result. I eventually settled on rdiff-backup as it seemed to be simple, has been around for a while and also looks very cron-friendly. So far, so good.

Read More

Building an OpenBSD WireGuard VPN server part 3 – Unbound DNS filtering

In part 2, I reconfigured my WireGuard VPN to use an Unbound DNS server on the VPN server rather than rely on a third party server I had used for the original quick and dirty configuration. It was important for me to set up a validating DNS server, which I did in that part.

In this part, I’m extending the existing configuration to include some basic block lists for known ad and tracking servers. As I’m mainly trying to use the VPN while on the road, I want to ensure that anything I end up doing using the VPN is as secure as I can make it with reasonable effort. That makes tracking and preventing malicious ads the next step. That said, I’m not planning to go for a full Pi-Hole like setup. Initially, I am trying to do is integrate one known good blocklists into the Unbound configuration and automate the process. I can get fancy with a more Pi-Hole like setup later if I want to.

Picking a DNS filter list and using it from Unbound

I started by using a single DNS block list from StevenBlack’s github repo. deadc0de.re’s blog has a good how-to post including the necessary awk incantations for converting the file into the format Unbound needs. I used that blog post as my as my starting point.

The process overall is relatively simple. First, pick the flavour of blocklist you want. Download the hosts file for the blocklist. I used wget to download it as it’s already installed on my OpenBSD system, but curl would also work. My assumption is that you’re running these commands in /var/unbound/etc, so all the absolute paths refer to that location.

wget -O /tmp/blocklist-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

This list is in the traditional host file format, so we need to convert it into the format that Unbound needs. Fortunately, deadc0de.re’s blog post contains the necessary code to accomplish that transformation:

cat /tmp/blocklist-hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > adblocker.conf

We also need to update our unbound.conf file to include the freshly generated host blocklist. To do that, we have to add an include statement to the server section of the configuration file. The change looks like this:


server:
  ...
  include: /var/unbound/etc/block-ads.conf

I also had to set up unbound-control to ensure that I didn’t have to restart the server every time I updated the block list. Thus, I had to make a slight detour that involved running unbound-control-setup to generate the necessary certificates. I also needed to add the following statement to my unbound.conf:

remote-control:
  control-enable: yes

Note that remote-control is its own section and is not part of the server: section. The default setup only accepts remote control connections on the loopback interface. This is perfect for my setup, so I didn’t need to make any other changes.
After restarting unbound to pick up the configuration changes I was able to test the VPN with the new DNS blocking and it proved to be mostly effective on the websites I visited.

Automating the process

Of course I’m a software engineer by background, so clearly I wanted to automate the process. Fortunately this is about as complicated as pasting the commands I listed above into a shell script and periodically run it. My script looks like this:

#!/bin/sh

cd /var/unbound/etc
wget -O /tmp/black-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
cat /tmp/black-hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > block-ads.conf
rm /tmp/black-hosts
unbound-control reload

I’m running this script via cron once a week, as that seems to roughly correspond to the update frequency of the list. I may change that in the future, especially if I end up combining the multiple block lists instead. In the meantime I don’t think it’s necessary to run the cron script more often.

What’s next?

Right now it looks like WireGuard VPN server is working as intended and does what I need it to. TuM’Fatig has a great blog post on expanding the approach I’m using to include all of the public block lists that Pi-Hole is using, so I’m probably going to look into that sooner or later. Other than that and keeping the machine up to date, the next project will be to create orchestration scripts to recreate the server from scratch. I found setting up WireGuard on OpenBSD much easier than doing it on a Linux host. Nevertheless having a Terraform or Ansible script that allowed me to quickly stand one up from scratch if the current instance was damage would likely be a good thing.

Would I go the OpenBSD route again?

I had mentioned at the start of this series that I hadn’t had done much with OpenBSD for years. Was it worth choosing OpenBSD over FreeBSD, which I use on a very regular basis? Maybe, maybe not. OpenBSD required less additional fiddling to secure it out of the box. Some of the time saved was taken up by reacquainting myself with the system, so I’d call that a wash. I suspect that setting up the actual WireGuard VPN instance would take the same amount of time on FreeBSD as I’d run the same commands, so no clear advantage there. I do like the newer version of pf that comes with OpenBSD – FreeBSD’s is a few versions behind and is missing some features, and I doubt it’ll ever catch up.

Put that down as a ‘maybe’.

How well does WireGuard work for me?

So far it is proving performant enough to run on one of Vultr’s $5 instances that aren’t exactly swimming in computing power. Of course this is only a personal VPN and I’m not running a whole company’s worth of traffic across it. That would very likely require more oomph (technical term) on the server, but overall WireGuard VPN appears to be both very lightweight and more importantly, easy to configure both at the server and client end.

Throughput is good enough to use it to watch YouTube videos in good quality on crappy hotel Wifi without visibly dropping frames. That’s good enough for me as a performance metric because it shows that the VPN server is not the bottleneck. The real use cases for the VPN really are for accessing more sensitive information like financial information, and that is usually all interactive anyway.

Getting WordPress to work on FreeBSD 10 with PHP 7.0 and Jetpack

This blog is self-hosted, together with some other services on a FreeBSD virtual server over at RootBSD. Yes, I’m one of those weirdos who hosts their own servers – even if they’re virtual – instead of just using free or buying services.

I recently had to migrate from the old server instance I’ve been using since 2010 to a new, shiny FreeBSD 10 server. That prompted a review of various packages I use via the FreeBSD ports collection and most importantly, resulted in a decision to upgrade from PHP 5.6 to PHP 7.0 “while we’re in there”.

Read More

Right now, it’s a good idea to check for FreeBSD bash port updates daily

If you haven’t heard about the bash “shellshock” bug yet, it may be time to peek out from underneath the rock you’ve been under ;). While bash isn’t installed as standard on FreeBSD, there’s a very good chance that someone either installed it because it’s their preferred shell or because one of the ports lists it as a dependency. Either way, now would be a really good time to check if your machine has bash installed if you haven’t done so already. Go on, I’ll wait.

Anyhow, right now you really need to check for updates on the bash port on a daily basis as the updates are coming in at a pretty furious rate. I’m guessing we’ll be back to normal pretty soon, but right now with exploits already in wild, your server will need some extra grooming.

Of course the other, simpler option is to uninstall bash, unless one of the ports you are using has it as a dependency…

Getting emacs’ansi-term to play nicely with FreeBSD

I was playing with the various shell options – sorry, trying to learn eshell – this evening. While playing with eshell I learned about the second, fully fledged terminal emulator ansi-term.

Most of my machines here run FreeBSD, as does the machine that hosts this blog. FreeBSD’s terminal emulators don’t recognise eterm-color as a valid terminal type, plus FreeBSD uses termcap and not terminfo, so the supplied terminfo file was of limited use.

As usual, StackOverflow came to the rescue with this answer. After adding the termcap entry and rebuilding termcap.db, ansi-term is now recognized as a full featured terminal emulator, running inside emacs. So now I can do this:

Screen Shot 2014-01-07 at 10.09.17 PMThat’s midnight commander running on one of my FreeBSD servers, inside GNU Emacs on my Mac. Job done.

And yes, ESC-0 is passed through correctly so I can actually exit mc.

The time has finally come to rebuild my home server

Back in 2009 I built a “slightly more than NAS” home server and documented that build on my old blog. I’ve migrated the posts to this blog, you can find them here, here, here, here and the last one in the series here.

The server survived the move from the UK to the US, even though the courier service I used did a good job of throwing the box around, to the extent that a couple of disks had fallen out of their tool less bays. Nevertheless, it continued soldiering on after I put the drives back in and replaced a couple of broken SATA cables and a dead network card that hadn’t survived being hit by a disk drive multiple times.

I was recently starting to worry about the state of the disks in the server. They were the still original disks I put in when I built the machine, and they were desktop drives that weren’t really built for 24×7 usage, but they coped admirably with running continuously for four years. One started showing a couple of worrying errors (command timeouts and the like), so it was past time to replace all of them before things got any worse.

After some research into affordable NAS disk drives, I ended up with five WD Red 2TB disks for the ZFS raid portion of the NAS and an Intel X25-M SSD drive for the system disk.

First order of the day was to replace the failing disk, so armed with these instructions, I scrubbed the array (no data errors after over four years, yay) and replaced the disk. This time I did use a GPT partition on the zfs drive instead of using the raw disk like I did when I originally built the system. That way it’s at least obvious that there is data on the disk, plus I can tweak the partition sizes to account for slight differences in size of the physical disks. You can replace a raw disk with a GPT partitioned one, you just have to tweak the command slightly:

zpool replace data ada1 /dev/ada1p1

Basically, tell zfs to replace the raw device (ada1) with the newly created GPT partition (/dev/adap1).

The blog post with the instructions on how to replace the disks mentioned that resilvering can take some time, and boy were they not kidding:

root@nermal:~ # zpool status
  pool: data
 state: DEGRADED
status: One or more devices is currently being resilvered. The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Fri Dec 27 07:17:54 2013
        131G scanned out of 3.26T at 121M/s, 7h32m to go
        32.8G resilvered, 3.93% done

At that rate I ended up replacing one disk per day until I managed to replace all four disks. Not that big a deal if I have to do this once every four years.

Why all four disks after I said above that I bought five? I had forgotten that you can’t add a disk drive to a raidz array to extend its capacity – if you want more disks in your array, you have to create a new array instead. Ideally I should have moved all the data to a separate, external disk, recreated the array with all five disks and then moved the data back from the external disk. As the motherboard doesn’t have USB3 connectivity this would have taken way to long so came up with a different approach.

Basically, instead of creating a single big zfs partition on every drive, I created two. One was the same size as the partition on the old 1TB drive, the second one filled the rest of the disk. I also needed to make sure that the partitions were correctly aligned as these drives have 4K sectors and I wanted to make sure I didn’t suffer from performance degradation due to the partition offset being wrong. After some digging around the Internet, this is what the partitioning commands look like:

gpart create -s gpt ada1

gpart add -a 4k -s 953870MB -t freebsd-zfs -l disk1 ada1 gpart add -a 4k -t freebsd-zfs -l disk1_new ada1

I first used a different method to try and get the alignment “right” by configuring the first partition to start at 1M, but it turned out that that’s probably an outdated suggestion and one should simply use gpart’s “-a” parameter to set the alignment.

Once I’ve redone the layout for all four disks with the dual partitions as I showed above, I should be able to add the fifth disk to the array and create a second raidz that uses five disks and also 4k sector alignment (which requires playing with gnop).