Building an OpenBSD WireGuard VPN server part 3 – Unbound DNS filtering

In part 2, I reconfigured my WireGuard VPN to use an Unbound DNS server on the VPN server rather than rely on a third party server I had used for the original quick and dirty configuration. It was important for me to set up a validating DNS server, which I did in that part.

In this part, I’m extending the existing configuration to include some basic block lists for known ad and tracking servers. As I’m mainly trying to use the VPN while on the road, I want to ensure that anything I end up doing using the VPN is as secure as I can make it with reasonable effort. That makes tracking and preventing malicious ads the next step. That said, I’m not planning to go for a full Pi-Hole like setup. Initially, I am trying to do is integrate one known good blocklists into the Unbound configuration and automate the process. I can get fancy with a more Pi-Hole like setup later if I want to.

Picking a DNS filter list and using it from Unbound

I started by using a single DNS block list from StevenBlack’s github repo. deadc0de.re’s blog has a good how-to post including the necessary awk incantations for converting the file into the format Unbound needs. I used that blog post as my as my starting point.

The process overall is relatively simple. First, pick the flavour of blocklist you want. Download the hosts file for the blocklist. I used wget to download it as it’s already installed on my OpenBSD system, but curl would also work. My assumption is that you’re running these commands in /var/unbound/etc, so all the absolute paths refer to that location.

wget -O /tmp/blocklist-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

This list is in the traditional host file format, so we need to convert it into the format that Unbound needs. Fortunately, deadc0de.re’s blog post contains the necessary code to accomplish that transformation:

cat /tmp/blocklist-hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > adblocker.conf

We also need to update our unbound.conf file to include the freshly generated host blocklist. To do that, we have to add an include statement to the server section of the configuration file. The change looks like this:


server:
  ...
  include: /var/unbound/etc/block-ads.conf

I also had to set up unbound-control to ensure that I didn’t have to restart the server every time I updated the block list. Thus, I had to make a slight detour that involved running unbound-control-setup to generate the necessary certificates. I also needed to add the following statement to my unbound.conf:

remote-control:
  control-enable: yes

Note that remote-control is its own section and is not part of the server: section. The default setup only accepts remote control connections on the loopback interface. This is perfect for my setup, so I didn’t need to make any other changes.
After restarting unbound to pick up the configuration changes I was able to test the VPN with the new DNS blocking and it proved to be mostly effective on the websites I visited.

Automating the process

Of course I’m a software engineer by background, so clearly I wanted to automate the process. Fortunately this is about as complicated as pasting the commands I listed above into a shell script and periodically run it. My script looks like this:

#!/bin/sh

cd /var/unbound/etc
wget -O /tmp/black-hosts https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
cat /tmp/black-hosts | grep '^0\.0\.0\.0' | awk '{print "local-zone: \""$2"\" redirect\nlocal-data: \""$2" A 0.0.0.0\""}' > block-ads.conf
rm /tmp/black-hosts
unbound-control reload

I’m running this script via cron once a week, as that seems to roughly correspond to the update frequency of the list. I may change that in the future, especially if I end up combining the multiple block lists instead. In the meantime I don’t think it’s necessary to run the cron script more often.

What’s next?

Right now it looks like WireGuard VPN server is working as intended and does what I need it to. TuM’Fatig has a great blog post on expanding the approach I’m using to include all of the public block lists that Pi-Hole is using, so I’m probably going to look into that sooner or later. Other than that and keeping the machine up to date, the next project will be to create orchestration scripts to recreate the server from scratch. I found setting up WireGuard on OpenBSD much easier than doing it on a Linux host. Nevertheless having a Terraform or Ansible script that allowed me to quickly stand one up from scratch if the current instance was damage would likely be a good thing.

Would I go the OpenBSD route again?

I had mentioned at the start of this series that I hadn’t had done much with OpenBSD for years. Was it worth choosing OpenBSD over FreeBSD, which I use on a very regular basis? Maybe, maybe not. OpenBSD required less additional fiddling to secure it out of the box. Some of the time saved was taken up by reacquainting myself with the system, so I’d call that a wash. I suspect that setting up the actual WireGuard VPN instance would take the same amount of time on FreeBSD as I’d run the same commands, so no clear advantage there. I do like the newer version of pf that comes with OpenBSD – FreeBSD’s is a few versions behind and is missing some features, and I doubt it’ll ever catch up.

Put that down as a ‘maybe’.

How well does WireGuard work for me?

So far it is proving performant enough to run on one of Vultr’s $5 instances that aren’t exactly swimming in computing power. Of course this is only a personal VPN and I’m not running a whole company’s worth of traffic across it. That would very likely require more oomph (technical term) on the server, but overall WireGuard VPN appears to be both very lightweight and more importantly, easy to configure both at the server and client end.

Throughput is good enough to use it to watch YouTube videos in good quality on crappy hotel Wifi without visibly dropping frames. That’s good enough for me as a performance metric because it shows that the VPN server is not the bottleneck. The real use cases for the VPN really are for accessing more sensitive information like financial information, and that is usually all interactive anyway.

Building an OpenBSD WireGuard VPN server part 2 – Unbound DNS setup

In the first part, I described how I set up the basic OpenBSD WireGuard VPN server. I also hinted that I wanted to set up my own validating, filtering DNS server. With a little bit of spare time during the holidays I decided now was a good time as any.

Making sure the VPN server uses the local Unbound DNS resolver first

Before I set up Unbound itself, I need to make sure that the VPN server’s resolv.conf points at its local DNS server first.

My Vultr VPS has a static IP, which it receives via DHCP. It also receives information about its upstream DNS servers via DHCP. So as the first step, we need to update /etc/dhclient.conf to ensure that resolv.conf lists our local server first. We do this by adding the following line to /etc/dhclient.conf:

prepend domain-name-servers 127.0.0.1;

In fact, that is the complete dhclient.conf file on my VPN server.

Setting up Unbound as a validating DNS server

OpenBSD already includes the Unbound authoritative DNS server, so it’s a matter of configuring it to act as the local DNS server and also to enable validation. Everything you need to run the server is already on the system.

Root server list and trust anchor files

Once we told the local system to use the local Unbound as its DNS server, we need to configure it. Note that dhclient will still add the upstream provider DNS server to resolv.conf, so it is still there as a fallback.

Time to set up Unbound. A validating DNS server needs a few more pieces of data compared to a standard recursive DNS server. The two important items are the trust anchor and the root hints so Unbound knows where to find the root name servers. Note that OpenBSD runs Unbound in a sandbox and thus all the configuration is under /var/unbound.

We start with retrieving the root.hints file that contains the list of root DNS servers:

wget https://www.internic.net/domain/named.root -O /var/unbound/etc/root.hints

A quick check with more /var/unbound/etc/root.hints shows that we received the expected list of root servers.

The second part we need is the trust anchor, which is the cryptographic key our DNS server needs so it can validate DNSSEC. After a couple of false starts I simply ended up using unbound-anchor to download the trust anchor and configured Unbound via the auto-trust-anchor setting to automatically check and update the trust anchor when necessary.

Pretty much all of the Internet tutorials suggest that the trust anchor should be stored in /var/unbound/etc. In my OpenBSD setup, unbound can’t write to that directory so I ended up sotring it in /var/unbound/db instead.

Unbound.conf minimal setup for validating server

Now we’re finally ready to configure unbound itself. My basic, stripped down version of the configuration looks like this:


server:
  verbosity: 1
  #
  # Interfaces and access control. Make sure we're not suddenly
  # running an open public resolver
  #
  outgoing-interface: 'real' server ip'
  interface: 127.0.0.1
  interface: 192.168.1.1         # Internal VPN interface
  access-control: 127.0.0.0/8    allow
  access-control: 192.168.1.0/24 allow
  #
  # Basic configuration
  #
  port: 53
  do-ip4: yes
  do-ip6: no                     # VPN server currently doesn't support IPV6
  do-udp: yes
  do-tcp: yes
  #
  # Root servers, trust anchor and validation logging
  #
  root-hints: "/var/unbound/etc/root.hints"
  auto-trust-anchor-file: "/var/unbound/db/root.key"
  val-log-level: 2
  val-clean-additional: yes
  #
  # Some basic hardening
  #
  hide-identity: yes
  hide-version: yes
  harden-glue: yes
  harden-dnssec-stripped: yes
  use-caps-for-id: yes
  unwanted-reply-threshold: 10000
  #
  # Ensure we don't return private addresses to prevent DNS
  # rebinding attacks
  #
  private-address: 10.0.0.0/8
  private-address: 172.16.0.0/12
  private-address: 192.168.0.0/16
  #
  # Other settings. I don't expect to see a lot of DNS traffic so
  # I turned off prefetching
  #
  prefetch: no
  #
  num-threads: 1

Now that we have a working configuration file, we need to enable and start Unbound:

vpn# rcctl enable unbound
vpn# rcctl start unbound
unbound(ok)

Testing DNSSEC validation

At this point we have a working server with supposedly working DNSSEC validation. Obviously we work on ‘trust, but verify’. To check that we have indeed a working validating server, we can run the following command:

dig www.nic.cz. +dnssec

The header section of the result should look like this:

; <<>> DiG 9.4.2-P2 <<>> www.nic.cz. +dnssec
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 18417
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

See the bolded ‘ad’ in the flags line? Now compare this to the output of the same command, but run on my MacBook using the ISP’s resolver:

; <<>> DiG 9.10.6 <<>> www.nic.cz. +dnssec
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 12527
;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1

The ISP’s resolver doesn’t support DNSSEC in this case, so you can see the ‘ad’ flag missing. That flag indicates that the result from the upstream server validated.

Update the WireGuard config on the client

After I verified that the validation is working, it is time to update the WireGuard configuration on all clients so they point at our shiny new validating server rather than the third party DNS servers I used in the first installment.

Summary

At this point I have a validating DNS server, however it still passes through requests for every domain including the typical ad slinging domains. In the next installment of this series I’ll add filtering rules to try to get rid of most, if not all of them.

Building an OpenBSD Wireguard server

In my previous post, I mentioned that I somehow ended up with a corrupted filesystem on the WireGuard server I had set up earlier this year. That iteration of my VPN server was built on Linux as I expected I would get better performance using the kernel-based WireGuard implementation. It had taken me a while to set it up right, and I didn’t get the impression that the performance was so much better anyway. Keep in mind that I mostly use my VPN server from hotel WiFi and we all know how “good” that tends to be performance wise.

While I’ve done a fair bit of Linux admin work, I didn’t fancy re-doing the whole setup again. I also hadn’t scripted it up using Ansible or similar. I tend to prefer BSD anyway, and most of my personal servers run some flavour of BSD Unix. As I didn’t want to spend too much time securing this server, I used OpenBSD as it is a little more secure out of the box compared to FreeBSD. I also hadn’t experimented with OpenBSD for a while so I was curious to see the more recent improvements.

OpenBSD WireGuard Server setup at Vultr

I re-used the VPS I already had set up for the old Linux WireGuard VPN server at Vultr. Heck, it was corrupted already so formatting it was the only choice. In the interest of getting the VPN up and running quickly again, I used Vultr’s preconfigured OpenBSD image. With hindsight I probably wouldn’t do that again. More about this later. I followed a combination of instructions, mainly from the Cryptsus blog post on setting up WireGuard on an OpenBSD 6.6 server, but also referencing Jasper.La’s blog and Ankur Kothari’s blog. Setting up WireGuard on OpenBSD took me a lot less time than configuring the Linux version. I think it took me an hour or two to get the basic VPN tunnel up and working, including configuring the macOS client.

Overall I found this approach simpler than setting up WireGuard on Linux. I think there are two reasons for this – using the precompiled binaries for the user mode processes means you don’t have to futz around with kernel modules, third party repos and all that fun stuff, plus I personally find FreeBSD and to a certain extent OpenBSD easier to set up. This is obviously a personal preference. Plus, no systemd :).

Notes on the default Vultr OpenBSD Image

The Cryptsus instruction include setting up OpenBSD using full disk encryption. This is more secure than using Vultr’s default, preconfigured VPS image as that one doesn’t support full disk encryption out of the box. Using the preconfigured image makes the installation process easier and quicker, but at the expense of not getting the full official OpenBSD configuration out of the box. With hindsight, I probably what should have used a custom (well, the official) OpenBSD 6.6 image instead of the preconfigured Vultr one, but as mentioned I was in a bit of a rush and it was a case of “good enough and working” trumping “more secure”. When setting up a VPN server, there are basically multiple issues you want to protect yourself against:

  1. Someone remotely logging into your VPN server and getting hold of the encryption keys, and/or being able to capture traffic on your VPN server. That’s obviously the worst-case scenario from a security and privacy perspective, and the main one that I wanted to protect myself against. For that, I rely on OpenBSD’s security with strong passwords and key-based SSH authentication to log into the machine.
  2. Someone getting hold of the disk image – that’s what the full disk encryption part protects against. Something I will want to add to the server sooner or later – it does require a reinstall first, though. The risk of this is considerably lower than someone breaking into the VPN server itself and requires either an infrastructure fluke or an infrastructure compromise.
  3. Someone getting a level of access to the VPS host such that they can observe the running VPS and dump out its memory. Again, that’s possible but it requires a lot of work to get there. I like to think that there are a lot juicier targets than myself out there that would warrant this level of effort. So for right now, I’m discounting this threat model.

I’ve got the first point covered right now, will probably address the second one at some point and do my best to ignore the third one for now.

What’s still left to be done

The basic setup for my OpenBSD WireGuard server is up and running, and I’ve successfully used it while traveling. It’s definitely fast enough even on one of Vultr’s $5 instances. Most of the time, when I use the VPN I don’t need lots of bandwidth, but even when “testing” by watching YouTube videos, the performance was more than good enough.

I’m currently using the AdGuard DNS servers that were mentioned in the Cryptsus blog post I linked to above, but I really want to move to an Unbound DNS server on the VPN host itself that a) validates domains were possible and b) uses something like the Pi-hole blocklists to block ads and trackers. The latter is somewhat optional as I don’t really use the VPN for normal browsing, but it’s a definite nice to have.

Setting up my own VPN server on Vultr with Centos 7 and WireGuard

As an IT consultant, I travel a lot. I mean, a lot. Part of the pleasure is having to deal with day-to-day online life on open, potentially free-for-all hotel and conference WiFi. In other words, the type of networks you really want to do your online banking, ecommerce and other potentially sensitive operations on. After seeing one too many ads for VPN services on bad late night TV I finally decided I needed to do something about it. Ideally I intended to this on the cheap and learn something in the process. I also didn’t want to spend the whole weekend trying to set it up, which is how WireGuard entered the picture. I only really needed to protect my most sensitive device – my personal travel laptop.

As I’m already a customer at Vultr (affiliate link) I decided to just spin up another of their tiny instances and set it up as my WireGuard VPN server. Note that I’m not setting up a VPN service for the whole family, all my friends and some additional people, all I’m trying to do is secure some of my online communications a little bit more.

I also decided to document this experiment, both for my own reference and in the hope that it will be useful for someone else. Readers will need to have some experience setting up and administering Linux server. Come on in and follow along!

Read More

Good analysis on the Android security ecosystem

I recently blogged about Google and Samsung starting to offer regular security patches for their Android devices.

Over on ars technica, Ron Amadeo has an interesting article describing why the current Android ecosystem is not conducive to the quick and widespread distribution of security fixes and why this needs to change, urgently.

At this point in time it seems that in order to be halfway secure, one has to basically root the phone and run well-tested and well supported distribution like CyanogenMod. While I – and presumably most, if not all, readers of this blog – certainly have the technical know how and abilities to root a phone, that’s a poor approach to security because most people either will not or cannot root their phones.

Why I’m suspicious of car insurance dongles

Some security researchers from UCSD showed a proof of concept exploit via one of the dongles that appears to be also used by car insurance companies to monitor your driving “to give you discounts for good driving”. I’m not really a fully paid up subscriber of the tin foil hat brigade but stuff like this makes me glad that I’m still opting for the old-fashioned way of paying for car insurance. Of course the fact that over half our fleet is too old to be OBD-II compliant may have some bearing on that as well…

Not knowing much about CAN bus, my assumption is that in order to get access to certain pieces of data, the dongle will have to put commands on the bus and read the responses. That part is blindingly obvious. Good security practices however would suggest that such a dongle would have a built-in mechanism that restricts the commands it can issue to the set of commands it actually needs to issue to fulfill its function rather than just allowing commands through unfiltered, especially if said dongle is connected to the outside world. I mean, with the ability to issue arbitrary commands to a pile of steel weighing a couple of tons and potentially moving at 70-80 miles per hour, what could possibly go wrong?

On the other hand, instead of having to invent one of those EMP devices as “showcased” in Fast & Furious to stop a street racer, all law enforcement has to do is to send the car an appropriate (or rather, inappropriate) text message.

If anybody needs me, I’m over on Hemmings.com looking for a Ford model A. Try texting that one to stop.

avast!’s virus scanner uses a self-issued trusted root certificate

tl;dr – avast’s web shield functionality appears to insert itself into SSL connections using a self signed trusted root certificate and a simple kind of man-in-the middle “attack” on SSL. I would recommend you turn off web shield’s https scanning or choose another virus scanner.

I read about this on a blog post that was linked from Hacker News where someone claimed that Avast’s virus scanner for Mac OS inserts itself into SSL-encrypted connections using a self-signed certificate. A quick check on a Windows machine in my household confirmed that this was also true for Windows:

Google Maps showing the Avast! certificate
Google Maps showing the Avast! certificate

I case you’re not that familiar with what the connection information is supposed to show you, you can be pretty sure that Google doesn’t have their identity verified by avast!.

As I was preparing the images for the blog post, I noticed the same behaviour on Outlook.com. Have a look at the “Issued by:” entry for the certificate information and also the encryption information in the URL box:

outlook-avast

Now compare the identity information for the “clean” connection with avast! web shield disabled to the information that was showing up in my first example showing Google Maps. This is what a real certificate is supposed to look like:

outlook-clean-1

outlook-clean-2

 

Did you notice the inscription “Microsoft Corporation [US]” next to the padlock symbol that indicates you’re on an encrypted connection? That shows Microsoft is using an EV (Extended Validation) certificate for this particular site. EV certificates are backed by a much more thorough identity verification process, not to mention that they’re considerably more expensive than a normal certificate. EV certificates are intended to give the user additional confirmation that you’re indeed talking to who you think you’re talking to and not Joe’s Phishing & Hijacking service. With avast!’s inserted security certificate, you lose this additional protection. Oh, and for additional comedy value – avast! themselves use an EV certificate on their own website. Which is commendable, only that you don’t necessarily see it if you use the software you downloaded from that website.

Now, is this a big security crisis and we all need to break out our tinfoil hats? I seriously doubt that. In fact, I believe avast! is acting in good faith with their web shield – after all, if you want to protect your users from malware downloaded over an encrypted connection to a dodgy site, the only way to do this in real-time is basically to insert yourself into the connection in the first place.

However, I personally am a bit concerned that Avast! installs a Trusted Root certificate into the Windows certificate store:

avast-certmgr

Call it “echos of SuperFish” (see Lenovo’s security advisory here, Google for the tons of spilled electrons on the Internet), but that makes me uncomfortable, especially if the machine you find it on a machine that you use for online banking.

For my part, I’ve turned the web shield https scanning functionality off for now and will probably spend some more time on this sunny afternoon researching alternative virus scanners.