Using tortoisehg and mercurial on Windows with openssh

The default setup for the Mercurial DVCS on Windows with tortoisehg uses plink and Pageant to manage SSH keys when you are using ssh as the transport protocol for mercurial. That’s most likely the right choice for a normal Windows setup, but if you already have openssh installed and configured to talk to various servers, it’s easy to switch mercurial and tortoisehg to use openssh. It’s also very helpful if you’re forgetful like me and forget to start pageant, add new keys to it etc.

Just add or modify the ssh setting in the [ui] section of your .hgrc. In my case, I’m using very basic settings and already have the ssh executable in my path so my settings look like this:

... settings ...
ssh = ssh -2 -C -x

In order for this to work, $HOME must be set such that openssh can find the existing private keys and configuration in $HOME/.ssh. Set the ssh parameters to your personal preference, in my case I’m requesting it to use ssh protocol v2 only (-2), disable X forwarding because I don’t need it (-x) and enable compression (-C).

Improving the performance of Git for Windows

Admittedly I’m  not the biggest fan of git – I prefer Mercurial – but we’re using it at work and it does a good job as a DVCS. However, we’re mostly a Windows shop and the out of the box performance of Git for Windows is anything but stellar when you are using ssh as the transport for git. That’s not too much bother with most of our repos but we have a couple of fairly big ones and clone performance with those matters.

I finally got fed up with the performance after noticing that cloning a large repository from the Linux git server to a FreeBSD box was over an order of magnitude faster than cloning the same repository onto a Windows machine. So I decided to start digging around for a solution.

The clone performance left a lot to be desired using either PuTTY or the bundled OpenSSH as the SSH client. I finally settled on using OpenSSH as I find it easier to deal with multiple keys. Well, it might just be easier if you’re a Unixhead.

My search led me to this discussion, which implies that the problem lies with the version of OpenSSH and OpenSSL that comes prepackaged with Git for Windows. The version is rather out of date. Now, I had come across this discussion before and as a result attempt to build my own SSH binary that included the high performance ssh patches, but even after I got those to build using cygwin, I never managed to actually get it to work with Git for Windows. Turns out I was missing a crucial detail. It looks like the Git for Windows binaries ignore the PATH variable when they look for their OpenSSH binaries and just look in their local directory. After re-reading the above discussion, it turned out that the easiest way to get Git for Windows to recognise the new ssh binaries is to simply overwrite the ones that are bundled with the Git for Windows installer.

*** Bad hack alert ***

The simple recipe to improve the Git performance on Windows when using a git+ssh server is thus:

  • Install Git for Windows and configure it to use OpenSSH
  • Install the latest MinGW system. You only need the base system and their OpenSSH binaries. The OpenSSH and OpenSSL binaries that come with this installation are much newer than the ones you get with the Git for Windows installer.
  • Copy the new SSH binaries into your git installation directory. You will need to have local administrator rights for this as the binaries reside under Program Files (or “Program Files (x86)” if you’re on a 64 bit OS). The binaries you need to copy are msys-crypto-1.0.0.dll, msys-ssl-1.0.0.dll, ssh-add.exe, ssh-agent.exe, ssh-keygen.exe, ssh-keyscan.exe and ssh.exe

After the above modifications, the clone performance on my Windows machine went from 1.5MB/s – 2.5MB/s to 10MB/s-28MB/s depending on which part of the repository it was processing. That’s obviously a major speedup for cloning, but another nice side effect is that this change will result in noticeable performance improvements for pretty much all operations that involve a remote git repository. They all feel much snappier.