Improved syntax highlighting with org2blog and custom CSS

As mentioned in an earlier post, I changed my blogging workflow to org2blog for writing and editing posts in Emacs and only push them up to my WordPress blog when the posts are almost done. I still do the final editing in WordPress so I can tweak the SEO settings and all that, but the majority of the work happens in org-mode now.

One area that really needed improving was the appearance of the source code that I put in posts. Before I started using org-mode for blogging I edited the code in Emacs, ran it through htmlize and pasted it into my blog post.. That way the Emacs theme I’m using also determined what the generated HTML looked like and it allowed me to control the code’s appearance to a certain extent. Once I moved to org-mode, I just got a generic htmlize output.

In order to improve the appearance output, I switched from the inline styled export to CSS-based export. That way, I can tweak the CSS for the basic syntax highlighting, pick appropriate background colours and all that fun stuff.

First, prepare WordPress for additional custom CSS

The web is full of warnings that you shouldn’t edit your installed WordPress theme so you don’t lose all your customisations the next time the theme is updated. The usual recommendation is to create a child theme instead. This does exactly what I needed to, but felt awfully heavyweight for the comparatively simple task at hand. All I wanted was jam some additional CSS onto a page and be done with it.

I’m using Jetpack on my self-hosted WordPress install for certain additional features that you’d normally only get when hosting a blog on WordPress.com. Jetpack has a neat feature called “Custom CSS” that allows you to add additional CSS to your existing theme’s stylesheets. In other words, it allows you to provide exactly the kind of customisation I need to tweak the source code highlighting. If you haven’t activated the Custom CSS feature yet, go to your Jetpack settings and activate it.

Adding org-mode’s code export CSS

This is a relatively simple operation. First, you need to tell org-mode to export the source code formatting information using css and not the more usual inline style. This is easily done by adding the following setting to your .emacs and re-evaluate that particular piece of code:

(setq org-export-htmlize-output-type 'css)

Once that’s done, the easiest way to get the parts of the CSS that are relevant to source code formatting is to create an org file, stick a #+BEGIN_SRC/#+END_SRC block in there and export that to HTML. You can then pull the relevant parts of CSS out of the resulting HTML buffer and put it into the custom CSS settings provided by Jetpack.

In my case, the relevant CSS looks like this:

pre {
  border: 1px solid #ccc;
  box-shadow: 3px 3px 3px #eee;
  padding: 8pt;
  font-family: monospace;
  overflow: auto;
  margin: 1.2em;
  -moz-border-radius: 5px;
  border-radius: 5px;
}
pre.src {
  position: relative;
  overflow: visible;
  padding-top: 1.2em;
  background-color: DarkSlateGrey;
  color: WhiteSmoke;    
}
pre.src:before {
  display: none;
  position: absolute;
  background-color: white;
  top: -10px;
  right: 10px;
  padding: 3px;
  border: 1px solid black;
}
pre.src:hover:before { display: inline;}
pre.src-sh:before    { content: 'sh'; }
pre.src-bash:before  { content: 'sh'; }
pre.src-emacs-lisp:before { content: 'Emacs Lisp'; }
pre.src-R:before     { content: 'R'; }
pre.src-perl:before  { content: 'Perl'; }
pre.src-java:before  { content: 'Java'; }
pre.src-sql:before   { content: 'SQL'; }
pre.src-clojure:before { content: 'Clojure'; }

Copy your customized CSS into Jetpack’s “Custom CSS” edit control, save your changes and you’re done. The next time you publish a post from org-mode using org2blog, it’ll automatically apply the above CSS to all your code blocks that were marked as such.

GNU Emacs 24.5 on (X)ubuntu 14.10

GNU Emacs 24.5 was released on April 10th. I’m in the process of setting up a dual boot Windows/Linux machine right now as I’m slowly moving away from Mac OS X, mainly because of the cost of the hardware but also because I don’t like it that much as a Unix-y development environment anymore.

Xubuntu 14.10 only comes with Emacs 24.3 and it looks like 15.04 will “only” include 24.4 so now is a good time as any to manually install 24.5. Please note this is not a complaint about Ubuntu and its many contributors. They do a great job, I’d rather have a stable system comprised of well-tested packages and install the exciting, cutting edge stuff manually.

A manual “out of the box” install is actually pretty easy. I followed the instructions for installing 24.4 on UbuntuHandbook, substituting 24.5 where appropriate. I also installed Emacs in a user-local directory tree as I really don’t like mixing system-wide tools that have been installed by the package manager and manually installed packages in the same directory tree. Yes, I’m weird like this.

All in all the install was completely hassle free and allows me to send a test blog post from Emacs 24.5 on my new Xubuntu box to the blog using org2blog as usual.

Update: The instructions linked to above also work for Ubuntu 15.04 – you have to rebuild your Emacs 24.5 for 15.04 as some of the dependencies have changed.

Setting up Emacs spell checking on OS X

As I mentioned in yesterday’s post, I’m trying to improve my blogging workflow by using org2blog to draft my posts before pushing them to my WordPress blog. When I posted yesterday I had the basic workflow going, could edit posts in Emacs, save them, update drafts and push them to WordPress. The last piece that was missing was getting spell checking to work.

I’ve actually never spent much time thinking about spell checkers until I discovered that OS X doesn’t come with a spell checker that ispell recognises. A little research led me to Joel Kuiper’s blog post on spell checking in Emacs on Mac OS X. I decided to install Hunspell as it seemed to be modern, supported and able to do the job. Plus, it’s available via Homebrew which I’m already using to install other Unix software on my OS X machine. A quick

brew install hunspell

followed by me grabbing the German and English dictionaries from OpenOffice.org and I was in the spelling business. Just make sure that you unzip the .oxt files containing the dictionaries, copy/move the .dic and .aff files into either ~/Library/Spelling or /Library/Spelling (I prefer the first one) and run hunspell -D to check that it’s picking up the files. Oh, and don’t forget to set up soft links from default.dic and default.aff to your default language’s dictionary files.

Now I should be able run M-x flyspell-mode and we’re in business, correct? Only we weren’t, because my Emacs setup couldn’t find hunspell. Oops. Turns out I had neglected to add the Homebrew installation directory to the exec-path, with predictable results. The following code in my .emacs fixes that:

;; On OS X/Darwin, make sure we add the path to the homebrew installs
(when (string-equal system-type "darwin")
  (setq exec-path (append exec-path '("/usr/local/bin"))))

With the above in place and Emacs being able to find the hunspell executable, it was time to add the following code to my .emacs to ensure that ispell and flyspell use hunspell if it’s available:

(when (executable-find "hunspell")
  (setq-default ispell-program-name "hunspell")
  (setq ispell-really-hunspell t))

Now flyspell-mode is happily running using Hunspell at it’s time to crack open an adult beverage.

Improving my blogging workflow using Emacs (of course)

I try not to post too many metablogging posts. Other people do it better and I’m trying to focus on journalling what I learn as a software engineer and manager, not what tools I use for blogging. However after losing another post to WordPress’s built-in editor I decided Something Must Be Done. I think this is only the second post I lost, but it’s a fairly regular occurrence for a journalist friend of mine and I really don’t have that much time to retype blog entries that ended up in Bit Nirvana.

My first attempt was to resurrect the weblogger-mode setup I used to have a while ago but after switching the admin interface on my WordPress install to https, I couldn’t quite get it to work again. Plus it was a bit of a half hearted attempt as I never quite warmed to this mode in the first place. It’s actually quite odd as I tend to use gnus semi-regularly and the interface is very similar, but it never quite clicked for me for writing blog posts.

If I would exclusively blog on Windows, I’d just use Windows Live Writer, but as I switch between Windows, OS X, Linux and FreeBSD depending on which machine I’m on, Windows only software just isn’t going to cut it.

As everybody raves about org-mode (which I admittedly have never used) I decided to give org2blog a chance. It’s probably not the smartest idea to try to learn too many new tools at the same time but at least Emacs doesn’t occasionally eat my scribblings. Plus, I’ve started using Jekyll for another one of my experimental blogs, so using org mode and having they ability to publish to a Jekyll blog is also very useful.

So far I’ve got the basics up and running and the main blog configured. I’m using visual-line-mode to do automatic line wrapping and now will have to set up flyspell on the machines that haven’t got it installed yet so I can have basic spell checking.

So far, the basic workflow I’m planning is:

  • Sketch the post(s) and write the drafts in Emacs in the comforts of my local machine
  • Publish them as drafts to my standalone WordPress install
  • Do the final editing and spill chucking in WordPress
  • Ignore or heed the recommendations from the WordPress SEO plugin. That’ll be mostly ignore, then
  • Schedule the final publishing on the WordPress admin console

Hopefully that should work better than the “log into WordPress and start typing” approach I’ve used so far.

Large file handling in Emacs revisited, with a quick look at VLF

I recently blogged about installing a 64-bit build of Emacs for Windows because I was dealing with a bunch of large and very large files.

While the 64-bit build definitely handled the really large files much better than the 32-bit build, there were still some performance issues. The main advantage of using the 64 bit build was that I could finally load a couple of files that I wasn’t able to load on the 32-bit build, but opening the files severely tested my patience.

The other problem that I didn’t think about initially was that if you happen to have desktop-save-mode turned on, Emacs will of course reload the large files every time you restart it, unless you remember to close large files before shutting down Emacs. That tends to take a considerable amount of time because either build of Emacs is rather slow when it comes loading files in the hundreds of megabytes or bigger category.

A commented in the previous post kindly suggested to use VLF for large files, which is what I’ve been doing instead. I keep forgetting to turn on the integration which should save me from having to remember to manually kick off vlf every time I want to open a large file, but other than that it just works.

Highly recommended if you need to deal with large files from time to time.

Using ELPA with pinned packages in GNU Emacs 24.4

Yes, I promise I’ll shut up about Emacs package management via ELPA any minute now.

Based on the feedback I had on my last post about using a combination of melpa and melpa-stable, I looked into using pinned packages via the package-pinned-packages variable that’s new in Emacs 24.4’s package.el. I couldn’t find any simple examples on how to use it, but a quick look at the source code and some playing around in ielm got me there. Well, after I finally upgraded to Emacs 24.4 on my main machine.

Using pinned packages via package-pinned-packages is actually pretty simple. First, you need a list of ELPA package archives to pull packages from. If you’re already using ELPA, you most likely have package-archives set up already. Second, you need to create the list of pinned packages. For me that was the majority of the work and it required a few round trips through the *Packages* buffer. It would have been smarter to uninstall all the base packages, run list-packages once and note which package should be installed from which archive instead of relying on my famously selective memory. Don’t do what I did, just do it once. Either way, I got there. Eventually.

Once you have both variables set up, you’re left with a massively simplified loop that really just takes a list of packages and calls package-install on them if they’re not already present in the system.

For those who like me learn better by looking at code but like me couldn’t find an example on how to use package-pinned-packages, here’s the updated code. It’s Emacs 24.4 only, partially for clarity and partially because I don’t need to support anything older than 24.4 in my .emacs anymore.

 
(require 'package)

(when (>= emacs-major-version 24)
  (setq package-archives '(("ELPA" . "http://tromey.com/elpa/")
                           ("gnu" . "http://elpa.gnu.org/packages/")
                           ("melpa" . "http://melpa.org/packages/"<span">)
                           ("melpa-stable" . "http://stable.melpa.org/packages/")
                           ("marmalade" . "http://marmalade-repo.org/packages/")
                           )))

;; Check if we're on Emacs 24.4 or newer, if so, use the pinned package feature
(when (boundp 'package-pinned-packages)
  (setq package-pinned-packages
                '((bm                 . "marmalade")
                  (smex               . "melpa-stable")
                  (zenburn-theme      . "melpa-stable")
                  (anti-zenburn-theme . "melpa-stable")
                  (zen-and-art-theme  . "marmalade")
                  (cider              . "melpa-stable")
                  (clojure-mode       . "melpa-stable")
                  (htmlize            . "marmalade")
                  (rainbow-delimiters . "melpa-stable")
                  ;; "unstable" package
                  (icicles            . "melpa"))))

(package-initialize t)

(defun install-required-packages (package-list)
  (when (>= emacs-major-version 24)
    (package-refresh-contents)
    (mapc (lambda (package)
            (unless (require package nil t)
              (package-install package)))
          package-list)))

(setq required-package-list '(bm icicles smex zenburn-theme zen-and-art-theme htmlize cider clojure-mode rainbow-delimiters))
 

You can see the whole code is a lot more compact even with the formatting I use and most importantly, it’s a lot more readable. Keep in mind that I stripped out all the code that made the melpa-stable/melpa combination work in Emacs 24.3; the code gets more complicated again when you’re trying to accommodate both versions. For my use, I was happy to just get everything working for Emacs 24.4 after I upgraded the installs on my various machines.

So what’s the take away here?

Package management is hard. Well, all of us who have worked on medium and large software systems know that from first hand experience. Emacs is no different. I think having package-pinned-packages available is a nice feature in my use case. I only use the above code to bootstrap my various Emacsen and individual Emacs instances usually have a few more packages installed. If I wanted to pin all the packages I use I’d probably be grumpy by the end of the exercise but for the basic set of packages I use, this works better than my previous attempts.

Set up Emacs to use both melpa and melpa-stable

I’ve blogged about a little elisp snippet I use to install my preferred base set of Emacs packages before. Thanks for all the feedback, it definitely helped improve the code.

One issue that kept annoying me is that there is no simple way to tell ELPA to mainly pull packages from melpa-stable and only fall back to melpa for those packages I can’t get on melpa-stable yet. I decided to extend my code to handle that situation with some manual inputs as I know which packages can’t be found on melpa-stable. It proved surprisingly easy to do so after mulling over the problem a little.

First, I updated my function install-required-packages so that it accepts an optional parameter containing a list of packages repositories. When the parameter is non-nil, I make a temporary copy of the existing packages-archives list to preserve my default settings and replace it with the list that’s been passed in. Then the function checks and install the packages as before and then restores the original package-archives variable. The code now looks like this:

 
(defun install-required-packages (package-list &optional package-archive-list)
  (when (>= emacs-major-version 24)
    (if package-archive-list
        (setq temp-package-archives package-archives
              package-archives package-archive-list))
    (package-refresh-contents)

    (mapc (lambda (package)
            (unless (require package nil t)
              (package-install package)))
          package-list)
    (if package-archive-list
        (setq package-archives temp-package-archives))))
 

As you can see, the function is now a just little more complicated thanks to the additional state preservation code. The big bonus  is that it now lets me specify which packages I don’t want to pull from my list of default repositories. To make things easier I also pre-populated the lists of my preferred ELPA repos:

 (setq stable-package-archives '(("ELPA" . "http://tromey.com/elpa/")
                                ("gnu" . "http://elpa.gnu.org/packages/")
                                ("melpa-stable" . "http://stable.melpa.org/packages/")
                                ("marmalade" . "http://marmalade-repo.org/packages/"))
      unstable-package-archives '(("melpa" . "http://melpa.org/packages/")))
 

Now I can simply tell ELPA which packages should be pulled from the default repositories and which ones come from the special repositories:

 
(install-required-packages '(smex zenburn-theme zen-and-art-theme htmlize cider clojure-mode rainbow-delimiters))
(install-required-packages '(bm icicles) unstable-package-archives)
 

Quite simple, isn’t it?

Obviously this is still work in progress. The whole approach feels clunky to me and that suggests there is room for improvement. Yes, it works and there is a lot to be said for that, but ideally I would like to build it out in such a way that I specify with package to pull from which repository and add functionality to semi-automatically update the packages as well. I don’t like the idea of fully automated upgrades – especially not from melpa – as I’ve ended up with broken packages before when I took that approach, but a manually triggered auto-update.

Isn’t it great that we spend hours customizing our tools to save five minutes?

The official GNU Emacs 24.4 build is available for Windows

Looks like the Windows build of Emacs 24.4 has been released to http://ftp.gnu.org/gnu/emacs/windows/ on November 15th. As usual, I appear to be a few days behind the times.

Time to upgrade and see how it compares to the unofficial 64 bit builds I’ve been using recently.

Top new feature in 24.4 for me so far is the new rectangle-mark-mode. I’m doing a bunch of code conversions right now – basically transferring a bunch of manual assignments into a big lookup table – and it’s seeing a lot of use because of it.

Finally trying out a 64 bit Emacs on Windows

I’ve been using the official GNU distribution of Emacs for Windows for the last few years and am very happy with it. Well, usually I am very happy with it until someone sends me a 25GB log file I need to analyse and the 32 bit Emacs refuses to play when faced with the enormity of the file in question.

This happened again recently so I finally decided to go look for a 64 bit Emacs for Windows, which led me to emacs-w64:

emacs-24-64-bit

I haven’t spent a lot of time with it (yet) but first impressions are very favourable. It starts quickly, seems responsive and, well, edits texts. In other words, I like it so far and I’ll keep using it for a while to see if first impressions hold up.

Oh, and there’s also the side benefit of being emacs-w64 already having a 24.4 build available. At the time of writing, the official GNU Windows build still appears to be 24.3.

MELPA has a new URL

MELPA has recently got its own domain (melpa.org) so it’s time to update your list of package repositories with the new URL.

Speaking of MELPA, I recently switched to their stable repository instead of their “regular” nightly build/snapshot repository after I accidentally ended up with a cider build that didn’t want to playing ball. This is not a complaint – if you use nightly builds etc you know what you’re getting yourself into – but it prompted me to switch over to using the stable package repository instead on those machines that I consider production machines. This of course require me to uninstall and reinstall a bunch of packages but that only took a few minutes.

Anyway, got update your ELPA/package.el configuration settings. And if you don’t use ELPA, spend the ten minutes or so to switch from manual package management to ELPA. Of course, if you are using a different package manager, feel free to ignore all my ramblings and keep using that one instead.