On Not Optimising for Last Century's Hardware

Published 20 November 2016

Once upon a time I wrote a super-optimised algorithm for rotating data in an array. At least, it was meant to be super-optimised, but its real-world performance turned out to be terrible. That’s because my intuition about performance was stuck in the 20th century:

Break a program down into basic operations like multiplication and assignment
Give each operation a cost (or just slap on an O(1) if you’re lazy)
Add up all the numbers
Try to make the total in step #3 small

A lot of textbooks still teach this “classic” thinking, but except for some highly constrained embedded systems, it just doesn’t work that well on modern hardware.

Switching from StartSSL to Let's Encrypt

Published 06 November 2016

Tags: Ops and Security

When I first set up this blog I got an SSL certificate from StartCom’s StartSSL authority, simply because it was the easy way to just get a site on HTTPS.

However, StartCom has recently fallen out of favour with browser vendors, starting with Mozilla and now Chrome. If you haven’t been following the world of certificate authority politics, the story starts with a CA called WoSign. A university sysadmin created a big tech news story earlier this year after revealing that WoSign gave him a certificate for github.com. This is a pretty big deal, and Mozilla’s investigations into WoSign have raised even more concerns about WoSign’s mistakes and handling of mistakes. One of those concerns is this:

[In November 2015] WoSign purchased the CA “StartCom” and did not disclose the transaction as a change of ownership, which we believe violates section 5 of the Mozilla CA Certificate Maintenance Policy. Furthermore, when this clause was brought to their attention, they denied that any changes fell under it, and they attempted to suppress further information about the ownership transfer when it was brought to the community’s attention.

A couple of weeks ago Mozilla announced that Firefox will be phasing out support for both WoSign and StartCom certificates. Chrome made a similar announcement a week later.

Function Attributes and the D ABI

Published 05 October 2016

Tags: dlang , Software Engineering and Low Level

The D programming language has a bunch of built-in attributes like pure and nothrow. I was wondering how things like libraries might break if function attributes changed between versions, so I gave it a try.

A Tale of Three Server Caching Architectures

Published 30 July 2016

Tags: Systems Design , Servers , Performance and Software Engineering

Exactly where you put caching in a distributed system has a significant impact on its effectiveness, in ways that aren’t always obvious during the design phase of development.

DConf 2016 Talks

Published 24 July 2016

Tags: dlang and Software Engineering

The DConf 2016 talk videos have been released.

Here’s my (very short) list of special recommendations. They’re worth watching even if you don’t use D.

A Quick and Hacky Way to Serve a Git Repo over HTTP

Published 02 July 2016

Tags: Ops and Tools

More and more development tools are supporting git repository URLs as a way of pointing to code or data. That’s convenient if you’re working with a mature, third-party repository that’s already hosted, but it means that sometimes you just need a quick way to serve a repo over a web interface for experimental work.

Code Versus Data

Published 21 June 2016

Tags: Software Engineering and Systems Design

Some years back I wrote a program in C++ using an obvious object-oriented architecture. Then, later, I had to rewrite it in C, and I learned some pretty good lessons about software design.

Offline Compression with Nginx

Published 06 June 2016

Tags: Ops , Servers and Performance

There’s a clear tradeoff with compressing HTTP responses on the fly: compress “harder” and you’ll (hopefully) get a smaller file that takes less time to send over the network – but the net benefit might be negative if the extra work takes too much time, or (when under heavy load) too much CPU. A lot of work has been done analysing this tradeoff, but for static content there’s a neat and simple way to avoid the tradeoff completely: compress offline before serving. Nginx supports this using the gzip_static module.

The Enterprise Pushbutton

Published 11 May 2016

Tags: Software Engineering , Anecdotes and Low Level

Let’s talk about a hardware driver for a pushbutton. A pushbutton driver isn’t as completely trivial as it might sound because you need debouncing logic to ensure a crisp on/off signal, but it’s hard to imagine how it might need more than about 100 lines of C code.

After working on this particular embedded system, I didn’t need to stress my imagination any more. This pushbutton driver was modelled as an explicit finite state machine, and all the possible states and transitions were specified in a spreadsheet. Then there was a python script that processed this spreadsheet and generated state table data as C code. This was linked to an FSM evaluator in C. The FSM was controlled by a bare-metal driver and triggered callbacks on each state transition.

Most of the callbacks were marked “not yet implemented”. In fact, only two states were even reachable: BUTTON_UP and BUTTON_DOWN. Eventually the entire project was canned, but not because of missing support for BUTTON_TIMEOUT or any of the other states.

The FSM didn’t do any debouncing, so the low-level driver had to do that before passing button up/down events to it.

Turning any Live CD ISO into a Live USB with Writable Storage Partitions

Published 21 April 2016

Tags: Ops and Tools

Here’s a guide to installing a bootable CD-ROM ISO image as a bootable partition on a (USB) drive, while using other partitions as normal filesystems. The most complicated part is partitioning and formatting the drive, which is well documented elsewhere, but I didn’t see anyone else explaining exactly how to do this trick.