Unfortunately, Garbage Collection isn't Enough

Published

Tags: , , and

Here’s a little story of some mysterious server failures I had to debug a year ago. The servers would run okay for a while, then eventually start crashing. After that, trying to run practically anything on the machines failed with “No space left on device” errors, but the filesystem only reported a few gigabytes of files on the ~20GB disks.

Understanding a *nix Shell by Writing One

Published

Tags: , , , and

A typical *nix shell has a lot of programming-like features, but works quite differently from languages like Python or C++. This can make a lot of shell features — like process management, argument quoting and the export keyword — seem like mysterious voodoo.

But a shell is just a program, so a good way to learn how a shell works is to write one. I’ve written a simple shell that fits in a few hundred lines of commented D source. Here’s a post that walks through how it works and how you could write one yourself.

Using the Bourne Shell as a Cheap and Nasty Templating Language

Published

Tags: , and

Sometimes you need a really simple way to generate parameterised text without pulling in a full-blown templating language as a dependency — for example, when writing an install script that needs to generate a simple configuration file. Using the classic *nix Bourne shell that’s installed on practically every *nix system is one option. To be honest, it can be a terrible option, but it often gets simple jobs done, so I think it’s a trick worth remembering.

Popular Web Servers Compared

Published

Tags: , and

Here’s a comparison of the web servers I’ve used the most.

Terraform is Best for Configuring Hashicorp Vault

Published

Tags: , and

Hashicorp Vault is a handy tool for scalable secrets management in a distributed system or team-based project. Unfortunately, the only out-of-the-box way to configure it is through its API (or a UI), but most projects that need Vault will need to manage the configuration in source control.

There’s a workaround explained on the Hashicorp blog. It’s a neat hack, but here’s a quick note about why using Terraform’s Vault integration is a better idea for production use.

If You Need Lifeboats, That Means Your Ship is Sinking

Published

Tags: , and

It’s 1912 and Captain Edward Smith is boarding the RMS Titanic. He sees the lifeboats on deck and shakes his head with a heavy sigh before turning to the crew. “In my experience, I’ve never needed lifeboats. They’re not best practices — if you need lifeboats, that means your ship is sinking!” The crew members are enlightened and eagerly throw all lifeboats overboard. The Titanic begins its voyage to New York.

Switching from StartSSL to Let's Encrypt

Published

Tags: and

When I first set up this blog I got an SSL certificate from StartCom’s StartSSL authority, simply because it was the easy way to just get a site on HTTPS.

However, StartCom has recently fallen out of favour with browser vendors, starting with Mozilla and now Chrome. If you haven’t been following the world of certificate authority politics, the story starts with a CA called WoSign. A university sysadmin created a big tech news story earlier this year after revealing that WoSign gave him a certificate for github.com. This is a pretty big deal, and Mozilla’s investigations into WoSign have raised even more concerns about WoSign’s mistakes and handling of mistakes. One of those concerns is this:

[In November 2015] WoSign purchased the CA “StartCom” and did not disclose the transaction as a change of ownership, which we believe violates section 5 of the Mozilla CA Certificate Maintenance Policy. Furthermore, when this clause was brought to their attention, they denied that any changes fell under it, and they attempted to suppress further information about the ownership transfer when it was brought to the community’s attention.

A couple of weeks ago Mozilla announced that Firefox will be phasing out support for both WoSign and StartCom certificates. Chrome made a similar announcement a week later.

A Quick and Hacky Way to Serve a Git Repo over HTTP

Published

Tags: and

More and more development tools are supporting git repository URLs as a way of pointing to code or data. That’s convenient if you’re working with a mature, third-party repository that’s already hosted, but it means that sometimes you just need a quick way to serve a repo over a web interface for experimental work.

Offline Compression with Nginx

Published

Tags: , and

There’s a clear tradeoff with compressing HTTP responses on the fly: compress “harder” and you’ll (hopefully) get a smaller file that takes less time to send over the network – but the net benefit might be negative if the extra work takes too much time, or (when under heavy load) too much CPU. A lot of work has been done analysing this tradeoff, but for static content there’s a neat and simple way to avoid the tradeoff completely: compress offline before serving. Nginx supports this using the gzip_static module.

Turning any Live CD ISO into a Live USB with Writable Storage Partitions

Published

Tags: and

Here’s a guide to installing a bootable CD-ROM ISO image as a bootable partition on a (USB) drive, while using other partitions as normal filesystems. The most complicated part is partitioning and formatting the drive, which is well documented elsewhere, but I didn’t see anyone else explaining exactly how to do this trick.