Understanding a *nix Shell by Writing One

Published

Tags: , , , and

A typical *nix shell has a lot of programming-like features, but works quite differently from languages like Python or C++. This can make a lot of shell features — like process management, argument quoting and the export keyword — seem like mysterious voodoo.

But a shell is just a program, so a good way to learn how a shell works is to write one. I’ve written a simple shell that fits in a few hundred lines of commented D source. Here’s a post that walks through how it works and how you could write one yourself.

Using the Bourne Shell as a Cheap and Nasty Templating Language

Published

Tags: , and

Sometimes you need a really simple way to generate parameterised text without pulling in a full-blown templating language as a dependency — for example, when writing an install script that needs to generate a simple configuration file. Using the classic *nix Bourne shell that’s installed on practically every *nix system is one option. To be honest, it can be a terrible option, but it often gets simple jobs done, so I think it’s a trick worth remembering.

Popular Web Servers Compared

Published

Tags: , and

Here’s a comparison of the web servers I’ve used the most.

Relational Databases Considered Incredibly Useful

Published

Tags: , , , and

A year ago I worked on a web service that had Postgres and Elasticsearch as backends. Postgres was doing most of the work and was the primary source of truth about all data, but some documents were replicated in Elasticsearch for querying. Elasticsearch was easy to get started with, but had an ongoing maintenance cost: it was one more moving part to break down, it occasionally went out of sync with the main database, it was another thing for new developers to install, and it added complexity to the deployment, as well as the integration tests. But most of the features of Elasticsearch weren’t needed because the documents were semi-structured, and the search queries were heavily keyword-based. Dropping Elasticsearch and just using Postgres turned out to work okay. No, I’m not talking about brute-force string matching using LIKE expressions (as implemented in certain popular CMSs); I’m talking about using the featureful text search indexes in good modern databases. Text search with Postgres took more work to implement, and couldn’t do all the things Elasticsearch could, but it was easier to deploy, and since then it’s been zero maintenance. Overall, it’s considered a net win (I talked to some of the developers again just recently).

Terraform is Best for Configuring Hashicorp Vault

Published

Tags: , and

Hashicorp Vault is a handy tool for scalable secrets management in a distributed system or team-based project. Unfortunately, the only out-of-the-box way to configure it is through its API (or a UI), but most projects that need Vault will need to manage the configuration in source control.

There’s a workaround explained on the Hashicorp blog. It’s a neat hack, but here’s a quick note about why using Terraform’s Vault integration is a better idea for production use.

A Quick and Hacky Way to Serve a Git Repo over HTTP

Published

Tags: and

More and more development tools are supporting git repository URLs as a way of pointing to code or data. That’s convenient if you’re working with a mature, third-party repository that’s already hosted, but it means that sometimes you just need a quick way to serve a repo over a web interface for experimental work.

Turning any Live CD ISO into a Live USB with Writable Storage Partitions

Published

Tags: and

Here’s a guide to installing a bootable CD-ROM ISO image as a bootable partition on a (USB) drive, while using other partitions as normal filesystems. The most complicated part is partitioning and formatting the drive, which is well documented elsewhere, but I didn’t see anyone else explaining exactly how to do this trick.