Dreamdir and the Unix Philosophy, or Likable Software

This weekend I released version 2.0.0 of my open-source Dreamdir file format and associated application, termed dr. Dreamdir is one of my tools for keeping personal records, specifically for keeping a dream journal.

I realized while I was working on the series of updates that led up to this public release that dr is one of a very few pieces of software that I actively like. Let me expand on the meaning of liking software. The majority of software, of course, sucks. I am willing to tolerate a somewhat smaller category of software – without swearing all the time while I’m using it – because of what it lets me do, even if it’s not that great. I respect a smaller yet but still moderately large category of software; anything that does its job well, isn’t horribly ugly, and is designed in a consistent, usable way earns my respect (those three conditions are quite hard enough to meet). But it’s real hard to reach the like category – that’s software that makes me happy when I use it, regardless of what I’m actually doing with it. Believe it or not, such software actually does exist!

Lest you think I’m just a self-promoting narcissist, I think dr is the only software I’ve personally written or even contributed to that I like, and I’ve written quite a bit of software. My Me/Software folder contains plenty of respected programs and more than a few tolerated ones, but no other liked ones. Here are several other tools written by others that come to mind when I think about software I like:

• Git: “the stupid content tracker,” for tracking different versions of software, documentation, or any other files
• Windows PowerShell: Microsoft’s scripting language, remarkably consistent, easy to use, and even fun to work with
• vim: the text editor I live half my life in

Side note: Everyone knows Linus Torvalds for his Linux operating system, which he started in his basement as a fun project and which now runs most of the world’s server infrastructure and smartphones. But I think his real masterpiece is Git, which he took a few weeks off to write because the version control system Linux development was using was no longer available to them. It’s probably not possible to fully appreciate Git without using it, but the design is beautiful and the power and productivity of the system is phenomenal.

How Dreamdir works

To see why I find dr (and the other tools I listed) “likable,” we need to understand a little bit about Dreamdir.

The file format

Dreamdir relies on a specially formatted kind of text file – the kind you can open in Windows Notepad, with no bold text or anything in it. In that way, it’s really simple! The user creates a new file for each entry, all in the same folder (a.k.a. “directory”, the “dir” part of the name). Here’s what one of those entries looks like:

Id:	01461
Date:	2019-02-04
Time:	3:20
Title:	A Violation of Campanile 8
People:	K.S. [redacted]
Tags:	moving, law, lease, phone

[...text clipped for brevity] Somehow I do eventually get [my landlord] to
agree that I have the right to stay in my apartment until the lease
terminates, and I explain that I might very well leave earlier, but I
certainly *can* stay there until the 31st if I so choose.

We start out with a series of headers, which are basically key-value pairs: the ID is 1461 (meaning this is the 1,461st dream in the collection), the date is February 4, 2019, and so on. A few of them can have multiple values, like Tags (moving is a tag, law is a tag, and so on). Then there’s a blank line and the actual text of the entry.

The Dreamdir user can write these simple files herself using any tool she likes. I use vim, of course, since that’s another piece of software I like, but there are literally hundreds of text editors that would serve this purpose just fine. The dr program itself isn’t even involved yet.

The program

The main purpose of the actual program, dr, is also simple: it searches through all the dreams in your collection. Searches are composed of two parts: an expression (which indicates what you want to find), and an action (which indicates what you want to do with what you find). You communicate with dr by typing combinations of actions and expressions. Lest this simple concept sound technical, here are a few examples:

• dr edit 45: find the dream with ID number 45 and open it in a text editor
• dr word-count 101-200: find the dreams with ID numbers 101 through 200 and count the number of words in them
• dr get-header Title random 5: randomly select 5 dreams from the collection and show their titles
• dr header-values -f People 500-: look through all dreams with an ID number greater than 500, tally up all the people mentioned in them, and list out each name along with how many times the person is mentioned
• dr find grep jabberwock: display a list of the ID numbers and the total number of dreams which contain the word jabberwock (the explanation of the origin of the word grep is too long for this article, but it’s basically Linux-speak for “find something in text”)
• dr filename-display 100-200 300-400 | dr winnow -r tagged Places Owatonna | dr act dump-headers: find dreams 100 through 200 and 300 through 400, filter that list to retain only the ones that took place in Owatonna, reversing the order so the newest results appear at the top, and display the header portion of those dreams (i.e., the part before the first blank line).

Note: You may notice that actions are basically verbs and expressions are basically nouns. We’ll talk more about the role language has to play in likable software momentarily.

You can see that these queries can be combined to answer virtually any question a user might have about her collection of dreams, provided the dreams have appropriate metadata fields or keywords to help out.

We saw only the input above and not the output. To give you the full picture, a typical session looks something like this, where $ indicates the prompt at which you type searches:

$ dr find grep jabberwock
No results.

$ dr find grep dragon
3 matches: [431, 639, 1200]

$ dr header-values -f Tags
    107 device
    100 driving
     93 school
     91 travel
     76 computer
     66 OC
     66 train
     64 music
     49 church
     44 elevator
     [...]

Integrating with other programs

Actually, the queries and actions can get even more complex, because we can almost effortlessly integrate with other programs too, using Unix’s concept of pipes (|), which feed information from one program into another:

• dr header-values -f Tags | awk '$1 >= 20': display all tags used and how often they’re used, showing only those that occur at least 20 times (awk is a program that performs inline edits to text in accordance with rules you supply; in this case, it shows only the lines where the part before the first space is a number greater than or equal to 20)
• dr filename-display date '<' 2018-01-01 | xargs cp -t old-dreams: copy all of the dreams dated 2017 or earlier to a folder called old-dreams (xargs is a program that executes a command, here the cp copy-file command, on each of the files piped to it)
• dr cat tagged People Jane | mutt "contact@sorenbjornstad.com" -s "Dreams with Jane": use the mutt email program to send an email to my contact address containing all the dreams that Jane appeared in (cat stands for concatenate, i.e., smash the text of those dreams together in sequence, not for the furry animal you keep in your house for company and pest control).

Let’s think about that last example for a second. If we didn’t have a program as flexible as dr, how would we even start going about the task of emailing someone a copy of all the dreams that reference a specific person? We’ll assume we’re using specialized software designed for keeping a dream journal, rather than just a Word document or something (which is probably much more common). Even with this specialized software, it’s highly unlikely the author of dream journal software would have envisioned this exact situation, or even included any kind of feature to email a dream to someone, much less a series of dreams matching some criteria. Thus, we would have to open up the program we were storing the dreams in, manually go through and find the appropriate dreams, either by filtering or even by reading them carefully if a search feature capable of specifically identifying people didn’t exist, copy and paste them individually into a new email in a different program, fix any formatting inconsistencies we created by haphazardly copying and pasting, address the email, and send it.

With dr, because of its modularity and interoperability, I didn’t envision this use case when I wrote dr, and yet it took me all of 30 seconds to make it happen. All I had to do to compose this command was read mutt --help to remind myself of the -s option for setting the subject, and I was set! The part of the command before the pipe is one I already use all the time, so I just had to know what other command to feed that information into to email it. dr – and its Unix environment with pipes – let me tell the computer exactly what I wanted it to do, rather than having to do a long, boring task myself.

Qualities of likable software

I don’t have a precise formula for creating likable software, and I’m certain that nobody does. If anyone had figured out how to consistently turn out mass-market likable software, they would be making billions and everyone would be raving about their amazing software. (Apple probably comes the closest of any major company making software today, but its ecosystem falls far short in interoperability, and its price makes it inaccessible or distasteful to many.) I do, however, know a few qualities that seem to be common to likable software, including my Dreamdir tool and the others I listed at the beginning of this article, and that make software, if not likable, at least more likely to be respectable. I think most of the qualities are part of what’s often known as the Unix philosophy. Here are the top four qualities likable programs have:

• Feature creep is avoided like the plague. That means leaving out and sometimes even removing features that aren’t necessary. To the extent practical, each program should do exactly one thing, and take advantage of that simplicity of mission to do it well. As everywhere in life, trying to do everything well at once invariably yields (at best) mediocre results. dr contains less than 1,500 lines of code; it does the minimum necessary, and I don’t make up features in advance, I add them when I repeatedly try to do something and find it awkward, so I know they’ll be useful.
• The program defines a flexible language that users can work with. That is, the program allows users to combine simple building blocks to do exactly what they want. Users are not forced to choose from a small set – or even a large set – of predefined actions. As I mentioned in the linked article, vim is the classic example of a program that uses language well.
• Programs for simple use cases are simple. Programs for complex use cases can be complex, but they should be based on as simple a model as possible. For instance, dr has a lot of search functions, but it ultimately works on a few fields of text, and anyone can understand what the fields are for and what the search functions do. If many options are warranted, it’s okay to give the program hundreds of them – but users should only have to dig into those options when they really need them. Git is an excellent example of this design done well; you can start tracking versions of your software and sharing them with others knowing just the commands git add, git commit, git push, and git pull, and punching something into Google if you run into trouble. But the documentation for the git log command, which shows your previous versions, also lists over 30 pages of options if you need something special.
• In order to be usable, programs that do exactly one thing well integrate well with other programs. This way, when users encounter something the program doesn’t do well – which will surely happen sooner or later no matter how complete it is – they can easily delegate that task to a different program that does do it well. Sometimes this comes in the form of extensions or add-ons to the program itself, sometimes an import/export feature, and sometimes a Unix or PowerShell pipe.

The future of likable software

The Unix philosophy doesn’t by any means require using Unix or its descendant Linux. For all their benefits, and all the people who swear by them, Unix and Linux are arcane, hard to learn, inconsistent, unforgiving of mistakes, and burdened by 60 years of historical baggage – and I say this as a longtime user and a fan. I, and many others, still use Unix-based systems because as whole platforms, they follow these principles excellently, unlike any other platform out there today, and the benefits of those principles judiciously applied are so great that they easily outweigh the disadvantages of the Unix platforms, at least for a skilled user. But we could absolutely do much better!

I mentioned earlier that I like Windows PowerShell. PowerShell is a huge step in the right direction; it brings parts of the Unix philosophy to Windows, a platform where interoperability and the potential for automation and language-based software has historically been horrendous. Further, because of its centralized design and the benefits of starting over after the world had seen the mistakes of 50 years of development on a similar platform that never got to start over, it’s clean and consistent in a way that Unix has never been. It’s at least a little bit harder to accidentally erase your whole hard drive due to a typo. That’s not to say it’s perfect; as with any product, certain design decisions Microsoft made are at best debatable and at worst completely brain-dead. But PowerShell, overall, is a significant advance in communication with computers and a future where we have more likable software.

Unfortunately, it’s still a little island of sensibility in a sea of ugly, bloated, static Windows programs, and Microsoft seems to be bizarrely targeting it only at system administrators, when any serious computer user working on any task could benefit from judiciously applied PowerShell, if only people were writing more PowerShell modules. I can easily imagine a world where when you signed up with a company – say, a bank – you wouldn’t just get a link to its website, you’d also get information about where to download their PowerShell module to manage your money and your account. It’s really that easy to use, and that powerful. Of course it wouldn’t have to be PowerShell either; any system that offered small tools that do one thing well, integrate well with other tools, and allow using language to interact with the computer, and that made it easy to develop additional such tools, could fit the bill.

But for now, I’m sitting here with dr 2.0, running Linux.

---

If Dreamdir sounds like something you might like to use yourself, check it out on GitHub.