On Scales, Skills, and Leaky Abstractions

Imagine for a moment one of the old-style balance-beam scales at a doctor’s office. You get on the scale and the nurse slides the weights across the beam until it sits level, then records your weight in your chart. It requires a little bit of experience to use efficiently, but it’s straightforward: anyone with a few minutes to spare and a basic understanding of physics can figure out not only how to use it but exactly how it works. Moreover, if anything ever goes wrong with it (for instance, an angry patient grabs the beam and bends it out of shape, or something gets caught under the platform), the problem will be obvious. We could of course imagine a situation where this isn’t true (say, an evil spy breaks in at night with a file and a can of paint and secretly removes 1% of the material from the weights), but such situations are improbable or even absurd. And the balance-beam scale will work faithfully for decades.

Now think about one of the new “smart” bathroom scales. You step on the scale, it does some magic, and your weight and maybe some body composition info shows up on your smartphone. It’s pretty convenient: you don’t have to think about weighing things, copying down the number, converting units, or even noting what your weight was last week so you can see if you’re gaining or losing weight. The scale just does it all for you. The digital scale is an everyday example of an abstraction: it hides the details of the act of weighing something from the user, allowing them to focus on the results rather than the method.

Abstractions

Abstractions are an essential part of any technology, electronic or not. Even the simple balance-beam scale can be seen as an abstraction, albeit a less complicated one – the user of such a scale may have to do some work to weigh an object, but they don’t have to worry about finding appropriate reference weights or a straight beam to suspend weights from, much less understanding the work done by the makers of the reference weights (comparing against standards, understanding the relationship between weight, mass, and gravity, and so on).

A car is another great example. Instead of worrying about how to coordinate the components of your engine, when in the combustion cycle to add fuel to the cylinders, and so on, you just turn the key in the ignition and manipulate the gearshift, steering wheel, and pedals, and the car takes care of the details behind the scenes so you can easily move from place to place. This brings up another point about abstractions: because humans have limited short-term memory and execution speed, they allow us to do things that would be entirely impossible without them. No matter how good you are at driving, you cannot manually control all of the parameters of an engine while also driving your vehicle safely. (In fact, you can’t manually control everything even while not driving: to manually work an engine moving at thousands of revolutions per minute, you would have to have well past superhuman speed and precision.)

Leaks

For all their benefits, abstractions create a major problem. As software developer Joel Spolsky so elegantly put it in his timeless 2002 article, abstractions leak: that is, some of the details the abstraction is designed to manage and hide end up affecting the output it provides to its user. Moreover, this happens covertly: because the abstraction is hiding the details of how the output is created, the user doesn’t know why it doesn’t work the way he expects – or even that there’s a problem at all.

Take the digital scale. If it stops working as expected and changing the batteries doesn’t work, the user is likely forced to throw the entire thing away. If something is damaged inside the scale that affects its accuracy, he will never notice unless he has past measurements to compare it to or the value is so far off from the expected figure as to be impossibly wrong. Or consider the car. If you leave your lights turned on overnight and drain the battery or you run out of gas, the car’s interface won’t be able to fulfill its promise of moving when you turn it on, put it in gear, and press the gas pedal. If one of the sensors in a modern engine is damaged, the car will start performing erratically, and you’ll likely have to take it to a mechanic or study the workings of the vehicle in depth and invest in some specialized equipment to diagnose and fix the problem. Looking at the car or the scale only from its standard interface, it is impossible to tell what the problem is, yet you also can’t continue without figuring it out.

So the trick is to find the perfect balance (no pun intended). Too few abstractions and a system will waste its users’ time and attention with pointless details; too many abstractions and the leaks cause the system to become unpredictable, thus unreliable and frustrating to use.

Note: Unpredictability from the outside is the key characteristic of abstractions that leak. When a trained mechanic looks under the hood of a malfunctioning car, it may be obvious to her why the car is behaving the way it is. It might be easy and cheap to fix. The fact that a problem exists isn’t what makes leaky abstractions frustrating (indeed, most people enjoy solving problems that are well-defined and suited for their skill level); it’s the fact that the abstraction separates us from the problem so that we don’t know how to fix it. That separation makes us feel powerless and stupid and encourages us to get angry at the machine even though it’s behaving rationally and doesn’t have an agenda.

left-pad

My impression is that currently, most systems are heavily unbalanced to the side of too much abstraction. The left-pad fiasco in 2016 provided a particularly embarrassing and instructive example. left-pad was a JavaScript software package – basically, an abstraction written by someone else that software developers can link into their own code to avoid having to shift the weights along the beam themselves. The goal of left-padding is to make a string of text a fixed number of characters long by adding blank space or some other character to the left of it. For instance, if I left-padded the string the to 5 characters long with hyphens, I would get --the, while if I left-padded the string Soren, I would just get Soren, since it’s already 5 characters long. The code of the left-pad package is so brief that I’ll quote the entirety of it right here:

module.exports = leftpad;
function leftpad (str, len, ch) {
  str = String(str);
  var i = -1;
  if (!ch && ch !== 0) ch = ' ';
  len = len - str.length;
  while (++i < len) {
    str = ch + str;
  }
  return str;
}

Here’s exactly what this does, because it’s not complicated:

1. Define a piece of code – an abstraction – called leftpad that takes three inputs: a string of text to pad (in my example, the), a length to pad it to (5), and a character to pad with (-).
2. Convert the string to a data type String if it isn’t one already (the need for this step is JavaScript trivia irrelevant to this discussion).
3. Set a number called i to -1.
4. If the user didn’t give a value for the character to pad with, set it to a space.
5. Change the number called len to represent the number of padding characters we have yet to add.
6. Until we have added len padding characters, as counted by the value of the number we called i, tack one padding character at a time onto the front of the string.
7. When we’re done, make the resulting string the output of this code.

Any reasonably competent software developer could rewrite this in 10 minutes, tops, without looking at the reference – probably 5. So it’s not a particularly powerful abstraction, by itself. Yet when this silly little package was unexpectedly removed from a public repository, suddenly tens of thousands of software applications across the entire world would no longer build. A gigantic web of interdependencies had been created by developers keen to use this little abstraction rather than do it themselves, so when that one little package went away one afternoon, all the packages that used it stopped working, and all the packages that used those stopped working too, and on and on, creating a cascading failure that disrupted almost every major JavaScript package, and somewhere around a million dollars of productivity were likely lost debugging the issue! (By my own Fermi estimate. A developer’s salary and benefits run something like $60 an hour, and each might have spent about two and a half hours to track down and hopefully correct the issue, so that’s a cost of $150 per affected application, divided into a million dollars is 6,667 apps, which lowballs the number that broke.)

The fantastic satire website http://left-pad.io makes fun of our tendency towards overabstraction by creating an even more abstracted way to left-pad text: a complete, working web service. You can try it in your own browser on my --the example: https://api.left-pad.io/?str=the&len=5&ch=- (You put the parameters in the URL after the question mark, and when you browse there, the website sends back the padded string in a format called JSON.)

Skill and abstraction

left-pad.io says:

Less code is better code, leave the heavy lifting to left-pad.io, The String Experts™.

Sarcastic as it is here, this attitude is so common in real life as to almost prevent it from being funny. The assumption seems to be that people are too dumb, or too lazy, to handle life without abstractions over the simplest things. The truth is that the fewer abstractions you can work with and still accomplish a given task, the more skilled you are. When you use an abstraction, you don’t understand what’s going on underneath the hood. That’s exactly the point of abstractions (you can skip understanding the details), but it means that when the abstraction inevitably breaks its promise and leaks on you, you’re left high and dry. Figuring out what went wrong, even if you notice something did go wrong and you’re capable of sorting out the cause, is ugly and difficult. Abstractions make things appear simpler, but they create fragility and frustration. The fewer levels of abstraction you can use without compromising your ability to think about everything at once, the more competent you are and the more capable of responding to unexpected changes or problems.

Microsoft Windows’s default behavior of hiding file extensions is a prime example of the wrong attitude toward skill and abstraction. For some reason, when Windows 95 came out, Microsoft decided that users should no longer see any part of a filename that came after the final dot, despite this information having been regularly shown to users for several decades. So a file named myfile.doc, since 1995, by default appears in Windows’s file explorer as just myfile. Why exactly users would have difficulty understanding or manipulating file extensions was never made entirely clear. Of course, it comes back to bite us in the rear: this behavior is arguably a security risk, and when you have two files that have precisely the same name except for different extensions, or you download a file from the web called myfile.doc and then suddenly it’s not called myfile.doc anymore on your computer, confusion reigns. And heaven help you if you need to change an extension – not an unusual request, seeing as it’s part of the file’s name – when you can’t see it!

Further, the extension is a critical part of the way Windows handles files: it exclusively determines what program opens the file when you double-click on it. Rename a file from .doc to .html, and your browser will open the file instead of your word processor. Hiding the extension discourages users from learning this straightforward fact and thus makes them less capable of using their own computers.

Moving forward

One way to combat leaky abstractions is to use your own abstractions. If you create an abstraction yourself and you come back to it occasionally to fix any issues it has, you can hold its workings in your head and easily work out what’s happening and how to correct it when it leaks. When that’s not practical, using fewer and simpler abstractions can help, too. Simpler abstractions tend to leak less often, and the fewer you use, the fewer chances leaks have to show up.

Also, I have to admit, I own a smart scale and I even kind of like it. Even the seemingly unnecessarily complicated abstractions can be useful at times. Your takeaway should not be that all avoidable abstractions are to be avoided – rather, that every abstraction used in a system introduces additional fragility, risk, and potential for frustration, and you shouldn’t underestimate this factor when deciding whether to use one. Taking charge of details might look like extra work up front, but sometimes it ends up being less effort once you account for better understanding of the system and easier solutions to the problems that come up.