Gather Information

It is impossible to solve a problem without some kind of information about the problem.

OK, it’s not actually impossible; but it will take much, much longer, and you’ll learn next to nothing from solving it, and when you do solve it, it will be by dumb luck. As one of my programming instructors once said, “Let’s not depend on dumb luck.” Most of us presumably in fact don’t want to depend on dumb luck. Yet most of us also try to solve problems without first gathering information about the problem.

Debugging with information

I recently judged student presentations at an undergraduate statistics competition. Students received a prompt on Saturday morning and had roughly 24 hours to read and understand the prompt, identify a method for their study, obtain the data, analyze it with statistical tools, and prepare a presentation and a short paper with their results. I was perhaps more shocked than I should have been by how few of the groups spent any significant amount of time researching the subject (water quality) even though they had clearly come in knowing nothing about it.

Sure, they only had 24 hours, so it’d be easy to claim they didn’t have time to learn that information. But actually, they didn’t have time not to learn it – because statistics is not about computing numbers that just inherently mean something (the closest thing to that would be theoretical math, maybe something like logic or set theory or even abstract algebra). It’s about interpreting results and explaining what the numbers mean and why certain analytical choices the researchers made make sense (besides “I got a statistically significant result that way”). It’s also about what the numbers and the analysis suggest we should think or do. How the heck is anyone supposed to answer those kinds of questions without actually knowing what the data is about? The answer, of course: too many of the groups didn’t, or else they tried and came up with unconvincing or outright wrong answers. My ruthlessly critical statistics-PhD judging partner and I were not impressed by numbers without context or by some of the ridiculous recommendations we were pitched (and our scores and comments hopefully suggested this as an area for improvement).

In software, a classic debugging “technique” popular with new programmers is to randomly change things that look potentially fishy and see if they resolve the problem. As I admitted earlier, this method usually does work (eventually). But it usually doesn’t lead to a clean, fully correct solution, and it never teaches anything of value about how to debug. If later in their programming career those people sit down with a vague description of a bug reported by a user that probably sits somewhere in a 50,000-line program, how well is randomly changing things likely to work? Worse, randomly changing things could have broken something else, or masked the problem without actually solving it, and thus left the bug lurking for another unfortunate soul to find later (and masking the problem makes it even harder to find and fix next time). If someone can’t explain what the bug was, why it was there, how they fixed it, and why it will never come back, they haven’t truly “debugged” anything; at best, they managed to convince it to work (for now).

Instead, all good debugging techniques begin with information. Information is first obtained by reproducing the bug, thus learning under which narrow set of circumstances it happens. It might proceed with reading over the source code that runs when taking the action that results in the buggy behavior, or reading logs created during the reproduction of the bug, or using a debugging tool or print statements to investigate what variables are set to throughout the program as the bug is reproduced (since most program instructions involve setting variables, any bug will usually manifest in an unexpected variable value at some point or another).

That might not be enough to solve the problem, but it’s usually enough to come up with a working hypothesis which suggests more information to look for. After gathering enough information and testing more and more specific hypotheses (of course throwing away the ones that aren’t borne out by evidence and trying others), eventually the diligent problem-solver is all but guaranteed to hit upon some piece of information or realization that explains what the bug is, and from there working out why it was there and how to fix it is straightforward.

Everyday examples

Do you forget to stop and gather information in your life? I guarantee you do it sometimes; we’re all susceptible! Here are some common examples:

• When you lose an item like your keys or a book, do you end up wandering around the house searching frantically? You probably know a lot more about where that item went than your flustered state would show; next time try using your brain before your feet and you may find it in record time. (We’ll talk more about this one in a future post, because it’s a great example.)
• Do you pick up your phone or start browsing the web without having a reason? Almost without exception, this habit causes people to spend more time on screens than they would have intentionally chosen. Pause to identify why you’re starting to use a device, and you’ll recognize immediately whether it’s something you want to do or you’re doing it for no good reason at all and would rather reclaim your time and attention for something else.
• Ever rushed to blame someone or something else for a problem, only to later realize it was actually you? Start with gathering sufficient information to make a reasoned judgment, and you’ll make better decisions, waste less time complaining about something you could fix yourself, and look smarter and more respectable.

Stop and think

I once heard the following advice:

If you can’t explain why you are doing something, you should stop until you can figure it out.

Running ahead without the why is almost universally a bad move. Sure, often enough exploring beyond what you know is important; I’m not arguing every decision and every action can or should be based on complete understanding and certainty. But even when you’re pushing the boundaries, you’ll be more productive and learn more from your experience if you have an idea of why you want to go there and what you’re hoping to find. We live in an era where nearly any information that’s part of objective reality is available to us within fifteen seconds by tapping a few keys on the computer we’re sitting in front of or pulling our phone out of our pocket, and most of the information that’s not part of objective reality is easy enough to find if we take a moment to think and look at ourselves. Let’s not ignore that gift and stumble blindly in the dark, depending on dumb luck.