by Laurence Chen

Have you ever had one of the following experiences before?

1. Your job is in customer service, and your company’s client reported that they encountered certain troubles in using your products. You forwarded the client’s complaint to the service team, but the service team told you, “Our product has no bugs.”
2. You needed to use a complex gadget, and it requires 15 steps. You followed the manual, but it did not work well. What do you do next?
3. You needed to use a shell command, even one which you’ve used successfully before. However, when you use it, it doesn’t work. The console gives you some error messages which you do not understand. You searched StackOverflow, but no answer could really get you to progress.

The above difficulties are typical difficulties that users using a new system will encounter, no matter whether that system is a software product, a complex gadget, or a shell command. I have implemented some systems and have been the users of many systems. Having strong feelings about this kind of situation, I believe that people who want to help others to use a system smoothly or people who want to help themselves need to understand “how to debug.”

The scientific method

The core idea of debugging is quite similar to the scientific method.

When scientists want to explain a certain natural phenomenon, they first propose a hypothesis, do experiments which help to verify the hopothesis, observe the results, and then falsify the hypothesis or modify the hypothesis by the experimental results. After several rounds of repetitions, they provide an explanation which has been verified: we call it a theory. Debugging is quite similar to the above activity. If we want to catch a bug inside a system, we also prepare a hypothesis first. The hypothesis is our guess to the bug.

Here in Taiwan science is one of the things that our education system emphasizes. However, in reality, when I have helped or taught others to debug, I realized that merely eplaining what the scientific method is was not enough to get the learners to acquire this skill. The results were usually quite bad.

The scientific method of debugging consists of 4 steps:

1. hypothesis
2. experiment
3. observe
4. modify hypothesis by observation

My observation is that there typically are two obstacles to applying the scientific method:

1. Not doing experiments well
2. Not knowing how to make a good hypothesis

Change one thing at a time

Sometimes, even myself, I change three different places inside the system and then hope to see the system magically work. However, most of the time, this kind of effort does not get me to my destination faster.

There is an important property in the experimental part of the scientific method: controlled variables. Every experiment should be used to explain a hypothesis of why the system does not work. Therefore, it is fine that you change three places together if they are dependent and you need to change them together to verify one hypothesis. However, it is not effective to change three places when each place corresponds to a distinct hypothesis.

A good hypothesis seldom appears when you don’t understand how the system works

“The errors inside a system usually have a relationship with the physical outer world.” This is something I keep reminding myself when I examine my hypothesis.

When I was in college, I once conducted a circuit experiment. After I followed the circuit map and created a circuit on breadboard by placing all the active and passive components, the output voltage was not correct. My hypothesis was that probably one of the resistors is broken, and given that examining each one is a little bit too much effort, I decided to reimplement the whole circuit again. Before I began, the teaching assistant showed me how to debug. He picked several points inside the circuit, and he measured the voltages. Then, he pointed out where the bug was.

In the above example, the younger me made a hypothesis by assuming that anything can break randomly. On the other hand, the teaching assistant, he made a hypothesis based on the mental model of the system and he focused on thinking about the interaction of the circuit components inside the system. Also, he used a multimeter to measure the voltage (the experiment and observe parts).

Debugging in our daily life

I was once asked by my wife, “Do you think that debugging is an exceptional thing? Well, it depends.” I then replied, “Last time you drove your car to get it fixed by a car mechanic, that mechanic listened to the sound of the car and then made his diagnosis. Does this story sound similar to debugging?”