Misconceptions about Scientific Experiments

I am afraid that the reader has quietly read the last paragraphs about experiments but still hasn't begun to rethink his ideas of what a scientific experiment is. I taught physics laboratory - pretty intensive with four hour labs once a week - and after fifteen labs the students were just beginning to let go of their misconceptions about the scientific method. So now I want to address three popular misconceptions straight on.

The first misconception is just the popular meaning of the word "experiment." "I experimented with adding nutmeg to the pancakes." "He experimented with drugs." In everyday usage "experiment" means doing something new in hope of obtaining some improvement. Often one doesn't know ahead of time what improvement may be obtained, except perhaps in very general intuitive terms, and in fact one often hopes for a surprise. Along the same lines, we could say that early humans experimented with eating various substances, domesticating animals, and sowing seeds; this was long before natural philosophers and has little to do them. Experimentation in this sense belongs instead to the normal range of activities centered on "getting things done" which I contrasted earlier (link) with philosophy. It is oriented toward results, and at a deeper level it is oriented toward human beings and their interests.

In contrast, scientific experimentation is oriented away from ourselves, toward nature, toward observing nature as it is. Compared to normal experimentation, scientific experimentation involves a lot more planning attention to observing what happens during the experiment or what is produced by it. Experimental scientists spend their entire careers inventing and refining new tools and techniques for observing things - seeing them more clearly, more accurately, with more detail, and extending the range of observation. The observational procedures and tools are checked and rechecked in meticulous detail, calibrated, refined, rebuilt, all to ascertain that the new vision the produce is real and not just fancy or mirages. While readying a new experimental tool one plans exactly how to use it, what to measure, so that afterwards one can check and rethink whether your observational procedure really did what you thought it was doing. Also one writes out in detail one's expectations about the results of the observation, in order to be able to contrast one's expectations with the actual observation. It is well known that humans are much better at noticing things that change than things that stay the same, so writing out our expectations ahead of time improves our observational abilities.

A second misconception about scientific experiments comes from our "experiments" in school laboratories. One is given some materials (wire, chemicals, roller carts, etc.), some measurement devices (electrical meters, microscopes, rulers, etc.,) and instructions about about what to do with them. The instructions typically require reading and recording one or more numbers, computing something based on those numbers, and then reporting the results of your computation. The emphasis is on following a procedure and producing some desired numbers.

In one way these school laboratories do teach something about scientific experimentation: depending on the lab format, the students may be encouraged to encounter and manipulate objects, perhaps even with some care and attention, and will typically be asked to use at least one number to describe the "outcome" of the experiment. However these labs can also give entirely the wrong impression. Probably their single biggest defect is that they focus on procedure more than observation. I would like to see labs where the students are taught to observe and think clearly about everything before them, to really see the wires and chemicals, sense them, measure everything about them and then measure everything about the measurement devices. A scientist doing an experiment is really interested in the thing she is measuring, and wants to know everything about it, and finally to understand it. She won't be content with a single number, or a hundred numbers. Preliminary studies of a newly encountered material or phenomenon usually involve many different measuring instruments, each producing large quantities of information. A large part of a scientist's life is sorting through the information, checking it and the way it was produced, processing it, piecing it together to get a full vision of the thing being observed. The end results may be in a synthetic summary form, or may highlight the most striking information, but this conciseness belies the care that has been taken to observe as much as possible, as comprehensively as possible. A good scientist really wants to get know what she is measuring, through and through.

Of course this is not an innate skill: if you tell a typical student that you want him to understand everything about a particular experiment including the measurement apparatus, he will look at it for some minutes, maybe measure a couple of things, write down a few sentences and numbers, and be done. The observational skills, and curiosity to think of things to measure, are not built in. Neither is the ability to check one's own measuring devices and procedures and invent and refine new measurement procedures as necessary. So I haven't figured out yet exactly how to make a school laboratory that really duplicates the experience of a scientific experiment. Probably the best one can do is to help the student to grow in the right direction, beyond procedures and a few numbers toward real observation and real care for what is being observed.

The third misconception about scientific experiments is exemplified in movies, where the scientist spends some time and effort, builds a new device or something, and tada! she discovers a new thing. It's presented almost as a guarantee: tinker around some, build something strange and new, and then - guaranteed! - you'll make a Discovery! I sometimes get the same feeling when we talk about national spending on research - spend x billion dollars and you'll get a proportionate return in discoveries. It's as if you could build science factories: build twice as many factories and you'll get twice as many discoveries. This sort of reasoning is true for engineering and industry where you know how things work already. It's not true about discovery: if it is really a discovery then that means that you did not know ahead of time what you were going to discover, or whether you would discover anything at all. And really the emphasis on Discovery! is inappropriate because of its emphasis on achievement and being right: really a scientist's business is to somehow discover where she is wrong - and the atmosphere is much more one of desperate hope for enlightenment. The more triumphalist one gets, the less one is humble about the mysterious depths one is just beginning to touch, the less one is able to do real science.