bias in science--6/18/2026

18 Jun 2026

bias in science--6/18/2026

0 COMMENTS

Today's encore selection -- from Science Delusion by Rupert Sheldrake. The physical sciences rarely employ methodologies to ensure objectivity in their research:

“Peter Medawar was an articulate British biologist who won the Nobel Prize for Medicine. In a witty talk on BBC radio in 1963, he asked, 'Is the scientific paper a fraud?' and answered, ‘Yes.’ He was not referring to fraudulent data, but to the way that scientific papers are conventionally written. In scientific journals, the standard format for articles, then as now, is to start with a neutral-sounding Introduction, setting out the problem and referring to earlier research, followed by a Methods section, then the Results and finally a Discussion. As Medawar described it:

‘The section called 'results' consists of a stream of factual information in which it is. considered extremely bad form to discuss the significance of results you are getting. You have to pretend firmly that your mind is, so to speak, a virgin receptacle, an empty vessel, for information which flows into it from the external world for no reason which you yourself have revealed. You reserve all appraisal of the scientific evidence until the 'discussion' section, and in the discussion you adopt the ludicrous pretence of asking yourself if the information you have collected actually means anything’.

“Medawar pointed out that this procedure, still standard today, gives an entirely false impression of the way science works, suggesting that scientists collect facts and then draw general conclusions from them. In fact, scientists start with an expectation or a hypothesis that provides the incentive for the enquiry in the first place. It is only in the light of these expectations that some observations are thought to be relevant and others not; that some methods are chosen and others discarded; that some experiments are done rather than others. Medawar suggested a more honest approach: putting the discussion at the beginning:

‘The scientific facts and scientific acts should follow the discussion, and scientists should not be ashamed to admit, as many of them apparently are ashamed to admit, that hypotheses appear in their minds along uncharted by-ways of thought; that they are imaginative and inspirational in character; that they are indeed adventures of the mind.’

“Most medical researchers are well aware that their beliefs and expectations can influence the results of their experiments, which is why many clinical trials are carried out double-blind: neither the researchers nor the patients know who has received which treatment (see Chapter 10).

“Experimenter effects are also well known in experimental psychology. This principle was illustrated in a classic experiment in which the experimenters trained a group of psychology graduate students to administer the Rorschach test, in which subjects were asked to identify patterns in inkblots. The experimenters told half of the students that experienced psychologists obtained more human than animal images from their subjects. They told the other half of the group the opposite. Sure enough, when they administered the test the second group found more animal images than the first.

“Even in experiments with animals, experimenters' expectations can influence the results. In a classic experiment at Harvard, Robert Rosenthal and his colleagues instructed students to test rats in standard mazes. They asked them to compare two strains of rat produced by generations of selective breeding for good and poor performance in mazes. But they deliberately deceived their students. In fact, the rats came from a standard laboratory strain and were divided at random into two groups labelled 'maze-bright' and 'maze-dull'. ·

“Trusting what they had been told, the students expected the bright rats to do better than the dull ones and, sure enough, they found that the 'bright' rats learned much faster than the 'dull' ones. Since the rats were more or less identical, these dramatic differences must have resulted from the students' expectations.

“Although experimenter-expectancy effects are widely recognised in psychology and medicine, in the 'hard' sciences most scientists assume that they are irrelevant. They take it for granted that their own expectations have no influence on their experiments and on the recording of data.

From 1996 to 1998, I carried out a survey of more than 1,500 papers in leading scientific journals to find out how often the researchers used blind methods. Caroline Watt and Marleen Nagtegaal later replicated this survey, using a different selection of journals.

“Watt and Nagtegaal found a higher percentage of papers with blind methodologies in more areas than I did, and a slightly lower percentage in parapsychology, but in both our surveys, in the physical sciences almost no research involved blind methodologies, and in the biological sciences very little, less than 2.5 per cent. Even in experimental psychology, animal behaviour and the medical sciences, where the effects of experimenters' expectations are widely recognised, a minority of studies used blind methods. By far the highest percentage was in parapsychology.

“I also organised a telephone survey of senior researchers in fifty-five departments in eleven British universities, including Oxford, Cambridge, Edinburgh and Imperial College, London. My research assistant, Jane Jurney, carried out the interviews by telephone. She asked the professors or other senior scientists if anyone in their department used blind methodologies, and also whether they taught students about such methods.

“Some of the scientists did not know what was meant by the phrase 'blind methodology.’ Most were aware of blind techniques, but said that they were necessary only in clinical research or psychology. They thought they were used to avoid biases introduced by human subjects. The commonest view among physical and biological scientists was that blind methodologies were unnecessary because 'Nature itself is blind,’ as one researcher put it. A professor of chemistry added, 'Science is difficult enough as it is without making it even harder by not knowing what you are working on.’

“Out of twenty-three physics and chemistry departments, only one used blind methods and taught the students about such methods. Out of forty-two departments in the biological sciences, twelve (29 per cent) sometimes used blind methods and taught about them. But only in exceptional cases were blind techniques used routinely. My survey revealed three examples, all of which involved industrial contracts that required the university scientists to evaluate coded samples without knowing their identity.

“The discussion so far concerns collecting data under blind conditions. Even scientists who use such procedures do not usually analyse their results blind, and can easily discard results that do not agree with their expectations. Blind analysis is a further level of blinding that is almost unknown in most areas of science, but some researchers in sub-fields of particle physics and cosmology are leading the way in promoting the blind analysis of results. They point out that bias and selective use of data are all too common, and advocate making all analytical decisions and all debugging procedures before the relevant results are revealed to the experimenters. They recognise, however, that it may be very hard to persuade scientists to forgo their chances of coming up with desirable, though possibly false, results.”