Interpretations and criticisms of the hawthorne studies
H. McIlvaine Parsons (1974) argues that in the studies where subjects had to go for long drives with no toilet breaks, the results should be considered biased by the feedback compared to the manipulation studies. He also argues that the rest periods involved, possible learning effects and the fear that the workers had about the intent of the studies may have biased the results.
Parsons defines the Hawthorne effect as "the confounding that occurs if experimenters fail to realize how the consequences of subjects' performance affect what subjects do" (i.e. learning effects, both permanent skill improvement and feedback-enabled adjustments to suit current goals). His key argument is that in the studies where workers dropped their finished goods down chutes, the "girls" had access to the counters of their work rate.
It's possible that the illumination experiments were explained by a longitudinal learning effect. It is notable however that Parsons refuses to analyze the illumination experiments, on the grounds that they haven't been properly published and so he can't get at details, whereas he had extensive personal communication with Roethlisberger and Dickson.
But Mayo says it is to do with the fact that the workers felt better in the situation, because of the sympathy and interest of the observers. He does say that this experiment is about testing overall effect, not testing factors separately. He also discusses it not really as an experimenter effect but as a management effect: how management can make workers perform differently because they feel differently. A lot to do with feeling free, not feeling supervised but more in control as a group. The experimental manipulations were important in convincing the workers to feel this way: that conditions were really different. The experiment was repeated with similar effects on mica splitting workers.
Richard E. Clark and Timothy F. Sugrue (1991, p. 333) in a review of educational research say that uncontrolled novelty effects cause on average 30% of a standard deviation (SD) rise (i.e., 50%-63% score rise), which decays to small level after 8 weeks. In more detail: 50% of a SD for up to 4 weeks; 30% of SD for 5-8 weeks; and 20% of SD for > 8 weeks, (which is < 1% of the variance).
A psychology professor at the University of Michigan, Dr. Richard Nisbett, calls the Hawthorne effect 'a glorified anecdote.' 'Once you have got the anecdote,' he said, 'you can throw away the data."
Harry Braverman argues in "Labor and Monopoly Capital" that the Hawthorne tests were based on behaviorist psychology and were supposed to confirm that workers' performance could be predicted by pre-hire testing. However, the Hawthorne study showed "that the performance of workers had little relation to ability and in fact often bore a reverse relation to test scores. What the studies really showed was that the workplace was not "a system of bureaucratic formal organization on the Weberian model, nor a system of informal group relations, as in the interpretation of Mayo and his followers but rather a system of power, of class antagonisms". This discovery was a blow to those hoping to apply the behavioral sciences to manipulate workers in the interest of management.
The Hawthorne effect has been well established in the empirical literature beyond the original studies. The output ("dependent") variables were human work and the educational effects can be expected to be similar (but it is not so obvious that medical effects would be). The experiments stand as a warning about simple experiments on human participants viewed as if they were only material systems. There is less certainty about the nature of the surprise factor, other than it certainly depended on the mental states of the participants: their knowledge, beliefs, etc.
Research on the demand effect also suggests that people might take on pleasing the experimenter as a goal, at least if it doesn't conflict with any other motive, but also, improving their performance by improving their skill will be dependent on getting feedback on their performance and an experiment may give them this for the first time. So you often won't see any Hawthorne effect—only when it turns out that with the attention came either usable feedback or a change in motivation.
Adair (1984) warns of gross factual inaccuracy in most secondary publications on Hawthorne effect and that many studies failed to find it. He argues that it should be viewed as a variant of Orne's (1973) experimental demand effect. So for Adair, the issue is that an experimental effect depends on the participants' interpretation of the situation; that this is why manipulation checks are important in social sciences experiments. So he thinks it is not awareness per se, nor special attention per se, but participants' interpretation must be investigated in order to discover if/how the experimental conditions interact with the participants' goals. This can affect whether participants believe something, if they act on it or do not see it as in their interest, etc.
Rosenthal and Jacobson (1992) ch.11 also reviews and discusses the Hawthorne effect.
In a currently unpublished working paper, economists John List and Steven Levitt claim that in the illumination experiments the variance in productivity is partly accounted for by other factors such as the weekly cycle of work or the seasonal temperature and so the original conclusions were overstated. If so, this confirms the analysis of SRG Jones's 1992 article examining the relay experiments.