RSS RSS

Pygmalion revisited

By Hugo Schouppe, 2009-09-22 23:27

Rosenthal and Jacobson (1968) did the following famous but much criticized  experiment. You can download the original text at the Indiana University.

A non-verbal intelligence test, disguised as a test designed to predict academic “blooming” or “spurting”, was administered to all children of an elementary school (May 1964). After that, in each of the 18 classes, an average of 20% of the children (on average 5 pupils per class) were randomly selected. Rosenthal and Jacobson told the new teachers in September that these pupils have a potential for intellectual gains and will show a sudden and dramatic intellectual spurt over the upcoming school year. Eight months later (January 1965), all children were retested and again at the end of the school year (May 1965). It is this last test that served as the basic post-test.  For the school as a whole those children whom the teachers had been led to expect “blooming” showed a significantly greater gain in IQ score (12 IQ points) than did the control children (8 IQ points). This gain was attributed to the self-fulfilling prophecy effect. The teachers expect those children to “bloom” and changed their behaviour accordingly so that the prophecy was fulfilled. In May 1966, 2 years after the pretest, the children were given the test for the fourth and final time.

The experiment has been widely criticized. Let’s take first a closer look at the results.

As the authors themselves recognised, the effect was unequal for the different grades. The lower the grade level, the greater was the effect.  The authors tried to explain these differences because younger children (a) have less well-established reputations (b) may be more susceptible (c) can differ from older children in characteristics other than age (d) have other teachers. A possible contamination was also the fact that the retesting was done by the teachers themselves. May be, the kids weren’t smarter but they received advantage in the testing procedure (e.g. receive more time to answer). However, according to the authors, three classes were also retested by the school administrator, which means that for these three classes the children have been tested one time more than the other classes. Those results did not differ from the results of the retesting of the teachers,

Results of the Rosenthal & Jacobson experiment

L. Jussim and KD. Harber (2005) give a state of the art in “Teacher Expectations and Self-Fulfilling Prophecies: Knows and Unknowns, Resolved and Unresolved Controversies”.  You can download the full text on the website of the second author; see also Samuel S. Wineburg The Self-Fulfillment of the Self-Fulfilling Prophecy (1987). They come to the conclusion that self-fulfilling prophecies in the classroom do occur, but that these effects are typically small and whether self-fulfilling prophecies affect intelligence remains unclear.

The effect is small. In the Rosenthal study, the difference between the experimental and control group is 4 IQ points. The control group itself, however, had gained 8 IQ points over a time span of 1 year, and this without any intervention.  The effect size  -this is the difference between the experimental and the control group in standard deviation units- is .30; which is typically considered as small. The correlation between the manipulation (bloomer or not) and the IQ is .15; which is also a small relationship.

There are also some strange things in the original dataset of Jacobson & Rosenthal. Because the authors reported only change scores (difference between the pretest and the basic post test 1 year later), we have no clues about the absolute scores of the pupils. Taking a closer look at the original scores reveal that there was for example one child with a pretest on the Reasoning IQ of 17 and a post-test of 148, 110 and 112. Before the experiment, this child was severely retarded; after a few months it became highly gifted, near genius. This is, of course, very hard to believe.

In June 1966, 16 of the 18 original teachers were interviewed. Of the 72 children in the experimental group, they could only recall 18 correctly, next to the 18 control subjects which they recalled incorrectly. The teachers didn’t remember the names of the spurters. Their expectancy of he children has raised the IQ of the children with 4 points, but after 2 years these teachers didn’t know even their names.

Perhaps, these results have more to do with the fact that the teachers themselves have administered (not corrected) the tests. Because they are familiar with the tests, they can prepare the pupils for them, much like you have prepared yourself for your driver license exam. In an experiment in which the expectations were manipulated, together with the familiarity of the teachers with the test, only in the case in which the teachers are familiar with the test, an expectancy effect was discovered.

There are literally hundreds of studies on the self-fulfilling prophecy. Rosenthal himself carried out a meta-analysis over more than 300 studies. The problem with these meta-studies is that most of them don’t test the relationship between intelligence and expectancy but between expectancy and something else. About 1/3 of the studies showed a significant expectancy effect. One factor that obscures the results is the timeframe of the expectancy induction. It seems plausible that the effect is the greatest within the first week of the new school year, when the teachers hadn’t formed already an opinion about the children.

Leave a Reply

Persephone Theme by Themocracy