« when is political participation good for the participants? | Main | the effects of Obama's organization »
May 8, 2008
the power of experimentation
Today's World Bank's meeting on community service programs turned out to be mainly a debate about the value of randomized, controlled experiments in evaluation. The Bank wants such evaluations (as does the US Government); many activists and proponents of youth service don't want to do them. I am also thinking right now about randomized experiments because I will soon play a role in running one in Florida schools.
Experiments don't work for all purposes; they are not always practical; and they're not the only legitimate methods.
I did, however, share a positive example of how experiments can be very helpful--in getting out the vote. There were no experiments with voter turnout in the United States for 50 years. That's surprising because voting is a very simple, measurable act that's well suited to experimentation. In the mid-1990s, some academics and foundations started pressing the nonpartisan voter turnout groups to use randomized evaluations. The pressure at first seemed unreasonable and even arrogant. It seemed as if the experts wanted randomization for the sake of it.
But now the nonpartisan voting groups are avid experimenters. They are always looking to randomize treatments and investigate the differences in results. When they send out mailings, they design two or more messages and randomize their lists. When they work in a limited number of sites or communities, they choose the sites randomly and reserve others as a control group. They do this even when they are not pressured by funders.
I think there are three major reasons for this "culture of experimentation":
1. Randomized experimentation is a simple, transparent process. It does not involve elaborate mathematics, which you do need for statistical models. Thus the grassroots groups control their own evaluations. They don't have to trust outside experts. Experimentation is actually non-technocratic.
2. Random experiments yield useful and counter-intuitive results. For example, going door-to-door is cost-effective even though it costs quite a lot of money per contact. Emails are not cost-effective even though they are cheap. This is good to know.
3. Experimental results can persuade powerful people who are predisposed to be skeptical. The American political parties traditionally assumed that it was a waste of money to mobilize young people. In the 1990s, political consultants often deliberately stripped young people from contact lists to save resources. But the experimental evidence showed that young people would vote if contacted. That led to much more partisan investment in youth turnout. The Obama campaign even has a youth director who comes straight out of the youth voting community in which the experiments were conducted from 1998-2006. Regardless of the experimental data, Obama probably would have campaigned to youth, because he has an appeal with the new generation. Still, it doesn't seem a complete coincidence that (a) we learn how to mobilize young voters by experimenting, and (b) a candidate captures the Democratic nomination by mobilizing youth.
May 8, 2008 7:47 PM | category: none