Peter Levine

Service-learning: Why We Do It, and How to Show It Works
Peter Levine, Nov. 1, 2006

I have been asked to speak about the measurement and assessment of service-learning. Our work at CIRCLE [The Center for Information & Research on Civic Learning & Engagement] involves civic engagement. So I will first discuss civic engagement, then argue for measurement and assessment, and finally try to say what is so important about service-learning.

CIRCLE recently released a major survey of young people that tracked 19 different forms of civic engagement--voting, volunteering, attending meetings, contacting the media, persuading other people about elections, boycotting, and more.

I think it’s a pretty good list; but a list is not a definition. So what is civic engagement?

Some people define it in terms of sectors. It’s civic engagement if you work without pay (which makes you a “volunteer”), or if you influence the government (then you’re a “voter” or an “advocate”), or if your paycheck comes from the government or a nonprofit organization (which makes you a “public servant”). In other words, you’re civically engaged when you’re outside of the market sector of society.

I don’t think that definition can work. Newspapers fill a civic role; people who work for them are civically engaged. But newspapers are usually profit-making corporations that pay their reporters and editors. Harry Boyte notes that grocery store owners who display fruits and vegetables outside their businesses at night contribute civically by making city streets safer and more attractive.[i] When people boycott and “buycott,” we say they are civically engaged even though they are consumers who attempt to influence companies.

I would drop talk of sectors. As a rough alternative, I’d say that “civic engagement” is any ethical way of addressing a public or common problem.

But what’s a public problem? Is pornography a public problem? Maybe one person’s private behavior isn’t other people’s business. What about poverty? Some don’t think that’s their business or problem, either. Cancer? That does seem to be a public problem, but if it is, then scientific researchers must be civically engaged when they’re working in their labs. (And maybe they are.)

The definition is essentially controversial, but I don’t think there’s any substitute for defining legitimate public problems and then saying that civic engagement addresses those problems. Participating in the debate about what is a public problem is itself a form of civic engagement.

Why do we need civic engagement? Why can’t we leave governing to the government, and expect public institutions (such as schools) to provide public services? Why must many citizens participate?

First, institutions work better when participation is widespread.

For example, Robert Putnam has shown that schools work better in “states where citizens meet, join, vote, and trust [one another].” Putnam finds that such engagement is “by far” a bigger correlate of educational outcomes than is spending on education, teachers’ salaries, class size, or demographics.[ii]

Second, social outcomes are more likely to be just when participation is equitable.

We know that people who are better off participate more. Americans with family incomes under $15,000 voted at half the rate of those with family incomes over $75,000.

And they get results proportionate to their participation. Larry Bartels has found that wealthy constituents have three times more influence than poor ones on U.S. Senators. In fact, Bartels could find no impact—zero impact—of people in the bottom third of the income scale on their own “senators’ roll call votes.”[iii]

Third, some crucial public problems can only be addressed by people’s direct “public work”--not by legislation.[iv]

Effective governments are capable of redistributing money and defining and punishing crimes. But rarely can governments reduce prejudice, change public attitudes toward nature, or deliver personalized care. Even when the state funds healthcare and higher education, the actual work is usually conducted by associations that can be more diverse, participatory, and sensitive than the state.

Finally, broad civic engagement is necessary to support a healthy, democratic culture.

Today, various groups of Americans criticize mass media and mass culture for being secular, materialistic, superficial, violent, sexist, and racist, and for undermining local, traditional, and minority cultures. These critiques are not always mutually consistent and may not all be valid. But it seems clear that people feel powerless to change mass culture, and that feeling demonstrates the tension between mass culture and democracy.

A democratic group or community must be able to illustrate and memorialize its own values and present its own identity to outsiders and future generations of its own people. Many communities choose to display their identities through music, statuary, graphic design, narrative history, and other forms of culture. Their culture is idiosyncratic and local, because engaged, active people clump together in communities and associations, and each one takes on a distinct character through their work. Thus diverse culture is evidence of civic engagement.

On the other hand, a homogeneous, mass culture arises when people are not heavily engaged. Mass culture is a threat to democracy, because when only a few people produce products that reach a mass market, they obtain great influence.

If we want to improve public skills, attitudes, and habits relevant to liberal democracy, we must focus on youth.

It is very hard to think of programs, projects, or even movements that have changed passive adults into active citizens.

But many specific interventions aimed at youth have been found to work. The longest study that I know of has followed the high school class of 1965 until today. It was conducted by Kent Jennings and colleagues, and they found that participation in student government and other civic extracurricular activities in the 1960s still boosts people’s participation in civil society almost forty years later.

More than a dozen other longitudinal studies of adolescent participation in community service have found positive effects as much as ten years later. And Doug McAdam’s rigorous study of the Freedom Summer voting-rights campaign shows that the activists’ experience in Mississippi (admittedly, an intense one) permanently transformed them.

Also, whole generations have enduring civic characters, and that is probably because certain experiences were shared by many contemporaries in their adolescence. The he New Deal and World War II provided a form of “civic education” that caused members of the “greatest generation” to be engaged throughout their lives.

Most small children are insulated from the big world of politics and current events. They don’t have to have opinions of it. But teenagers are confronted with politics, social issues, and civil society and must develop some kind of stance. They may be uninterested, which is the default, or they may choose critical engagement, enthusiastic support, or some other response. Once they have formed a basic orientation, it would take effort and perhaps some psychological distress to change their minds. Therefore, most young adults settle into a pattern of behavior and attitudes in relation to politics that lasts for the rest of their lives, unless some major shock (such as a war or revolution) forces them to reconsider. When adults change their political identities, the change usually results from voluntary experiences, not from exhortations or any form of mandatory civic education.

It would be immoral to write off adults because they are much less malleable or “plastic” than adolescents and less susceptible to deliberate civic education. We should look for models, such as public meetings and innovations in the news media, than can enhance the civic engagement of people who have passed the age of 25 or 30. But it is especially important to invest in the democratic education of young people, since they will be permanently shaped by the way they first experience politics, social issues, and civil society

Young people also do better in life if they engage civically. Volunteering and belonging to groups improves their academic performance, it lowers their pregnancy rate, it reduced their tobacco use, it keeps them engaged in school.

The most ambitious explanation is the theory of Positive Youth Development, which says that kids flourish when they can use and develop their assets for valuable purposes. If we treat adolescents as a bundle of problems, we alienate some of them. But if we recognize that they have passion, energy, creativity, and their own social networks to contribute, we can help them to succeed.

Many students drop out of high school because the assigned work is boring and because they lack personal connections to teachers. In a 2006 study of recent dropouts, more than half said they had satisfactory grades before they left school (“C” or better), but half said that classes were boring. There have been rigorous evaluations of programs that help students to work on community problems in collaboration with adults. For instance, an evaluation of the Quantum Opportunities Program studied randomly selected students and a control group. For about $2,500/year over four years, Quantum was able to cut the dropout rate to 23 percent, compared to 50 percent for the control group. QOP’s approach included mandatory community service

What’s promising about service-learning as a form of civic education?

As you well know, “service-learning” means an opportunity for students actually to serve in their communities while they study, discuss, or reflect upon their service. It thus implies a deliberate combination of academic study and practical work. Service-learning has a long heritage in the United States and in many other countries and cultures. To consider just one example, as early as the twelfth century in Europe, mendicant friars (monk) of the Franciscan and Dominican orders were expected to learn from serving the poor.

In the mid-twentieth century there were some excellent programs that we would now call “service learning.” For instance, in the 1940s the City Planning Commission of a small city in the northeast of the United States asked social studies teachers to recruit high school seniors who designed and conducted community surveys, produced maps, and wrote recommendations.[v] Other settings for service learning (before that jargon was coined) included settlement houses like Hull-House in Chicago, the Appalachian Folk Schools (of which Highlander in Tennessee was most famous), and the Civilian Conservation Corps, which provided formal civic education programs connected to service work.[vi]

The phrase itself seems to have been coined in 1967.[vii] Service-learning then developed into a movement, complete with dedicated journals, standards for “best practice,” several annual conferences, public and private funding sources, and networks of practitioners and advocates. In 1999, about half of high schools claimed to offer service-learning opportunities.[viii]

In practice, both the service and the learning in “service-learning” differ widely. “Service” may mean tutoring, visiting elderly people, raising money for charity, cleaning up public spaces, taking soil or water samples for environmental monitoring, creating websites or broadcast segments, or organizing communities for political action. “Learning” may mean discussing a service experience in class, writing journal entries about the underlying issues, or even conducted elaborate research studies.

There is no doubt that the best service-learning works. It not only enhances students’ skills and interests; it changes their fundamental identities so that they become—and see themselves as—active citizens.[ix] However, there is a range of quality in service-learning. I’ll return to that issue in a few minutes.

What about measurement and outcomes?

In my experience, a pretty high proportion of service-learning folks are at least somewhat skeptical of quantitative research and evaluation. I think I understand their concerns, and I certainly do not dismiss them.

· Quantitative evaluation can miss subtle but important changes in youth that don’t show up in questionnaires.

· Quantitative measures are usually generic--they would apply anywhere. For example, we test students on their understanding of the US Constitution, or we ask them about their interest in voting. These are generic questions. But a good service-learning project might have idiosyncratic results appropriate only for the local community in which it occurs. For example, students who clean up a river might learn about that river, not about the US Constitution. To learn about their own river is an achievement, but not one that would show up on generic evaluations.

· Quantitative evaluation can be--or at least seem to be--highly technical, and therefore the business of experts. But service-learning is about allowing kids and other “ordinary people” to make their own decisions

· Quantitative evaluation makes everything sound worthwhile only if it achieves outcomes for individual kids. We’re used to saying, for example, that if Head Start does not raise kids’ test scores when they reach high school, it’s a waste of taxpayers’ money. But regardless of what skills schools provide, they are also places where we spend some 18,000 hours of our lives. Some activities during those hours ought to be intrinsically satisfying or else meaningful because they benefit other people (or nature), not because they enhance students' individual skills. Schools are communities; and communities ought to include service—regardless of the impact on those who serve.

A one-time service activity is very unlikely to make a lasting difference to the kids who serve. Does that make it pointless? Or might it be intrinsically or morally worthwhile?

· Finally, many of us think that we should be accountable to ourselves and to those whom we know personally for doing our best work. A good student feels that kind of accountability; she does her best work for her own sake or to satisfy her teacher or classmates. She doesn’t work hard to get a good grade. Quantitative evaluation makes us accountable for achieving targets that can seem external or artificial--kind of like doing our schoolwork just to get a high grade.

Those are valid points. Nevertheless, I am going to argue for using outcome measures and quantitative evaluation.

In fact, I think we need to go all the way to experiments, whenever possible. In an experiment, you randomly assign some kids--or some classrooms or schools--to receive a service-learning activity, while others do not, and then you compare the outcomes. That is challenging to organize, but if we were all looking for opportunities to conduct experiments, those opportunities would arise. And by the way, you do not have to deny opportunities to some kids in order to create a control group for an experiment. Usually, you are not able to serve everyone anyway--at least not all at the same time. So, instead of accepting people on the basis of merit, or first come/first served, you can randomly draw from the applicant pool and thereby create an experiment.

Here is my first argument for using experiments and other quantitative methods of assessing outcomes.

Service-learning is marginal in our schools. It’s not uncommon any more, but it is peripheral. Consider the way that funding for Learn & Serve America, in real dollars, has shrunk over the past decade.

This is because policymakers are not basically concerned about civic education, or moral education, or social and emotional learning. Even the most idealistic policymakers are mainly concerned that some of our kids cannot read or manage other basic academic skills.

If you can’t read, you’re on course to drop out and then to face poverty, ill health, and violence—especially in the increasingly competitive economy of the 21st century. So our educational leaders want to identify kids at risk of failing in basic academic subjects and help them. That is where all the energy is, and the money, and the instructional time.

Service-learning programs have sometimes been found to help keep kids in school and succeed academically. For instance, the Teen Outreach Program (or TOP) significantly reduced teen pregnancy, school suspension, and school failure. TOP was successful even though it focused “very little attention” on those problems. In other words, the staff did not directly address pregnancy or school-related problems. Instead, youth in the program were enrolled in service projects and asked to discuss their work in classroom settings. An average of 46 hours of service reduced teen pregnancy through the indirect means of giving young women valuable civic work to do.[x]

The evaluation of TOP was strategically powerful, because it might persuade policymakers to invest serious resources (money and in-school time) in service-learning. They would use service-learning to get what they say they want—better outcomes for kids.

But would other service-learning programs work as well as TOP? We need many more experiments to find out and to make the case to policymakers, even the most sincere and idealistic of whom are pretty skeptical.

In short if we are interested in expanding and enhancing Learn and Serve America as a program, we should be experimenting as much as possible, using outcomes that powerful people care about—not civic skills, but pregnancy rates, incarceration rates, and dropout rates.

Now a second argument for measurement and formal experimentation.

Even for our own purposes of increasing civic engagement, we need measurement to tell us what policies would help.

We can be sure that certain small-scale programs work. Our own eyes tell us that they are great when we observe these programs. But a policy is more than a small-scale program. By providing funds, or training, or incentives, or mandates--all different forms of policy--government could dramatically increase the quantity and quality of service learning. But would these policies work? We need hard data to know.

In their 1999 evaluation of Learn & Serve America, Alan Melchior, Larry Bailis, and colleagues found that funded programs had positive effects on students’ civic attitudes, habits of volunteering, and success in school. However, their study was limited to “fully implemented” service-learning projects: ones that involved “substantial hours” of high quality service, “face-to-face experience with service recipients,” and opportunities for reflection. Out of 210 programs funded by Learn & Serve America that the evaluators had randomly selected for their study, only seventeen met the criteria for being “fully implemented,” even though the rest would certainly call themselves “service-learning” and had won grants in a competitive process.[xi] If all 210 programs had been included, it is not clear that the average effects of service-learning would have been positive.

Alan and Larry collected their data almost a decade ago. The field has progressed since then. In a smaller study published in 2005, Shelley Billig and her colleagues found that average service-learning classes had slightly better civic outcomes than average social studies classes. Students who had been exposed to service-learning gained more knowledge of civics and government and felt more confident about their own civic skills, compared to a matched group of students who had taken conventional social studies classes. However, service-learning did not raise students’ sense of their own community attachment or their own ability to make a difference. (Possibly, the difficulty of the projects they undertook turned them into pessimists about achieving social change). In any case, these average results concealed very large differences between the best and worst service-learning. Some classes in Billig’s small study that claimed to use service-learning produced notably poor results.[xii]

If a school superintendent asked me what the research shows about service-learning, I would say that it supports creating a small competitive grant program and providing voluntary opportunities for teachers, such as seminars on how to organize a community-service project. The research does not, at this time, support allocating a lot of district money for service-learning or setting a high target for the rate of student participation.

In this respect, service-learning is different from social studies teaching. Standard social studies classes are much more common than service-learning programs and are probably distributed in a normal curve, such that classes of average quality are most common. We can tell from exam results that the average-quality classes have positive effects. Thus I would advise a superintendent or a state official to mandate social studies classes for all students (while also trying to support or weed out the worst teachers and reward the best ones). I would regard service-learning differently: as something to be cherished and admired when it is done well, but not to be rapidly expanded.

It’s not especially good news for Learn & Serve America if the existing research does not support the case for widespread adoption. But that’s partly because we don’t have much research that’s rigorous enough to persuade skeptics. Maybe more studies would reveal that some particular categories of service-learning are so good that they should be massively expanded, generously funded, or even mandated by law.

I have offered some arguments in favor of measuring and assessing service-learning, as a strategy for increasing its quality and quantity. Let me end by emphasizing what’s really at stake. We are not doing this because we want more dollars for Learn & Serve or bigger numbers of kids in service-learning programs.

We are doing it because service-learning represents an alternative to politics and education as we know them.

· In general, we treat young people as baskets of problems or potential problems and rely upon surveillance, assessment, diagnosis, discipline, and treatment to stop them from acting in damaging ways. But service-learning embodies the alternative approach of "positive youth development," which recognizes that young people have special assets to contribute to their communities—to repeat: creativity, energy, idealism, and a fresh outlook. If they are given opportunities to contribute, they develop in healthy ways. Major recent policies (such as the No Child Left Behind Act) have very little to say about providing positive opportunities for youth. Service learning is a powerful positive opportunity

· In general, our politics is state-centered. Liberals want the government to accept new tasks, such as health insurance; whereas conservatives believe that problems would be mitigated if the state were shrunk.

Governments are important, but they are not the only institutions that matter. Furthermore, a state-centered view of politics leaves citizens little to do but inform themselves and vote. Service-learning epitomizes a citizen-centered politics in which people form relationships with peers, deliberate about their common interests, and then use a range of strategies, some having little to do with the state.

· In general, our politics is manipulative. Experts--politicians, pundits, consultants, marketers, leaders of advocacy groups, and the like--study us, poll us, focus-group us, and assign us to gerrymandered electoral districts; they slice-and-dice us; and then they send us tailored messages designed to encourage us--or to scare us--into acting just how they want.

This is true of liberal politicians as well as conservative ones. It is true of public interest lobbies as well as business lobbies. It is true of big nonprofits as well as political parties.

People know they are being manipulated, and they resent it. They want to be able to decide for themselves what is important, what should be done, and then act in common to address their problems. They want an open-ended, citizen-centered politics in which the outcomes are not predetermined by experts.

And service-learning, at its best, is open-ended politics. We don’t try to manipulate our kids into adopting opinions or solutions that we think are right—-at least, we shouldn’t. We give them opportunities to deliberate and reflect and then act in ways that seem best to them. In a time of increasingly sophisticated manipulative politics, these opportunities are precious.

Finally, a point about civic education in an imperfect political system. Maybe it isn’t reasonable to expect our young people to hold positive civic attitudes and be actively engaged. Citizens (both young and old alike) may rightly shun voting when most elections have already been determined by the way district lines were drawn. They may rightly ignore the news when the quality of journalism, especially on television, is poor. And they may rightly disengage from high schools that are large, anonymous, and alienating.

Civic education that teaches people to admire a flawed system is mere propaganda. We must prepare citizens for politics, but also improve politics for citizens. Neither effort can succeed in isolation from the other. Educational curricula and programs, including service-learning, if disconnected from the goal of strengthening and improving democracy, can easily become means of accommodating young people to a flawed system.

However, political reform is impossible until we better prepare the next generation of citizens with appropriate knowledge, skills, habits, and values. Students should feel that they are being educated for citizenship, but also that they can help to reform and revive democracy.

That’s what we are gathered here to do. Evaluation and measurement are just means to that end. They are powerful means, but they are not our goal--and neither is service-learning. Our goal is to renew American democracy.

---

Sources

[i] Boyte and Kari, Public Work {full citation}

[ii] Robert D. Putnam, “Community-Based Social Capital and Educational Performance,” in Diane Ravitch and Joseph P. Viteritti, eds., Making Good Citizens: Education and Civil Society (New Haven: Yale University Press, 2001), pp. 69-72.

[iii] Larry M. Bartels, “Economic Inequality and Political Representation” (2004, revised 2005), at http://www.princeton.edu/~bartels/economic.pdf

[iv] Harry C. Boyte and Nancy N. Kari, Building America: The Democratic Promise of Public Work (Philadelphia: Temple University Press, 1996).

[v] James Beane, Joan Turner, David Jones, and Richard Lipka, “Long-Term Effects of Community Service Programs,” Curriculum Inquiry, vol. 11, no. 2 (Summer 1981), pp. 145-146.

[vi] Gary Daynes and Nicholas V. Longo, “Jane Addams and the Origins of Service-Learning Practice in the United States,” Michigan Journal of Community Service Learning, vol. 11, no. 1 (Fall 2004), pp. 5-13; Melissa Bass, National Service in America: Policy (Dis)Connections Over Time” (CIRCLE Working Paper 11) and “Civic Education through National Service” (CIRCLE Working Paper 12).

[vii] Peter Titlebaum, Gabrielle Williamson, Corinne Daprano, Janine Baer & Jayne Brahler, “The Annotated History of Service-Learning: 1862-2002” at www.servicelearning.org/welcome_to_service-learning/history/index.php

[viii] Almost half of US high schools offer service-learning programs. See U.S. Department of Education, National Center for Education Statistics, “Service Learning and Community Service in K-12 Public Schools” (Sept. 1999), table 1. Since the term was coined ca. 1990, it is difficult to measure the increase since the 1980s, but it appears to be very dramatic.

[ix] Youniss and Yates, pp.

[x] Evaluation by J.P. Allen et al, summarized in Eccles and Gootman, eds., pp. 181-184.

[xi] The Center for Human Resources, Brandeis University, Summary Report, National Evaluation of Learn and Serve America School and Community-Based Programs (Washington, The Corporation for National Service, July 1999), pp. 1, 2, 3..

[xii] Shelley Billig, Sue Root, and Dan Jesse, “The Impact of Participation in Service-Learning on High School Students’ Civic Engagement,” CIRCLE Working Paper 33, pp. 26-7.