a blog appears in the Washington Post | Main | Christopher Kutz on Complicity

April 27, 2004

the NAEP civics assessment

[update: At its May 2005 meeting, the National Assessments Governing Board increased the frequency of the NAEP Civics Assessment to every four years, twice the prior frequency of every eight years. This was a good decision, especially compared to the possibility that the Civics Assessment would be canceled altogethe.]

The 12th-grade NAEP Civics Assessment is threatened with termination. I know that this is not the #1 topic on the minds of millions of Americans, but it's an important and tricky issue. The federal officials responsible for NAEP are inviting comments right now (see below).

First of all, some general background about the NAEP itself. The National Assessment of Educational Progress, often called the "Nation's Report Card," looks like a test to the kids who take it, but it's an assessment of our schools and school systems, not of individual students. There are no "stakes" for the kids: no consequences for doing well or poorly. Indeed, no score is computed for a given student. That is because the test instrument is very long, and each person is asked to complete a random part of it. Asking many questions is the best way to measure skills and knowledge, so it's useful to ask a whole population more questions than any child or adolescent could sit still for. The results are then combined to generate statistics for the population.

Assessments are conducted in reading, mathematics, science, writing, U.S. history, civics, geography, and the arts. Traditionally, participation was voluntary; either a state, a school, or an individual child could decline to participate. This has changed somewhat with the passage of No Child Left Behind. Now all states are required to conduct the NAEP reading and mathematics assessments every two years at the 4th and 8th grade. They must conduct the NAEP in such a way as to generate statistically valid results for their whole student populations in those subjects. In principle, the NAEP is funded by the federal government and designed by the Educational Testing Service as a federal grantee. In practice, there are some costs for states and schools--in time if not in money.

In addition to knowledge and skill questions, students are asked some survey questions about their backgrounds and experiences in school and outside. Teachers complete a separate questionnaire. Students' transcripts are collected and connected to their performance on the assessment. All this data-collection makes NAEP a superb source of information about what seems to work in education. To name one example, Richard Niemi and Jane Junn have shown, using NAEP data, that social studies courses increase students' knowledge and also seem to improve their civic attitudes.

The National Assessment Government Board sets standards for Basic, Proficient, and Advanced in each NAEP. The Department of Education is thus able to say, for example, that 2 percent of students score at the Advanced level in civics at the 12th grade level. It is important to note that these cutoff points are subject to debate. They are set by panels of experts and citizens who have exercised their judgment to determine what should consitute Basic, Proficient, and Advanced mastery of a subject. A different group might reach a different conclusion.

A NAEP civics or citizenship assessment has been conducted five times since 1969 (and only three times at the 12-grade level), although history, geography, and economics have also been assessed periodically. The results were representative of the nation's student population, but there was no effort to assess a statistically representative sample in each state--or even in some of the states. Organizing state samples is more expensive than measuring a national sample.

Many experts consider the NAEP civics assessment to be a fine instrument for measuring skills and knowledge. Many would also like to measure civic attitudes and behaviors (tolerance, patriotism, concern for the common good, voting, volunteering, and many more). By law, NAEP scores only reflect students' knowledge and skills, although a few attitude measures were included in the survey portion of the 1998 civics NAEP.

The next NAEP civics assessment is tentatively scheduled for 2006, after an eight-year gap. It will again be a national sample without separate state results. Given the long gap and the lack of state-level data, we can't observe trends in civics. Nor can we compare state standards and curricula to find out what works, nor can we measure progress either nationally or by state, nor can we hold policymakers accountable for civics. Some people also think that it is symbolically damaging to assess civics every eight years if we are going to test reading and mathematics every two years, because what we assess is what we seem to care about. For these reasons, the many and diverse signatories of the Civic Mission of Schools report called for civics assessments every three years, with representative samples in each and every state.

This target appears to be receding. The National Commission on NAEP 12th Grade Assessment and Reporting issued a report on March 5, 2004 that calls for making reading and math NAEPs mandatory in all states. Twelfth-grade civics is to be an entirely optional assessment, conducted only at the national level and only if funding permits. Civics is explicitly placed in a third tier below reading and math (which are to be mandatory) and science and writing (which are treated as highly desirable). The report is a purely advisory document which the National Assessment Governing Board will review critically. NAGB is currently inviting public comments.

There appear to be three reasons that the advisory commission recommended emphasizing reading and mathematics and making civics a low priority. First of all, reading and math are the priorities of the No Child Left Behind law, which mandates NAEPs in those fields at the 4th and 8th grade. NCLB is silent about the 12th grade NAEP, but this report is in the spirit of NCLB. As the Washington Post reported recently, schools are dropping other subjects in order to concentrate on the NCLB mandates. Education Secretary Rod Paige defends the emphasis in the law. "A child that can't read is not going to learn history or civics," he says. But the narrowing of the curriculum has attracted critics as diverse as the National Conference for the Social Studies; NAGB's Executive Director, Charles Smith; and the Fordham Foundation's Chester Finn, who wrote:

the omission of social studies-and, more importantly, of history, geography, and civics-from NCLB is beginning to have deleterious effects. It's causing some states and schools to downplay these subjects in favor of those for which they'll be held publicly accountable and compared with each other. As the old educator truism puts it, what gets tested is what gets taught.

Second, only 55% of high school seniors who are asked to take NAEP assessments are now complying. It is likely that those who decline to participate are not a random group but have particular characteristics: compared to other students, they may be busier, or enrolled in poorer or more focused schools, or less academic. Such a low participation rate makes the results virtually meaningless. The report suggests solving this problem by making state participation in reading and mathematics mandatory, and conducting the other assessments occasionally, with national samples, if resources allow.

The third reason is an apparent assumption that schools do not have an essential civic mission. The report urges that NAEP "report on the readiness of 12th graders for college, training for employment, and entrance into the military." It passes over the readiness of 12th graders to be citizens, active in civil society, communities, and politics. This omission feeds fears in our community that testing is driving out civics.

In principle, there are at least four policies that could be adopted:

1. Separate federal legislation could mandate NAEP civics assessments in every state on a regular basis (e.g., once every three years), as a condition of federal funding.

2. NAEP civics assessments could be offered every three years, and funding could be provided to encourage states to organize separate representative samples.

3. The NAEP civics, history, geography, and economics assessments could be combined into a single "social studies" NAEP (with separate subscores for each subject); this NAEP could then be offered every three years in as many states as possible.

4. A completely new--and much shorter--assessment of adolescents' civic knowledge, skills, attitudes, and behaviors could be conducted as a random survey, with a big enough sample to generate state-level results in at least the large states.

Posted by peterlevine at April 27, 2004 11:36 AM


Post a comment

This blog is under attack from comment spammers, who are causing a problem for the server. I believe I can block them by upgrading to a recent version of MoveableType. However, I do not have time to do that until late December. Therefore, I have temporarily disabled comments. Please feel free to email me feedback at plevine@umd.edu.

Site Meter