An unpublished op-ed: On the Collateral Damage of NWEA

We're honored to publish this op-ed which was written for the Chicago Tribune but went unpublished. It's worth a read.

The author is Martin Gartzman, a Senior Associate at UChicago STEM Education at the University of Chicago. From 2002–2006, Gartzman served as the CPS Chief Mathematics and Science Officer, where he directed the Chicago Math and Science Initiative (CMSI), the district’s touted program for mathematics and science improvement.

The Chicago Public Schools Inspector General’s recent report about “irregularities” in administration of the Northwest Evaluation Association’s MAP assessments generated discussion and disagreements about whether those problems amounted to cheating. As important as those issues are, there are far more profound issues surrounding the use of the MAP assessments that have implications for the many school districts throughout Illinois that use the MAP. 

Consider this non-education-related analogy: A thermometer is a useful tool to determine if your body temperature is within an expected range; but further investigation using more precise tools is needed before a doctor can make an accurate diagnosis. A doctor who prescribes treatment for the Coronavirus among all patients who have a fever, without further investigation of the fever’s cause, would be committing medical malpractice. 

The MAP assessment is designed to function much like a “thermometer check,” nothing more. MAP does an OK job of identifying the “ranges of learning” into which different groups of students fall. But MAP is not capable of pinpointing with any accuracy the skills and understandings that students do and do not possess, or doing anything more than rough speculation about the specific learning needs of individual students. Testing experts are unanimous about that. Unfortunately, the pressures of high stakes accountability have led CPS and other school districts to keep hoping that maybe the experts are wrong.

A “golden rule” involving assessments is that they should be used only for the purposes for which they were designed. Yet the MAP assessment is being used in nearly half  of Illinois school districts for a wide range of different purposes, from high-level, systemic evaluations to prescribing remediation activities for individual students. 

The hard truth is this. When systems like MAP report out test information in ways that encourage teachers and school leaders to think they can do more with the test results than is merited by the assessment’s design, it’s tantamount to encouraging educational malpractice. And prescribing education “solutions” based solely on that data is akin to educational quackery. A great irony of MAP and similar testing systems is that they are often touted as tools for improving data driven decision making. But after more than a decade of large-scale use, there is no independent evidence that MAP and other tests like it are contributing in any meaningful way to improved teaching and learning. 

The collateral damage done by the MAP assessment is broad and far reaching. It encourages teachers to dumb down the curriculum to a set of unrelated, discrete skills. This ends up robbing instruction of the kinds of applications, problem solving, and thinking that are essential in preparing for the world in which the students will graduate. Compounding that is the misalignment with Illinois learning standards and the questionable uses of MAP data for things that the MAP is not designed to do, such as teacher evaluations, high school admissions, and other high-stakes decisions for children.

Most of the information provided by the MAP assessment can already be inferred through the annual state assessment results and the day-to-day tests, quizzes, assignments, and teachers’ observations in classrooms. We use valuable instructional time and spend millions of dollars to gather information via the MAP that teachers and schools likely already have, leaving little time or resources to invest in strategies that have proven to make a difference in student learning, such as helping teachers do a better job with daily assessment and diagnostic techniques.

At best, the MAP and other assessments like it are making no real difference. At worst, but with good intentions, they are making challenging problems even more difficult.      


Here's what RYH said about the OIG report at the February 26, 2020 monthly meeting of the CPS Board of Ed