Guest Opinion | In 2011, PUSD was Thinking About its “Georgia issues” with Student Testing Cheating

We weren’t the first to recognize that Pasadena Unified School District test cheating has similarities to the Atlanta, Georgia, testing scandal that sent 11 educators to jail.

Yesterday’s Pasadena Now op-ed titled “Student testing cheating – Does PUSD have Georgia issues?” compared the Georgia cheating scandal to PUSD’s ignoring evidence that Roosevelt Elementary School cheating was more pervasive than admitted. But PUSD’s now-retired Chief Technology Officer Dr. Gary Carnow saw that parallel 5 years ago. Dr. Carnow was PUSD’s point man in 2011 who “investigated’ cheating at Roosevelt and Sierra Madre elementary schools. In his September 2, 2011, email to 6 PUSD administrators, Dr. Carnow told his fellow administrators “State of California dropped the part of the contract with ETS regarding Erasure Detection in 2009 to save money. There will be no Georgia issues found in California.”

Why did Dr. Carnow have a sigh of relief that the CDE dropped Erasure Detection in 2009?

The “Georgia issues” that Dr. Carnow says won’t be found because erasure detection is no longer in use in California refers, of course, to the test cheating scandal in Atlanta. “ETS” is the Educational Testing Service, a test development and testing company. The “Erasure Detection” program detects cheating by scanning answer-sheet erasures, identifying the first and second marks on paper test sheets. This cheating-discovery method’s premise is that a high percentage of wrong-to-right erasures deviating from the statistical norm probably indicates testing irregularities such as school employees prompting students to correct mistaken entries. The California Department of Education contracted with ETS for statewide erasure detection before 2009 and from 2012-2013. For budget reasons, it dropped statewide erasure detection between 2009-2011 – the period during which Roosevelt Principal Juan Ruelas showed remarkable test score increases at Madison. Erasure detection became an unnecessary cheating-detection method beginning with 2015 SBAC testing because paper tests were eliminated in favor of secure electronic testing.

Dr. Carnow’s email suggests a PUSD culture tolerating cheating

A school administrator who seriously intended to address cheating in PUSD’s schools and who knew that the CDE stopped erasure detection would likely say something like “We’re going to have to buy ETS’s Erasure Detection program in order to investigate the extent of the cheating.” Dr. Carnow’s comment communicates that cheating won’t be detected rather than asking “what do we do to discover the extent of cheating? He is saying that PUSD won’t get caught cheating coaching students to erase and replace wrong answers because the cheating-detection measures aren’t in place – so celebrate and give a sigh of relief that California dropped erasure detection! The PUSD culture reflected by his remarks is an attitude that cheating should be ignored when it can be. It suggests that the 2011 cheating scandals may have only been the tip of the iceberg, with 90% of the problem ignored because it wasn’t on the surface.

Additional statistical test-score analysis indicates the need to rigorously investigate Roosevelt’s test scores

Yesterday’s graphic chart was a macro-summary of Roosevelt’s test scores which suggests test score statistical patterns consistent with Roosevelt’s cheating opportunities, or lack thereof, from 2009-2015. Just like PUSD’s Board President Elizabeth Pomeroy, PUSD’s Board Member from the Madison District, Patrick Cahalan, dismissively rejects that such data can validly suggest that undetected cheating was going on at Roosevelt. Cahalan offers no alternative explanation for the data. He just says that you have to look at smaller data sub-sets to draw valid conclusions. But the first suggestion in the U.S. Department of Education’s 2013 “Testing Integrity Symposium: Issues and Recommendations for Best Practice” section on “Test-score analysis” is to do macro-examination and it suggests then drilling down to look at “individual student-level data as well as classroom or school-level results to identify unusual trends.”

The Roosevelt cheating that Dr. Carnow’s 2011 investigation identified was in the 2nd grade. Statistical analysis of a 2011 class from the 2nd grade is useful to determine whether it is likely that there was a failure to investigate other 2011 cheating at Madison. Statistical analysis of Roosevelt’s 2011 3rd grade testing at “proficient” or higher is thus revealing. Despite having a high English-second-language population, Roosevelt in 2011 tested that a remarkable 97% of its 3rd grade class were “proficient” or higher in English – a statistic that invites suspicion. 2011 was also the last year in the 3-year interval that the state did not conduct erasure testing. What would likely happen If Roosevelt took advantage of the 2011 opportunity to cheat when the CDE was not doing erasure detection but then curtailed cheating in 2012 when it had to avoid getting caught again because the CDE re-instituted erasure detection? The 3rd grade class would likely have significantly lower percentages of proficient and proficient-plus English ratings when the 3rd grad became to 4th grade in 2012 the 5th grade in 2013. This chart shows that happened.

 

 

 

 

 

 

 

 

 

 

 

3-YEAR CLASS PROGRESSION CHART

PUSD’s Roosevelt “investigation” records are inexplicably missing

Dr. Carnow’s attitude seems to reflect a PUSD culture tolerating student test cheating. Statistical analysis of both the macro-scores at Roosevelt and the micro-2011 3rd grade class both suggest cheating. With indications of much more systematic Roosevelt cheating than with just 3 students in Roosevelt’s 2011 2nd grade class, we suspected that Dr. Carnow’s attitude would lead to a cover-up investigation rather than a rigorous investigation. So we reached out to Dr. Carnow by email and voicemail to ask him about whether his investigation was sufficiently rigorous, He ignored our requests to talk to him. So our next step was to ask for PUSD’s investigation records to determine how rigorous Dr. Carnow’s investigation was. Instead of getting investigation records, we got a rude surprise.

In December, we made the public records requests to PUSD Superintendent McDonald for all documents showing testing cheating investigations at Roosevelt while Ruelas was its principal. McDonald’s January 22 response, through PUSD’s attorney Jeff C. Marderosian, admits that such PUSD files that should exist cannot be found, saying that “[a]fter being unable to locate a file that should exist, I instructed staff this week to contact the California Department of Education (CDE) and request a copy of the Department’s entire file regarding the testing issue at Roosevelt in the 10-11 school year. Staff has made the request to CDE and I will advise you when it is received. From the documents you were sent early in the week, it appears that Dr. Carnow handled the matter and that it involved 3 students at Roosevelt. I have no explanation as to why the file could not be located.” It turns out that the CDE also has no files on Madison’s student test cheating except for a single letter acknowledging that PUSD reported cheating concerning the testing of the 3 students.

PUSD needs to hire an independent investigator to systematically analyze Roosevelt’s testing under Ruelas

The issue now is not just whether Superintendent McDonald’s rationale for imposing Ruelas on Madison can any longer be defended. His rationale is in tatters and plainly cannot reasonably be defended. There are now broader issues as to whether PUSD has a culture of tolerance for cheating and has been ignoring systemic cheating issues. PUSD’s continuing to bob and weave to protect Superintendent McDonald for his disastrous Roosevelt decision is a recipe for disaster for a public school system that requires transparency and accountability. Dr. McDonald and PUSD’s Board need to end their ostrich-like evasion, face up to the problem by hiring an independent and reputable professional knowledgeable about cheating detection to do a thorough investigation and publicly disclose the complete investigation report, and remove Ruelas as Madison’s principal.

Dale Gronemeier and Skip Hickambottom are local civil rights attorneys who represent the Madison grassroots coalition, the Citizens Council for Empowerment and Justice at Madison.