Regents Approve Scoring Changes to Grade 3-8 Math and English Tests

The Board of Regents today strongly endorsed the rationale presented by Education Commissioner David M. Steiner to adjust the “cut scores” on the state’s grade 3-8 math and English assessments based on research that clearly suggests the need to more accurately indicate “proficiency” on those exams.  With the support of the Board of Regents, Commissioner Steiner will set the cut scores accordingly. School, district, and statewide results from these tests will be released during the week of July 26.

The Regents discussion of the cut scores was informed by a review of research that analyzed how student performance on the grade 3-8 state tests relates to performance on the National Assessment of Educational Progress (NAEP) exam; how student performance on the state’s eighth grade math and English tests relates to later performance on Regents exams; and how performance on Regents exams relates to first year performance in college. Additional data was provided that linked NYS assessments to SAT scores.

Senior Deputy Commissioner John King said, “The data shows that schools responded to the assignment they were given – they worked hard to help students achieve standards as measured by the state tests that were being given at that time. And more students did, in fact, pass those tests. The problem is that those exams didn’t sufficiently test students’ abilities – the bar was set too low. But we are changing that now. It’s time to end the annual debate about whether our tests have become easier and to put to rest questions about what it means to achieve proficiency in New York.”

Regents Chancellor Merryl H. Tisch said, “For the past several years, we have seen more and more students scoring ‘proficient’ or better on our state tests. At the same time, however, their performance on the NAEP exam – the gold standard in testing – has remained essentially flat. We haven’t been testing the right things in the right ways. ‘Proficiency’ on our exams has to mean something real; no good purpose is served when we say that a child is proficient when that child is not.  So we’re improving our assessments by raising cut scores, making the exams less predictable, testing more areas, and making the tests longer. But more rigorous exams are only one piece of the Regents broader reform vision – a vision that includes a more challenging curriculum, better training for teachers and principals, and a world-class data system. In short, we are lifting the bar to ensure that New York remains at the very forefront of the national effort to raise standards.”

Commissioner Steiner said, “If our tests are to be a useful tool for parents and teachers, they have to give us meaningful information – not only about a student’s current level of proficiency, but also about that student’s future prospects. So we looked at linkages and connections, to better understand the signals that indicate whether a student is on track to pass Regents exams and to go on to higher education prepared to do college-level work. And it’s clear from our review that some students who scored proficient on state exams found themselves unprepared, without remediation, to do the work required of them when they reached college. We are committed to providing both the tests and the cut scores needed to achieve that result.”

In May 2009, the Regents directed that the state ELA and math tests be moved from January (ELA) and March (math) to late April and early May beginning in 2010 so that the exams could cover more material. The Regents also requested a thorough review of the Testing Program to ensure higher achievement standards and valid and reliable test scores that would reflect the appropriate level of rigor to ensure college and career readiness.

To help inform the Regents deliberations, the Department – together with testing experts Daniel Koretz and Howard Everson (both are members of the State’s Technical Advisory Group) and CTB/McGraw-Hill, the state’s assessment contractor – conducted a series of studies and surveys concerning student cut scores and student proficiency. Among their findings were the following:

• Nearly a quarter of students in all New York State two and four-year institutions of higher education take remedial coursework;
• Students taking more remedial courses in their first year of college are less likely to persist in higher education;
• Students who score below an 80 on their math Regents exam have a much greater likelihood of being placed in a remedial college course;
• Students who score above an 80 on their math Regents exam have a good chance of earning at least a C in college-level math;
• Students who score at least a 75 on their English Regents exam have a good chance of earning at least a C in Freshman Composition;
• Institutions of higher education around the state consider a score of 75-85 on Regents exams to be the bare minimum for college readiness;
• Students at the current Level 3 proficiency standard on their 8th grade math exam have less than a 1 in 3 chance of earning an 80 on their math Regents;
• Students in high need districts at the current Level 3 proficiency standard on their 8th grade ELA exam have about a 50-50 chance of earning a 75 on the English Regents;
• Students scoring below 80 on their math regents and below 75 on their English Regents exams have a high likelihood of scoring below 500 on the SAT.

A slide presentation summarizing these and other findings is available at this website:

http://usny.nysed.gov/A_New_Proficiency_07_22.pdf.

"The ELA and math cut scores established by the Regents and Commissioner Steiner signal college-readiness standards. Their approach to setting these cut scores was supported by extensive statistical and psychometric analyses conducted by a team of experts that included colleagues from CTB/McGraw-Hill, Professor Dan Koretz of Harvard University, and myself,” said Howard Everson, Professor and Senior Research Fellow in the Graduate School and University Center of the City University of New York. “By using these empirical methods to set the cut scores, the Commissioner has strengthened New York’s testing and accountability system."

"We confirmed that the performance standards had become very lenient.  To address this problem, NYSED will benchmark more rigorous 8th-grade standards against later performance on Regents exams and use typical patterns of growth to adjust standards in the lower grades. These approaches are reasonable and technically sound," said Daniel Koretz, Henry Lee Shattuck Professor at the Harvard Graduate School of Education.

Cut scores classify student proficiency into one of four performance levels based on their test performance; currently, the levels are defined as follows: Level 1, not meeting learning standards; Level 2, partially meeting learning standards; Level 3, meeting learning standards; and Level 4, meeting learning standards with distinction.

Under current regulations, schools must provide Academic Intervention Services (AIS) to students scoring at Level 1 or 2 on the state exams. Because the new cut scores will result in fewer students scoring at Levels 3 and 4 and more at Levels 1 and 2 – and the financial impact that such an increase in the number of students required to receive AIS services would have on schools – the Regents today approved amendments that will provide greater flexibility to districts in meeting their AIS requirements, essentially holding districts harmless financially in 2010-11. Specifically, for the 2010-2011 school year only, AIS regulations will be adjusted so that cut scores by which students are mandated to receive AIS – based on 2010 assessments – will be identical to the 2009 cut scores that required students to receive AIS.

The greater number of students scoring at Levels 1 and 2 will also impact the ability of schools and districts to make Adequate Yearly Progress (AYP) for accountability purposes. That could result in more schools being labeled “In Need of Improvement” under the federal No Child Left Behind law. During this transition year, the Commissioner will ask the U.S. Department of Education (USED) to allow schools and districts that would have made AYP – had the proficiency cut scores not been revised – to get credit for making AYP in 2009-10.  If approved by USED, this will significantly reduce instances where schools and districts that would have made AYP – had the cut scores not been revised – will fail to do so based on the new cut scores. SED received permission from USED to make a similar transitional adjustment in 2005-06 when the current grade 3-8 assessment system replaced one based on testing only in grades 4 and 8.

The Regents are embarking on a new era of reform – the goal is to provide all students with a world-class education that prepares them for college, work, 21st century citizenship, and lifelong learning. The reforms are sweeping and encompass the critical components that impact a child’s education: providing teachers and students with a meaningful and challenging curriculum; preparing and supporting teachers and school leaders to be more effective in their classrooms and their buildings; turning around or closing chronically underperforming schools; and ensuring that the state tests provide the information needed to accurately gauge whether students are actually getting the knowledge and skills they need to succeed. The Regents College and Career Readiness Work Group is continuing to examine the relationships between the Regents exams, the current graduation requirements, and college and career readiness. The Work Group will make recommendations to the Board in the fall.

Supporting Materials