Scoring glitch delays ISTEP results - maybe until at least December

The delay is due to a glitch associated with the new test required to be used this year, after the state school board ditched Common Core and adopted new Indiana-only academic standards, which state lawmakers required be "the highest standards in the United States."

Ellen Haley, president of test vendor CTB/McGraw-Hill, explained to the board the new online test now requires students show their work to answer math problems instead of just selecting a letter indicating the correct solution.

She said the company prepared 10 expected responses for each of the 394 questions, but recently discovered on at least 100 of them, students were using correct — but unanticipated — methods of reaching the right solution.

The company's scoring system was marking them as wrong.

As a result, company scorers have to re-evaluate each of the unanticipated correct responses and add them to the computer program that tallies test results, Haley said.

Unexpected problems affect timely results

Once that's finished, every test will be re-scored -- a process expected to take until at least early October, she said.

"This is the nature of switching so dramatically," Haley said. "There's nothing left over from the previous test. We can't use the scale. We can't use the weighted formula. You started over again."

After scoring is completed, the state school board then must set "cut scores" by deciding which results qualify as satisfactory academic progress.

If CTB/McGraw-Hill has finished its work on time, the board could approve cut scores at its Oct. 14 meeting; the company then would need eight weeks to apply the cut scores to student test results and send score reports to parents and schools.

However, Steve Yager, a state school board member from Fort Wayne, told Haley he is skeptical CTB/McGraw-Hill will come through in light of numerous past failings, which could delay cut-score approval until the Nov. 4 or Dec. 2 board meetings and, with eight weeks of processing, push release of test results into 2016.

"What's happening is girls and boys are just being damaged, and teachers are being damaged, by the ineffective practices of your company," Yager said. "I'm looking forward to you delivering on what you said you can deliver, because we educators are extremely disappointed."

Delayed results affect myriad aspects of teaching, administration

The test score delay likely also will prevent the state school board from assigning A-F school grades by the end of 2015, as required by Indiana law. The board typically issues school grades two to three months after ISTEP results are released.

Those grades are used as a component in teacher evaluations; determine in part teacher pay and bonuses; and are a key factor in whether low-performing schools get taken over by the state or shut down.

"It's not that helpful, if I'm a teacher or a school, getting my letter grade almost a full year after testing," said Gordon Hendry, a board member from Indianapolis.

"So much has changed, it makes it much harder to be responsive to a letter grade that, for instance, has gone down. It almost sets you back two years."

In fact, Hoosier students may be taking next year's ISTEP tests -- through new vendor Pearson Education Inc. -- before their schools are rated based on last year's results.

Natalie Thorgren, a teacher at Valparaiso's Northview Elementary School, said the delayed test scores will make it hard for educators to adjust their teaching in time for the next test, if the results show something wasn't working the year before.

"What is the purpose of testing? It should be to see what students have learned, then help them to learn the things they aren't getting," Thorgren said. "We should be able to use that information to remediate, and we're not getting it."

The scoring delay is the second major problem to afflict this year's ISTEP exam following a February dispute over the extra-long duration of the test, also caused by the adoption new state academic standards.

The test ultimately was shortened by eliminating additional nongraded questions set to be used in future exams.

But Haley said doing that also made it impossible to throw out unanticipated problem questions, like the 100 involved in the scoring issue, and still maintain the reliability and validity of the exam.