Tuesday, February 10, 2015

So the score is scaled...

Print Friendly and PDF
Here is an Excel plot showing raw score on the January 2015 New York State regents in Algebra One (Common Core) at the bottom, and the scaled score (out of 100) to the left. The dividing spots for New York's 5 "Performance Levels" are given.
The 5 groups are described as such (from here:)
  • NYS Level 5 Students performing at this level exceed Common Core expectations.
  • NYS Level 4 Students performing at this level meet Common Core expectations.   
  • NYS Level 3 Students performing at this level partially meet Common Core expectations (required for current Regents Diploma purposes).   
  • NYS Level 2 (Safety Net) Students performing at this level partially meet Common Core expectations (required for Local Diploma purposes). 
  • NYS Level 1 Students performing at this level do not demonstrate the knowledge and skills required for NYS Level 2.
Take note here, the New York State Education Department established in May, 2011, that "the local diploma option remains available to all students with disabilities provided they earn a score of 55-64 on one or more required Regents examinations." (see here) The state has a clearly established goal of getting these students to level 2, thus the steepness of the blue in region 1 above makes political sense.

Naturally, this quick run to level 2 needs a compensatory move in the opposite direction, but not too quickly. In this case, the state makes group 3 a narrow group. Raw score 19 means level 1, raw score 30 means level 3. That is the narrowest band on the raw score scale. A quick jump from "failure" to get over the "safety net." All students at level 3 get the same New York State credit that level 5 students get.

Once we begin the transition from level 3, we have to "toughen" up things. The minimums for Regents Diploma or Advance Regents Diploma require a score of 65 (scaled scores). To get "with Honors" tagged on, a student must average 90% on all required Regents exams. To get to scaled score 90 on this exam is a massive leap from getting that 65.  

The bulge at the low end requires a lessening rate of increase as we move to the higher end. In this case the rate decreases enough to create a concavity shift in region 4. This created the opportunity for a "finishing lap" in level 5 that conforms more closely to the "percentage score". Divide raw scores at the upper end by 86 and you will see they give results fairly close to the scaled score. This is valedictorian/salutatorian space, and the scaling here probably helps quiet arguments down the line.

Please take note that when New York shifted to its "Regents for all" philosophy, it created a scenario where political economics demanded the tests be easier to pass, yet national and international standards dictated that these tests could not be too simple, they could not be "easy A's".  These dual goals are addressed via the scaling above.

Perform a similar analysis on exams since the first math A that had a conversion chart. The data can be found by plodding through http://www.nysedregents.org/.

No comments: