This week, the Mississippi Department of Education will release 2015-2016 MAP scores for all public school districts. These proficiency scores reflect achievement on the new state assessment called MAP that students took Spring 2016; they will not include student growth scores or school grades, which are expected to be released later this fall. Mississippi students have taken a different state assessment in each of the last three years, making understanding and interpreting school data challenging. This explainer is intended to help the press, parents, and the public understand and interpret the 2015-2016 MAP scores. In general, we expect to see scores remain at the level of the 2014-2015 PARCC scores, which is to say that the percentage of students scoring proficient will be much lower under MAP than under the old MCT2. To understand why, read more below.

What is MAP?

MAP,[1] the Mississippi Assessment Program, is the new state test developed by Questar. Mississippi students in grades 3-8 took a MAP test in math and English. High school students took a MAP test in Algebra I and English II.

MAP is the third state test that Mississippi has had in the last three years:

  • In 2013-2014, students took the MCT2.
  • In 2014-2015, students took PARCC.
  • In 2015-2016, students took MAP.

Students will also take MAP this year, 2016-2017.

Why has the test changed so much?

In the 2014-2015 school year, Mississippi finished phasing in its implementation of the new Mississippi College and Career Readiness Standards and implemented a new test to match those standards. That test, PARCC, was only used for one year after a problem arose with getting the test approved through the state contracting process. Mississippi sought bids to create a new test, MAP, and plans to use this test for the foreseeable future.

What will the MAP scores show?

The MAP scores to be released this week are called proficiency data because they tell us the percentage of students scoring in each of five achievement levels, one of which is proficiency. This a change from the old MCT2, which only had four levels. PARCC scores were the first scores reported in five achievement levels. MAP continues with five levels but with new names for each level. MDE defines each of the MAP levels as follows:

  1. Minimal—“A student performing below the Basic level inconsistently demonstrates the knowledge or skills that define basic level performance.”
  2. Basic—“Students at the Basic level demonstrate partial mastery of the knowledge and skills in the course and may experience difficulty in the next grade or course in the content area. These students are able to perform some of the content standards at a low level of difficulty, complexity, or fluency as specified by the grade-level content standards.”
  3. Passing—“Students at the Passing level demonstrate general mastery of the knowledge and skills required for success in the grade or course in the content area. These students are able to perform approaching or at the level of difficulty, complexity, or fluency specified by the grade-level content standards.”
  4. Proficient—“Students at the Proficient level demonstrate solid academic performance and mastery of the knowledge and skills required for success in the grade or course in the content area. These students are able to perform at the level of difficulty, complexity, or fluency specified by the grade-level content standards.”
  5. Advanced—“Students at the Advanced level consistently perform in a manner clearly beyond that required to be successful in the grade or course in the content area. These students are able to perform at a high level of difficulty, complexity, or fluency as specified by the grade-level content standards.”

What will the MAP scores not show?

The MAP scores will not show how much students grew from last year to this year. That data, which is called growth data, will not be released until later in the fall. To get a fair picture of a school’s performance, we must look at both proficiency data and growth data. Proficiency data tells us how many children are achieving the standards that we have set for them; growth data tells us how far a school moved a child, regardless of his or her starting place, towards the goal of proficiency. Unfortunately, growth data is also hard to calculate when the test changes. MDE will have to use statistical methods to calculate growth instead of merely comparing one score to the next; they are expected to release their growth estimates later this fall. Taken together with proficiency data, this will give us a more complete picture of school performance, but as we explain below, all school data should be interpreted this year with extreme caution.

What does Mississippi First expect the data to look like?

Typically, we would expect proficiency rates to rise slightly over the previous year. This has generally been the trend for proficiency data in Mississippi for the last decade. The last two school years however have been anything but normal. Mississippi’s state learning standards–which describe what children should know and be able to do–have grown far more rigorous and so have the state tests. This means that children are expected to know a lot more in a subject at each grade. As standards have gotten harder, the tests have gotten harder, too: now, children must have a better mastery of more difficult skills to score “proficient” on state tests. Furthermore, the move from four to five proficiency levels means that there is more differentiation among the levels, making it even more challenging. The PARCC test was the first test to reflect this change. This caused an expected but dramatic drop in proficiency rates in the 2014-2015 data. For example, third grade math students scoring proficient or above dropped from 62.4% to 33.1% or by 29.3 percentage points; third grade reading students scoring proficient or above dropped from 50.9% to 28.7% or by 22.2 percentage points.

Normally, we would see scores improve this year because students and teachers will have had more time to adjust to the harder standards and tests. Again, last year was anything but normal. As with the roll-out of any new test, PARCC had a lot of implementation issues. These issues delayed the release of the proficiency scores from 2014-2015 for over six months and the release of growth data for a year. Because of this, teachers were not fully able to use the PARCC information to adjust their instruction last year: they taught an entire semester without knowing if they had raised the difficulty of their lessons enough.

In addition to teachers being unable to adjust their instruction, Mississippi administered MAP for the first time in 2015-2016. Scores from the first year of a new test nearly always decline somewhat, even if the test is considered to be equal to the one it replaced. This is because children and teachers must adjust to the new test format. Within three years, this “implementation dip” usually resolves.

As a result of all of these factors, we expect 2015-2016 school data to be similar to 2014-2015 data in the percentage of students scoring in each category, especially the top two—proficient and advanced. In practical terms, this means that we expect scores to remain low.

For the charter schools which will be included in this score release for the first time, we expect low initial scores for many of the same reasons as we expect low scores for traditional public schools: harder standards, harder tests, five achievement levels, a lack of data from the prior year, and a new test. We also expect charter schools’ scores to be low in the first year because each of the schools started with middle school children who were, on average, 2-3 grade levels behind. Proficiency scores alone will not reveal how well these schools moved children from a very low starting point towards proficiency. A recent national charter school study shows that charter school students improve more each year they remain in a charter school, meaning that first-year performance is likely to underestimate how the school may perform over time. In order to fairly assess these schools, we will need to see growth data as well as data comparing charter school students to similar students who remained in traditional public schools. We do not expect either of those data points until later in the fall.

Will low MAP proficiency scores mean that Mississippi schools have gotten worse?

Not necessarily. Because of all the factors we discuss above, we should really understand the data this year as a new starting point for all schools. Children likely have not learned less—they may have even learned more—but because of the changes, it is not possible to truly compare performance to prior years. Within the next three years, we should see scores begin to climb again, although it may take many, many years for performance to rise to the level that it was when our standards were lower and the tests were easier.

[1] Some school districts use a progress monitoring test called Measures of Academic Progress (MAP), but the Mississippi MAP test is different.

3 Comments on “Understanding and Interpreting 2015-2016 MAP Scores: Proceed with Caution

  1. Changing tests from year to year is yet another reminder for teachers to TEACH TO THE STANDARDS and not TO THE TEST. If students have a deep conceptual understanding of the content, then it doesn’t matter what test is thrown at them. They’ll be ready.

    1. Luke,
      Thank you for your comment. You are quite correct. Teachers should always, always teach to the standards and not to any given test. Teachers who do this, and their students!, will experience much greater success no matter the challenge. We do want to clarify something important, though: even if a teacher has been teaching to the standards, her proficiency scores may be low this year. Here is why.
      First, teachers operate within a system, and they must rely on that system to set them up for success. School districts that committed to the new standards when they were adopted–in 2010–and began implementing them fully and seriously are likely to experience less of a proficiency decline during this transition. This is because the students in those districts will have had several years to adjust to the increased rigor of the standards and will have the necessary foundational skills to do well on the aligned tests. Unfortunately, we know that many districts waited until 2014-2015 to switch to the new standards, which meant that students and teachers were asked to make a large shift in one year. Even teachers doing heroic work to grow students more than one year’s worth in one year’s time would not have had enough time to get students all the way to the new proficiency mark.
      Secondly, we still expect a score decline (even if it is not as dramatic) in districts who implemented early and seriously for two reasons. Without the feedback that an aligned test can give, it may be hard for teachers to calibrate their instruction to the level of difficulty the new standards demand. As you know, teachers did not receive PARCC scores in a timely enough way to be able to incorporate that feedback for much of last school year, if they were able to at all. Teachers using the standards may have raised the level of their instruction a lot but not enough. Feedback for teachers is always critical, and it is likely that even the best-prepared districts needed to see the feedback from the first aligned test to be able to adjust properly. We also expect a score decline because not only have the standards changed but what it means to be proficient has changed as well. Students must master harder standards at greater depth and demonstrate that mastery on a much more challenging and discerning test. PARCC was entirely computerized with new types of questions that students, especially those right around the proficiency mark, may have struggled enough with that it could have made the difference. MAP was also computerized (except for the writing portion) with similar question types. The greater familiarity students should have had last spring should have helped students, but combined with the other issues, we still believe scores are likely to be depressed this year, even in districts doing everything “right.”

Leave a Comment

Your email address will not be published. Required fields are marked *