Glossary of Research, Evaluation, and Assessment Terms:
Achievement Level—Used to summarize student scores on the MCA, TEAE writing, and SOLOM into 5 categories (Levels 1- 5) and on the TEAE reading into 4 categories (Levels 1-4).

Alternate Assessment—Required assessment of developmental academic skills and functional living skills for special education students who do not take the MCAs. It consists of rating scales that are completed by teachers.

AMAO—Annual Measurable Achievement Objectives, also known as “English language proficiency performance targets,” used to track ELL student progress.

“Apples to Apples”—A method used to isolate the performance of a particular subgroup of students (e.g., Asian American, non-special ed, non-ELL, free or reduced lunch), also known as multi-level disaggregation.

Attendance—Attendance rate is defined as the Average Daily Attendance (ADA) divided by Average Daily Membership (ADM) as recorded in MARSS.  ADA equals the number of days a student is marked in attendance divided by the number of instructional days.  ADM equals the number of days a student is reported as enrolled divided by the number of instructional days.

AYP—Adequate Yearly Progress; reported annually for each Minnesota district and school by the state department of education.

BST—Basic Skills Tests; given to students in the following subjects and grades and then in each subsequent grade until a passing score is achieved (required for graduation):

 

Grade

Writing

10

x

Campus—The database used in the district to collect student information including enrollment, demographics, address, attendance, and discipline.

Cell Size Limitation— Cell Size Limitation is the minimum number of students needed to measure AYP for an indicator.  For participation and proficiency, the number of students in grades 3 and 5 are combined for a subject.  Note that on the proficiency indicator, the required minimum cell size is higher for the special education and LEP subgoups than for the other student subgroups.
           

Indicator

Subgroup

Cell Size

Participation

All students, All 8 subgroups

40

Proficiency

All students, All 5 ethnic subgroups, Free/Reduced-Price Lunch

20

Special Ed, LEP

40

Attendance

All students, All 8 subgroups

40

Graduation

All students, All 8 subgroups

40

Clusters/Subskills—One of a number of specific skill areas within a subtest or subarea of a test (e.g., Synonyms is a cluster within Vocabulary on the SAT10 and MAT7).

Confidence Interval—The boundaries, or limits, within which the true population mean lies.  A 95% confidence interval indicates that we are 95% certain that the interval contains the mean.  MDE calculated a confidence interval around the index target; schools/districts make AYP by exceeding the lower limit (score) of the confidence interval.

Continuous Enrollment—Students who were enrolled in a school (or district) for a “full academic year”—defined by MDE as enrolled in the same school/building on October 1 and on the day of the test—are considered “continuously enrolled” in that school/building.

Cumulative Pass Rate—This term refers to the number and percent of students that have passed the reading, math, and writing BSTs by their class’s expected graduation date.
 
Disaggregation—Refers to reporting data by a particular subgroup of the population (e.g., Asian American or special ed) versus reporting it for the entire population as a whole.

Dropout—A student in grade 7-12 who left school and did not reenroll by October 1 of the following fall.

Dropout Rate—There are two definitions of dropout rate in common use:

  • The Annual Dropout Rate is the number of dropouts divided by the number of students enrolled on October 1.
  • The Four-Year Dropout Rate is reported as a component of the SPPS Four-Year Cohort Graduation Rate (see graduation rate definition below).

ELL—English Language Learners, also known as LEP, Limited English Proficient.  MDE has defined three stages of ELL students.  Students in the first stage of ELL are those language minority students who have not scored at the proficient levels on the state language ability assessments (i.e., TEAE and SOLOM).  Students in the second stage, known as Transitional Language Learners (TLL), are those language minority students who have scored at the proficient levels on the state language ability assessments (noted above) but who have not yet scored at the proficient level on the state reading assessment (i.e., MCA) for three consecutive years.  Students in the third stage, known as Fluent Language Learners (FLL), are those language minority students who have achieved proficient scores on the state reading assessment for three consecutive years.  When reporting ELL analyses, only those students in the first stage are included. 

Equipercentile—Method of equating tests by determining which raw scores on two instruments have the same percentile rank.

Field Test—This is the first stage in MDE’s three-step test development process.  During field testing, the test is administered to a subset of the student population in order to determine how well the questions work.  From the field test, a “pilot test” is developed using proven test items and given to a larger pool of students.

First Attempt Pass Rate—BST test results are reported as the number and percent of students passing reading, math, and writing tests on the first attempt.  For math and reading BSTs, this first attempt at the test occurs in grade 8.  Students first attempt the writing BST in grade 10.

Graduation Rate—There are two definitions of graduation rate in common use:

  • The NCLB Cumulative Graduation Rate, defined and used by the state for NCLB purposes, counts the total number of students graduating divided by the total number of graduates and dropouts (i.e., students who were not found as re-enrolled in any other school in the state by the following fall) multiplied by 100.  Students who transferred into the district are included in this calculation.
  • The SPPS Four-Year Cohort Completion Rate is calculated by following a cohort of students who started 9th grade in an SPPS school.  Four years later this cohort is divided into three groups: those who graduated in four years, those who are continuing their high school education in SPPS, and those who dropped out and did not reenroll in SPPS by the following fall.  Students who transferred into or out of the district are not included in this calculation. 

Growth/Trend/Gain—Calculation of change in academic achievement over time.  An NCE gain score (calculated by subtracting the prior year’s score from the current year’s score) of zero refers to a gain of one year; any positive NCE gain suggests gain of greater than one year, and any negative NCE gain suggests gain of less than one year.  Growth is often reported as an average for a population or sub-population of students.

Index Point—Students are assigned index points based on their achievement level in each subject on the MCA:
1 point—Level 3 and above
½ point—Level 2
0 points—Level 1

Index Rate—The index rate for a group of students is their total number of index points divided by the total number of students enrolled on October 1 multiplied by 100 (i.e., average index point).

Index Target—The target set by MDE each year that must be reached in order to make AYP.

Indicator—A category of data used to describe the performance of a school or district.  MDE uses 4 indicators in determining AYP status—participation, proficiency, attendance, and graduation. 

Item—A question on a test.

Level—See “Achievement Level.”

MARSS—Minnesota Automated Reporting Student System.  Database of student information reported to MDE.  The data come from student enrollment collected by schools in the Campus system.

MDE—Minnesota Department of Education.

Multi-Level Disaggregation—A method used to isolate the performance of a particular subgroup of students (e.g., Asian American, non-special ed, non-ELL, free or reduced lunch), also known as “Apples to Apples” comparisons.

NCE—Normal Curve Equivalent.  NCEs are test scores that are based on a national sample of all students taking a particular test.  They range from 1 and 99 with a score of 50 being equal to the national average.  Unlike percentiles, NCE scores are equal interval, which means that the difference between any two adjacent scores is the same. This characteristic allows NCEs to be legitimately aggregated and averaged.  For this reason, NCEs are the scores of choice in reporting aggregated results or averages across various groups (e.g., schools, grade levels, classrooms).

NCE Change—The difference in NCE score from one year to the next on the same test.  An NCE change of zero indicates average or expected gain.  All positive NCE change is considered good, as it suggests greater than expected progress.

NCLB—"No Child Left Behind" Act of 2001, the reauthorization of the Elementary and Secondary Education Act that President George W. Bush signed into law in January 2002.

Operational Test—This is the third stage in MDE’s three-step test development process.  Once a new test has been “field tested” and “pilot tested,” it is considered to be a complete and “operational” test.  Tests are not used for AYP purposes until they are operational. 

Participation—Measure of % of students enrolled in a school or district that completed the MCA (grades 3 and 5 combined) or alternative assessment.

Percent—Not to be confused with percentile, a percent or percent score is the number of items correct divided by the total number of items and then multiplied by 100. 

Percentile/Percentile Rank—Percentile ranks show how a student compares with others who took the test on a scale of 1 to 99.  (For example, if a student scored in the 65th percentile on a test, the student performed as well or better than 65% of the other students who took the test.)  By definition, more students score at or near the 50th percentile than at either ends of the scale.  Therefore, percentiles differ from NCEs in that they do not represent equal units of achievement along the scale (e.g., the difference between percentile ranks of 5 and 10 is not the same difference in achievement as the difference between percentile ranks of 50 and 55).

Pilot Test—This is the second stage in MDE’s three-step test development process.  After a new test is “field tested” and before the test is administered widely and used for accountability purposes (i.e., “operational”), it is “pilot tested” with the population of students to ensure that it is working properly. 

Proficiency (MCA-II/AYP)—Defined by a cut score set by MDE.  On the MCA-IIs, any score of Level 3 or above is considered proficient; any score below Level 3 is considered not proficient.  For the purposes of AYP, proficiency calculations are across grades within a subject area.  Proficiency calculations for AYP are based on students who were enrolled in a school (or district) for a “full academic year”—defined as enrolled in the same school/building on October 1 and on the day of the test (see “continuous enrollment”).

Proficiency (TEAE/SOLOM)—Defined by a cut score set by MDE.  Any score of Level 4 or above is considered proficient on the TEAE reading and SOLOM; any score below Level 4 is considered not proficient.  Any score of Level 5 on TEAE writing is considered proficient; any score below Level 5 is considered not proficient.  For the TEAE and SOLOM, proficiency is reported as the percentage of students who achieved the required (proficient) achievement levels on all three assessments (TEAE reading, TEAE writing, and SOLOM).

Progress (TEAE/SOLOM)—On each of the three assessments (Reading, Writing, and SOLOM), an ELL student who has a score in two consecutive years contributes a “yes” or “no” to the question: did the student’s test score increase since the previous year?  A group of 1000 ELL students with scores in two consecutive years then has 3000 total possibilities for a “yes” or “no” to this question.  The proportion of “yes” marks over the sum of “yes” and “no” marks is the percent of ELL students making progress in any given year.

Raw Score—The number of items a student has answered correctly.

REA—Saint Paul Public Schools Department of Research, Evaluation, and Assessment housed in the Office of Accountability.

Safe Harbor—Safe Harbor is an AYP designation that is achieved when a school or district reduces the number of non-proficient (i.e., Level 1 and Level 2) students by 10% from the prior year.  Safe Harbor only applies to the proficiency indicator and is only calculated when the index target has not been met.  A school or district can make AYP based on Safe Harbor provided that group meets the AYP target for either attendance and/or graduation rate, as appropriate.

SAT10—Stanford Achievement Test Series, Tenth Edition; given in 2005 in grades/subjects:

Scaled Score—Scaled scores express student performance across all levels and forms of the same test on a single scale.  In other words, the scaled scores of second graders are generally lower than those of fifth graders.  Each subject area has its own system of scaled scores.  Therefore, for example, you cannot compare scaled scores on the Reading test with scaled scores on the Math test.  However, you can use scaled scores to compare an individual student’s test performance in a single subject over time.

SOLOM—Student Oral Language Observation Matrix, an oral language assessment given to students in grades K-12 who have been identified to receive ELL services.  (Also referred to as the MN-SOLOM.)

SPPS—Saint Paul Public Schools.

SSI—Scaled Score Improvement; growth indicator used to show change in achievement from one year to the next on the MAT7 or from the MAT7 to the SAT10.

Stanine—Stanine is short for standard nine, which comes from the fact that stanine scores range from a low of 1 to a high of 9.  Stanines of 1, 2, or 3 are below the national average; 4, 5, or 6 are average; and 7, 8, or 9 are above average.  These three clusters of stanine scores are often referred to as below average, average, and above average “ranges.”

Status—Snapshot of academic achievement at a specific point in time.

Strand/Subtest/Subarea—One of a few broad content areas within a subject of a test (e.g., Reading Comprehension and Vocabulary are called strands, subtests, or subareas of Total Reading on the SAT10 and MAT7).

Subgroup—Group of students identified by a particular demographic characteristic.  Schools and districts receive an AYP status based on the following groups and disaggregations, in addition to the “all students” group:

  • Black, Non-Hispanic
  • American Indian or Alaskan Native
  • Asian or Pacific Islander
  • Hispanic
  • White, Non-Hispanic
  • Free or Reduced-Priced Lunch
  • Special Education—currently identified
  • LEP (limited English proficiency)—currently identified

Subject—Used to refer to a particular content area of a test (e.g., Reading, Math).

Swift Knowledge—A data mining tool used for posting test data that will be connected to Campus and made available online.

TEAE—Test of Emerging Academic English, given to students in grades 3-12 who have been identified to receive ELL services.

Trend—Data analyses that report performance over time.