The Next Revolution in Item Format

The last revolution in the format of how Americans test students may have been during and just after World War I. That’s when the multiple-choice format replaced the essay format as the most prevalent item type in educational testing. In the decades that followed, the techniques around multiple-choice testing were developed and refined.

Let me put on my futurist hat for a moment and predict that the next revolution in the format of how Americans test students will happen when the computer-delivered simulation replaces the multiple-choice format as the most prevalent item type in educational testing. I’m not going too far out on a limb because the computer-delivered simulation is already well established and widely available on the web. For example, the Rice Virtual Lab in Statistics offers a set of simulations to demonstrate various statistical concepts. Or visit the business simulations offered by Forio Business Simulations. Some nice simulations in biology are offered at Biology Labs Online, by Benjamin Cummings, an imprint of Pearson Education.

Simulations can be used as items in at least two ways. First, simulations could be used like reading passages are currently used, as the stimulus preceding multiple-choice, short- and extended-response questions. Second, simulations can be used as the test “items.” Under this use of simulations, the term item is interpreted broadly to be a situation contrived to produce student behaviors or performance that are revealing of the constructs we are trying to assess. A conventional multiple-choice item is written to produce a student marking a bubble on an answer sheet in such a way as to reveal something about the student’s understanding. A simulation would be constructed to produce a student interacting with the software in such a way as to reveal something about their declarative and procedural knowledge.


To me, the assessment possibilities are in the use of simulations as the test “items.” Using simulations similar to those found in the Rice Virtual Lab in Statistics, I could ask students to create a distribution of cases and demonstrate the influence of outliers on the mean. By capturing mouse clicks, I can collect information about students’ understandings of the concepts of distributions and means. Alternatively, CardioLab, part of Biology Labs Online, allows students to measure arterial pressure and to manipulate five variables that effect arterial pressure. Vessel radius and heart rate are two of these variables. A test question would be: “Take as many measures as you need of arterial pressure under conditions that demonstrate the interaction of vessel radius and heart rate.” I can capture mouse movements to determine the level of understanding of the concepts of variable interaction and experimental control.

Even with my futurist hat on, I can’t foresee the psychometrics that will develop and be refined around simulations. Some techniques are under development, but that's a topic for another entry.