tag:blogger.com,1999:blog-121208302010-03-10T14:01:55.195-06:00Pointers and perspectives on psychometrics and test and measurement practices.R. Brophyhttp://www.blogger.com/profile/09372022262864253901noreply@blogger.comBlogger75125tag:blogger.com,1999:blog-12120830.post-85111126449357095302010-03-10T13:58:00.001-06:002010-03-10T14:00:01.855-06:00

I spend a lot of time thinking about ratings. One reason I spend so much time thinking about ratings is that I’ve either assigned or been subjected to ratings many times during my life. For example, I review numerous research proposals and journal manuscripts each year, and I assign ratings that help determine whether the proposed project is funded or manuscript is published. I have entered

Pearson's Research Teamhttp://www.blogger.com/profile/02697831921566125491noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-42222760140275024472010-03-03T08:09:00.001-06:002010-03-10T08:27:26.181-06:00

An ATP Newbie Reflects…I walked into the annual meeting of the Association of Test Publishers (ATP) opening shindig (appropriately Superbowl-themed on 2/7/10 – congrats Saints!) and was struck by déjà vu. I eerily felt the same trepidation and bemusement as at my first educational conference back in 2000. Despite many years in assessment, I knew very few people. It was only later that I

Pearson's Research Teamhttp://www.blogger.com/profile/02697831921566125491noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-81654515189402517162009-06-29T10:00:00.000-05:002009-07-21T09:56:56.765-05:00

One of primary objectives of Pearson as the leading provider of educational measurement research is to lead the effort on effective educational policy discussion. Sometimes these efforts are clearly articulated in customer facing actions (like legally defensible setting of student performance standards), academic research publications or conference presentations. Other times, policy and/or

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-19643142452292208432009-06-22T11:45:00.000-05:002009-06-22T11:45:01.770-05:00

This was just posted on the Pearson website. Pearson’s Universal Design for Computer-Based Testing Guidelines examines the specific student challenges related to each test question construct and pinpoints question design solutions that can make test questions more accessible to all students. The study touts the value of digital technology and its ability to incorporate multiple representations,

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-10674395456652575172009-06-04T11:00:00.000-05:002009-06-18T12:59:15.461-05:00

Assessing Writing Online: The Benefits and ChallengesThe transition to assessing student writing online presents both benefits and challenges to states and their students. This session will discuss the logistical, content, scoring, and political issues states face while implementing the transition to assessing student writing online. Louisiana will provide insight on implementing an online

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-34496251283918861532009-05-15T10:00:00.000-05:002009-07-21T09:53:16.180-05:00

So I admit it—I, too, like a good, sensationalized feel-good piece of literature, one that particularly has some application to what we do for a living. My very good but departed friend, Ed Slawski, use to say I was such a soft touch.Regardless, I recently read a piece by that world famous author Malcolm Gladwell (you know the Tippling Point, Blink, and Outliers guy) that did indeed move me to

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-83090148816669069152009-04-15T11:47:00.000-05:002009-04-15T15:09:21.870-05:00

Pearson once again dominates presentations (well, we have a lot anyway) at the annual meetings of the National Council on Measurement in Education (NCME) and the American Educational Research Association (AERA).NCMEUm, K., Way, W. D., Fitzpatrick, S. J., & Kreiman, C.The Effects of Response Probability Criteria on the Scale Location Estimation and Impact Data in Standard Setting.Wan, L., & Henly,

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-68733779480530032402009-03-09T11:50:00.000-05:002009-03-25T14:42:20.994-05:00

As the national debate about what to do with education reform rages, I hear repeatedly the need to have national standards, common core standards, or at the very least, a more standardized system of national guidelines in order for us to measure and, presumably then, improve education in America. Often this discussion leads to debate about the merits of a national test—which is often assumed to

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-60456362825488670142008-11-03T13:00:00.000-06:002008-11-03T13:00:00.959-06:00

This is the "full text" of a contribution I made to the NCME newsletter. I thought you might like to get the full inside story!Mark Reckase’s call for NCME members to become more involved in educational policy is timely and relevant, while perhaps also a little misleading. For example, some of my colleagues and I have been working with states, local schools, and the USDOE regarding implementing

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-50180330814340219122008-07-09T11:30:00.000-05:002008-07-09T11:30:01.520-05:00

I gave up reading editorials quite a long time ago. Not because they are too often misleading or inaccurate (many of them are), or that they are too often purposely written to be controversial and sensational (again, many of them are). Rather, I quit reading because the whole purpose of editorials seems rather futile to me.Let me explain. People who write editorials usually have a strong position

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-8368510933350740242008-06-04T15:00:00.000-05:002008-06-04T15:00:00.446-05:00

There are some things in educational measurement that are not debated. Foremost, the purpose of instruction is to improve learning. The purpose of assessment is to improve instruction, which in turn improves learning. In other words, it’s all about the learning—debate over.Some researchers (myself included) have become sloppy with our language, labeling assessments "for learning" to be formative

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-53965121627230111782008-04-01T11:00:00.000-05:002008-04-01T08:54:56.566-05:00

Dr. Len Swanson from ETS was the most recent keynote speaker during the Pearson Visiting Scholars program in Iowa City. Dr. Swanson talked about Computer Adaptive Testing (CAT) and the history of how we got to where we are. Dr. Swanson was a particularly good choice for this presentation as he has worked in CAT since its inception and was "on the floor" when most of the ground work was laid for

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-38589872627288218172008-03-19T16:35:00.000-05:002008-03-19T16:55:28.804-05:00

Sometimes I forget how big Pearson really is. Here are additional presentations at both the AERA and NCME national conventions.NCME Papers and presentationsChu, Kwang-lee, & Lin, Serena JieDistracter Rationale Taxonomy: A Formative Evaluation Utilizing Multiple-Choice DistractersJirka, StephenTest Accommodations and Item-Level Analyses: Mixture DIF Models to Establish Valid Test Score

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-2256481541608066002008-03-18T12:30:00.000-05:002008-03-18T10:29:58.727-05:00

The contingent of Pearson researchers has, once again, done an admirable job of representing our industry at the annual meeting of the American Educational Research Association (AERA) and the National Council on Measurement in Education (NCME) the week of March 24th in New York City.The following are the AERA paper and symposium submissions:Jason Meyers & Xiaojin KongAn Investigation of the

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-50338176415025845742008-02-21T12:00:00.000-06:002008-02-21T10:01:41.196-06:00

In the olden day, Raschites and 3PL researchers fought with so much vigor that they separated ways. I recall one AERA/NCME conference with Ron Hambleton on the right, Ben Wright on the left, and nothing but a "DMZ" in between. Well, times have changed, and more moderate heads have prevailed. Hence, those of you attending the AERA/NCME national conference in New York City should consider coming

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-61791298514317166852008-02-11T15:00:00.000-06:002008-02-11T12:52:01.190-06:00

The Pearson psychometric and research services team will be presenting a workshop at the annual conference for the Association of Test Publishers (ATP) held in Dallas, Monday, March 3, 2008. The title for the workshp is "Setting Performance Standards on High Stakes Tests."Pearson has arguably more experience setting performance standards under NCLB than anyone. Most of this research is not

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-1261511472530026122008-02-04T17:30:00.000-06:002008-02-04T15:38:08.968-06:00

Well, the long anticipated integration between Pearson and former test publishing giant Harcourt Assessment, Inc. (also known at times in it's history as The Psychological Corporation and the testing division of Harcourt, Brace and Jovonovich) is complete! Senior Pearson leadership were in San Anotonio this week to meet with the new Pearson employees and to celebrate the lengthy process of DOJ

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-47338449574895227322008-01-28T11:30:00.000-06:002008-01-28T09:31:17.917-06:00

I have had the occasion to know Dr. James Popham for many years and in many contexts. You might recall that Dr. Popham was the keynote speaker at the last ACT-CASMA conference and that I had devoted some space to the conference in a previous post.Now please understand, Dr. Popham has worked in measurement for many years and describes himself as a "reformed test builder," presumably implying some

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-23092408047287526442008-01-18T15:30:00.000-06:002008-01-18T13:25:14.112-06:00

Back in the 1980s when I worked on the development of the Wechsler Intelligence Scale for Children, Third Edition (WISC-III), I was fascinated with a process commonly referred to at the time as "continuous norming." Applied by Dr. Gale Roid as developed by Professor Richard Gorsuch, continuous norming was a slick way to improve the precision of empirical norms. While things seemed to get in the

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-67177641878272245122007-10-23T12:00:00.000-05:002007-10-23T09:56:29.048-05:00

Living in Iowa City, arguably the mecca of achievement testing, I am privileged and fortunate to be able to attend professional development activities that many of my colleagues located elsewhere can not. Take for instance the 2007 William E. Coffman Lecture Series sponsored by the University of Iowa and the Iowa Testing Program. This year the guest lecture was Dr. Daniel Koretz, Harvard

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-17774259648617329372007-10-08T12:30:00.000-05:002007-10-08T13:32:53.217-05:00

The CASMA-ACT Invitational Conference on Current Challenges in Educational Testing seems to offer a very interesting slate of speakers and topics, despite it's rather academic name.Dr. Dan Koretz, from Harvard and a national assessment expert, will speak about higher education accountability.Joe Crick, from the National Board of Medical Examiners, has been solving assessment problems almost as

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-47615994842098994122007-09-24T11:30:00.000-05:002007-09-24T09:18:27.022-05:00

While the presidential election may seem years away, it is right around the corner here in Iowa as evidenced by the many political pundits vying for sound bytes. An outsider may well consider this a time of confusion and turmoil prior to a more focused direction once the party candidates are chosen. This is also true on the education front. The reauthorization of NCLB is yet to be determined.

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-33319388912806211722007-09-11T10:00:00.000-05:002007-09-11T09:26:34.656-05:00

In a previous newsletter to NCME, Dr. Michael Kirst argued that our post-secondary and secondary education systems are disconnected. He argued, that:"A big issue is the proliferation of tests in grades 9 through 11 caused by the combination of post-secondary admissions assessments, and the new statewide tests created by the K–12 standards movement."Dr. Kirst also suggested that there are very few

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-89307078277701299572007-08-31T10:00:00.000-05:002007-08-31T13:46:33.129-05:00

A reminder to let everyone know that our annual conference for the Iowa Educational Research and Evaluation Association (IEREA) is fast approaching and will be upon us before we know it. One of the popular features of the conference is our poster presentation and our paper contest. We need lots of poster and paper submissions to make this part of our conference a success. This conference is also

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.comtag:blogger.com,1999:blog-12120830.post-69112798439176428242007-08-28T12:30:00.000-05:002007-08-28T10:36:44.691-05:00

While most people argue that you have to earn the respect you are given, this is not always the case. Take for example the hard working, informative, creative and open-ended test item type commonly known as the "griddable item." This item type gets no respect. In fact, my guess is you don't even know what I mean when I refer to a gribbable item. Let me elaborate.When criterion-referenced and

Jon S. Twinghttp://www.blogger.com/profile/09761278152108450542noreply@blogger.com