University of Illinois at Chicago
Browse
LYNN-DISSERTATION-2017.pdf (1.14 MB)

Looking Beyond Observations: Observed Scores versus Rasch Measures for the Analysis of Efficacy Data

Download (1.14 MB)
thesis
posted on 2017-10-27, 00:00 authored by Lisa Lynn
Funders of educational research devote hundreds of millions of dollars each year to support theoretically sound educational interventions and research on these interventions. Unfortunately, relatively few of these interventions have been established as clearly effective; most have been shown to be weak or ineffectual compared with normal educational practice (Coalition for Evidence-based Policy, 2013). The purpose of this study was to explore one possible reason for mixed findings in educational intervention studies: the use of observed scores in parametric statistics, where interval measures should be used. The study compared the statistical conclusions derived from observed scores and Rasch measures of several outcomes, using exemplar data from a large, grant-funded, PBL intervention study. Consistent with prior research, results showed that for an objective test, attitude measures, and a rater-scored essay, linear relationships between Rasch person measures and observed scores were very strong (r > .86). For the seven outcomes tested, Rasch person measures and observed scores consistently agreed on the statistical significance of findings. Based on the dataset and analyses used in this dissertation, there is no evidence to conclude that the use of observed/ordinal scores is a likely culprit for null or mixed findings. This is in contrast with certain earlier findings that showed at least one case wherein Rasch person measures and observed scores provided divergent statistical conclusions. Present findings suggest that, when sample size is adequately large, and when measures are already determined (i.e. Rasch analysis is not being used for scale development) and meet the general standards of reliability and model fit, there is little to be gained by using Rasch modeling to convert ordinal observed scores into interval Rasch person measures. There is no basis to recommend that researchers use Rasch modeling in this manner to improve the quality of their interpretation of efficacy or conclusion of the impact of an intervention.

History

Advisor

Lawless, Kimberly A.

Chair

Lawless, Kimberly A.

Department

Educational Psychology

Degree Grantor

University of Illinois at Chicago

Degree Level

  • Doctoral

Committee Member

Brown, Scott W. Smith, Everett Yin, Yue Yukhymenko, Mariya A.

Submitted date

May 2017

Issue date

2017-03-15

Usage metrics

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC