Dependent Dirichlet Process Mixture Modeling of Rating Category Usage

Educational and psychological tests are often utilized to measure latent constructs, such as math achievement or self-esteem, in a sample of persons. Comparisons of subgroups within the sample are made with respect to test scores. An underlying assumption when such comparisons are made is that the item scores have the same meaning across the subgroups under comparison. That is, the person characteristics that distinguish the subgroups (e.g., gender and race) are not part of the response process. Unfortunately, this assumption does not always hold and should be tested. What makes testing this assumption more challenging when the items are of rating type (i.e., polytomously scored) is that the person characteristics could have differential effects on the response process across the scores. That is, for some subset of scores for an item, the response process could be free of such effect while for another subset of scores for the same item, such effect could be part of the response process. A differential step functioning (DSF) analysis can determine whether person characteristics are part of the response process in rating scale items, and if so, whether person characteristics have an equal or differential effect across all response categories. Item response theory (IRT) models are convenient tools for performing a DSF analysis. The traditional approach using multiple-group IRT includes the person characteristics of interest into the model. This approach, however, has its limitations. It requires some method of linking across the subgroups, with the choice having potential consequences on the effectiveness of the model detecting DSF. Additionally, it cannot account for when the subgroups are latent (i.e., latent classes). A finite-mixture IRT model can examine for whether DSF occurs across latent classes. Unfortunately, it also has its limitations. It assumes that the same number of mixture components describe the data for all items. For this dissertation, I introduce a Bayesian nonparametric IRT model, based on covariate-dependent infinite-mixture modeling, to address these limitations of multiple-group and finite-mixture IRT models. The mixing distribution for this model is formed using the multiple Dirichlet Process (mDP), which is a type of dependent Dirichlet Process, and this distribution is allowed to flexibly vary across items. Two simulation studies and analyses of two real-life rating data sets indicated that DSF across latent classes is revealed in the shape of the posterior mean estimates of the mixing distributions produced with the mDP model. When an item is free of DSF, the posterior density corresponding to each category step is unimodal with small variance. When an item has DSF, the posterior density corresponding to the category step where the DSF resides is multimodal, with the number of modes indicating the number of latent classes contributing to the DSF. The simulation studies also indicated that sample size and the magnitude of the DSF influence the effectiveness of the mDP model’s ability to detect DSF. The results of the simulation studies show that, when an appropriate sample size and DSF magnitude are present, the mDP model provides a unique approach to identifying where the DSF resides in rating scale items. The DSF is displayed visually through the posterior densities of the mixing distributions, and the mDP model accomplishes this while addressing the limitations of the traditional IRT approaches to DSF analysis.