DISPERSION PREDICTION MODELS IN NESTED DATA STRUCTURES: EXAMINING THE PERFORMANCE OF DISPERSION INDEXES IN POLYTOMOUS ITEMS

[摘要] This study was designed to explore two possible causes of and solutions to poor dispersion prediction model performance in polytomous items. First, the impact of the correlation between the 'level’ (e.g., the average score of the distribution) and the 'strength’ (the dispersion among the data points in a distribution) on the dispersion effect was explored. Second, the extent to which non-linearity and heteroscedasticity influenced the dispersion effect was also explored. In order to explore these two factors, Monte Carlo studies were performed in which the dispersion index, the number of aggregated observations, the number of nested data points, the number of items from which the dispersion index was derived, the shape of the distribution, and the 'level’ covariate in the multiple regression model were varied. The studies used a 5 point response polytomous item context. The evaluation criteria included power/Type I error rates, model R2, sr2 for the dispersion index, the VIF of the dispersion index, linearity of the dispersion index, and homoscedasticity of the errors in the dispersion prediction model. The results suggest that none of the dispersion indexes systematically violate the multiple regression assumptions of linearity or homoscedasticity. They also suggest that the choice of the dispersion index, the number of items used, and the central tendency covariate used in the dispersion prediction model are the prominent determinants of good performance in a 5 point response scale polytomous item context. The sample standard deviation (SD) and average deviation indexes (ADm and ADmd) performed equally well and substantially better than the MAD, CV, and awg in terms of the evaluation criteria across the conditions of the study. The performance of the SD, ADm, and ADmd improved substantially when computed from 5 different polytomous items as opposed to a single polytomous item. Finally, the results suggest that in skewed distributions the performance of the SD, ADm, and ADmd decreases due to an increased correlation with the level covariate. This decrease in performance can be counteracted in skewed distribution by controlling for the median as the level covariate as opposed to the mean.

[发布机构] the University of Pittsburgh