12  Exploring alpha

Internal consistency is a measure of the strength of the relationship between questions in the same scale. There are many different measures of internal consistency, with the most common cited being Cronbach’s alpha. However, there are many other measures of internal consistency, including omega, glb, and others.

12.1 From alpha to omega

Read the following article, and note the three main assumptions of alpha. https://tidsskriftet.no/en/2022/08/medicine-and-numbers/internal-consistency-alpha-omega

12.2 Computing variations on alpha

rm(list=ls())
library(tidyverse)
library(psych)
# load in the dataset
responses <- read_csv('data/responses.csv')
#| label: sumscores
# take all the item scores
item_responses <- responses %>% 
  select(ends_with("_score"))

mdl_alpha <- psych::alpha(item_responses)
print(summary(mdl_alpha))

Reliability analysis   
 raw_alpha std.alpha G6(smc) average_r S/N    ase mean  sd median_r
      0.81      0.81    0.83      0.18 4.3 0.0048  0.6 0.2     0.16
NULL

mdl_omega <- psych::omega(item_responses)

print(summary(mdl_omega))

Omega 
psych::omega(m = item_responses)
Alpha:                 0.81 
G.6:                   0.83 
Omega Hierarchical:    0.59 
Omega H asymptotic:    0.7 
Omega Total            0.84 

With eigenvalues of:
  g F1* F2* F3* 
3.0 1.6 1.3 0.0 
The degrees of freedom for the model is 133  and the fit was  0.24 
The number of observations was  3061  with Chi Square =  726.19  with prob <  0 

The root mean square of the residuals is  0.03 
The df corrected root mean square of the residuals is  0.04 

RMSEA and the  0.9 confidence intervals are  0.038 0.035 0.041
BIC =  -341.34Explained Common Variance of the general factor =  0.5 

 Total, General and Subset omega for each subset
                                                 g  F1*  F2* F3*
Omega total for total scores and subscales    0.84 0.81 0.70  NA
Omega general for total scores and subscales  0.59 0.47 0.36  NA
Omega group for total scores and subscales    0.24 0.33 0.34  NA
NULL

12.3 Extension exercise

The value of alpha is closely related to the length of the test and the number of items. In general, the longer the test, the higher the alpha. Using the test dataset explore the relationship between alpha and the length of the test.

Andrich, David, and Ida Marais. 2019. “The Idea of Measurement.” In A Course in Rasch Measurement Theory, 3–11. Springer Nature Singapore. https://doi.org/10.1007/978-981-13-7496-8_1.

Bond, Trevor G., Zi Yan, and Moritz Heene. 2020. Applying the Rasch Model: Fundamental Measurement in the Human Sciences. Routledge. https://doi.org/10.4324/9780429030499.

Cappelleri, Jason Lundy, J. C. 2014. “Overview of Classical Test Theory and Item Response Theory for the Quantitative Assessment of Items in Developing Patient-Reported Outcomes Measures.” Clinical Therapeutics 36 (5): 648–62. https://doi.org/https://doi.org/10.1016/j.clinthera.2014.04.006.

Cronbach, Lee J. 1951. “Coefficient Alpha and the Internal Structure of Tests.” Psychometrika 16 (3): 297–334. https://doi.org/10.1007/bf02310555.

Goldstein, H., and Steve Blinkhorn. 1982. “The Rasch Model Still Does Not Fit.” British Educational Research Journal 8 (2): 167–70. https://doi.org/10.1080/0141192820080207.

Lord, Frederic M, and Melvin R Novick. 2008. Statistical Theories of Mental Test Scores. IAP.

McNeish, Daniel. 2018. “Thanks Coefficient Alpha, We’ll Take It from Here.” Psychological Methods 23 (3): 412–33. https://doi.org/10.1037/met0000144.

Panayides, Panayiotis, Colin Robinson, and Peter Tymms. 2010. “The Assessment Revolution That Has Passed England by: Rasch Measurement.” British Educational Research Journal 36 (4): 611–26. https://doi.org/10.1080/01411920903018182.

Robitzsch, Alexander, Thomas Kiefer, and Margaret Wu. 2022. TAM: Test Analysis Modules. https://CRAN.R-project.org/package=TAM.

Sijtsma, Klaas. 2008. “On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha.” Psychometrika 74 (1): 107–20. https://doi.org/10.1007/s11336-008-9101-0.

Wright, Benjamin D, and Magdalena Mok. 2000. “Rasch Models Overview.” Journal of Applied Measurement 1 (1): 83–106.