21 Exploring Differential Item Functioning with Dexter

Differential item functioning (typically abbreviated as DIF) is present when the probability of success on an item differs for two groups of people, even when we control for ability measures. For example, in mathematics, girls have been found to outperform boys on number items, while boys outperform girls on spatial items, even when the average abilities for boys and girls are the same.

rm(list=ls())
library(tidyverse)
library(dexter)
# load in the dataset
responses <- read_csv('data/responses.csv')
responses <- responses %>% drop_na(gender)
keys <- read_csv('data/key.csv')
# Create the rules
rules <- keys_to_rules(keys, include_NA_rule = TRUE)
db <- start_new_project(rules, db_name = ":memory:", person_properties=list(gender=""))

# Add item properties
properties <- read_csv('data/properties.csv')
# reduce to two properties
properties <- properties %>% mutate(
    main_test = case_when(
        subtest == 'subject-verb-object' ~ 'long',
        subtest == 'subject-verb' ~ 'short',
        subtest == 'conjunction' ~ 'long'
    )
)
# Add item properties
add_item_properties(db, item_properties = properties, default_values = NULL)
add_booklet(db, responses, "y7") 
get_booklets(db)

21.1 Is there DIF?

dif_gender = DIF(db, "gender")
dif_gender

Test for DIF: Chi-square = 67.498, df = 19, p = < 0.0006

21.2 Which items are affected?

    plot(dif_gender)

21.3 Is the DIF meaningful?

Read through this article and consider why the authors believe their approach to DIF is superior. https://dexter-psychometrics.github.io/dexter/articles/profile-plots.html

21.4 Do boys struggle more with run-on sentences than girls?

Does the fact that girls score higher on run-on sentences than boys prove DIF?

responses %>% mutate(
    run_on = (Q8_score + Q9_score + Q10_score + Q11_score)
) %>% select(gender, run_on) %>% group_by(gender) %>% summarise(n=n(),mean=mean(run_on,na.rm=T))

# A tibble: 2 × 3
  gender     n  mean
  <chr>  <int> <dbl>
1 F       1512  2.63
2 M       1527  2.44

profile_plot(db, item_property='run-on', covariate='gender')

close_project(db)

21.5 More on DIF!

There are a huge number of methods for testing for DIF. Have a look through https://cran.r-project.org/web/packages/difR/difR.pdf