ilIRTEvaluations

A Subplugin for IRT via OpenCPU for the ILIAS ExtendedTestStatistics Plugin

Integrating R with ILIAS to evaluate tests (and maybe much more in the future).

Usage

Install the plugin

mkdir -p Customizing/global/plugins/Modules/Test/Evaluations
cd Customizing/global/plugins/Modules/Test/Evaluations
git clone https://github.com/kyro46/ilIRTEvaluations.git

You have to set an URL for the REST-API of an OpenCPU-server in the ExtendedTestStatistics configuration.

OpenCPU is free software based on rApache and available as apt-repository package and CRAN. Transferred data is anonymized and encryption via SSL is supported. The public instance (https://cloud.opencpu.org) can't be used because some required packages are not installed there.

Features

The available pages might be selected in the plugin administration.

Classical Test Theory (CTT)

• Internal consistency (Cronbach's Alpha/Guttman's "Lambda 3"/Kuder–Richardson Formula 20 (KR 20))
• Internal consistency without a given item and the impact on the overall consistency
• Empirical Item Characteristic Curves
• Suggested test length to reach a desired reliability (Spearman-Brown-Formula)

Factor analysis

• Scree plot
• Graph for factor loadings

Raw score analysis

• Distribution
• Skewness
• Kurtosis

Item Response Theory (IRT)

Plots and difficulty/discrimination according to various models (R-Package LTM)

• Rasch Model
• One Parameter Logistic (1PL) Model (common discrimination <> 1)
• Two Parameter Logistic (2PL) Model
• Three Parameter Logistic (3PL) Model
• Graded Response Model
• Generalized Partial Credit Model

Plots for each model

• Item Response Category Characteristic Curves
• Item Information Curves
• Test Information Curve

Focus on the Graded Response Model (R-Package MIRT)

• Difficulty and discrimination per Item
• Model fit (AIC, BIC, SABIC, HQ, Convergence)
• Item fit (Zh, signed Chi-Square)
• Person fit (Zh)
• Person ability

Plots

• Expected total score
• Test Information and Standard Errors
• Item Tracelines
• Model-fit comparison

Additional comparison of evaluations following CTT and IRT

• Difficulty correlation
• Discrimination correlation
• Sumscore vs. estimated ability correlation
• Empirical Item Characteristic Curves vs. IRT Item Tracelines
• Observable factors vs. fit of multidimensional IRT

Various

Interactive R console

• Data from the test is prepared as R-dataframe "data"
• R-commands can be executed on this data and are shown via knitr inside the browser

Dichotomization (to use dichotomous models with polytomous data). Be advised, this may yield insufficient results due to loss of information, see LINK. Selectable in plugin-config:

• 50% of reachable points for the specific question (default)
• Mean
• Median
• Modus

Credits

• Development by Christoph Jobst for etstat version 1.1.2+
• OpenCPU by Ooms, Jeroen. (2014). The OpenCPU System: Towards a Universal Interface for Scientific Computing through Separation of Concerns. LINK