An automatic essay scoring method for Chinese L2 writing based on linear models, tree-structure models and logistic regression models, especially Ordinal Logistic Regression(OLR), combining 90 linguistic features and Tf-Idf textual representations.
Python 3.6+
numpy 1.19.5 pandas 1.1.5 scipy 1.5.2 sci-kit-learn 0.24.1 pyltp 0.1.9.1 jieba xgboost mord
- Configure the environment
- Prepare data
- Run main.py
We conduct 5-fold cross validation on HSK Dynamic Composition Corpus to evaluate our system. This dataset (this code can be run with part of the data in here) is publicly available here.
You can see the list of available options by running:
python main.py -h
For feature_mode
, t
means using text representation only when training the model, l
means using linguistics features only, and b
means using both the above features.
For feature_type
, c
means segmenting the essay to character level when generating text representation, w
means segmenting the essay to word level, cw
means using both the above, wp
means segmenting the essay to word level and tagging their part-of-speech(POS), and cwp
means using both c
and wp
.
The following command trains the best model among all automated models. Note that you will not get a convincing result with data provided here, because the amount of data is not large enough.
python main.py -m b -t wp
Wang Y., Hu R. (2021) A Prompt-Independent and Interpretable Automated Essay Scoring Method for Chinese Second Language Writing. In: Li S. et al. (eds) Chinese Computational Linguistics. CCL 2021. Lecture Notes in Computer Science, vol 12869. Springer, Cham. https://doi.org/10.1007/978-3-030-84186-7_30