L2C-rater (Automated Essay Scoring Method for Chinese Second Language Writing)

An automatic essay scoring method for Chinese L2 writing based on linear models, tree-structure models and logistic regression models, especially Ordinal Logistic Regression(OLR), combining 90 linguistic features and Tf-Idf textual representations.

Enviroment

Python 3.6+

Main Package

numpy 1.19.5 pandas 1.1.5 scipy 1.5.2 sci-kit-learn 0.24.1 pyltp 0.1.9.1 jieba xgboost mord

Set Up

• Configure the environment
• Prepare data
• Run main.py

Dataset

We conduct 5-fold cross validation on HSK Dynamic Composition Corpus to evaluate our system. This dataset (this code can be run with part of the data in here) is publicly available here.

Options

You can see the list of available options by running:

python main.py -h

For feature_mode, t means using text representation only when training the model, l means using linguistics features only, and b means using both the above features.

For feature_type, c means segmenting the essay to character level when generating text representation, w means segmenting the essay to word level, cw means using both the above, wp means segmenting the essay to word level and tagging their part-of-speech(POS), and cwp means using both c and wp.

Example

The following command trains the best model among all automated models. Note that you will not get a convincing result with data provided here, because the amount of data is not large enough.

python main.py -m b -t wp

Publication

Wang Y., Hu R. (2021) A Prompt-Independent and Interpretable Automated Essay Scoring Method for Chinese Second Language Writing. In: Li S. et al. (eds) Chinese Computational Linguistics. CCL 2021. Lecture Notes in Computer Science, vol 12869. Springer, Cham. https://doi.org/10.1007/978-3-030-84186-7_30