An automatic essay scoring method for Chinese L2 writing based on linear models, tree-structure models and logistic regression models, especially Ordinal Logistic Regression(OLR), combining 90 linguistic features and Tf-Idf textual representations.
An online demo of L2C-rater can be seen at https://l2c.shenshen.wiki/.
- Python 3.6+
- numpy 1.19.5
- pandas 1.1.5
- scipy 1.5.2
- sci-kit-learn 0.24.1
- pyltp 0.1.9.1
- xgboost 1.3.3
- mord 0.6
- Configure the environment
- Prepare the data
- Run main.py
We conduct 5-fold cross validation on HSK Dynamic Composition Corpus 2.0 to evaluate our system. Currently, the code can be run with the provided samples.
You can see the list of available options by running:
python main.py -hFor feature_mode, t means using text representations only when training the model, l means using linguistics features only, and b means using both the above features.
For feature_type, c means using character level features when generating text representations, w means the word level, cw means using both c and w, wp means using word and part-of-speech(POS) features, and cwp means using both c and wp.
The following command trains the best model among all models. Note that you will not get a convincing result with only the samples.
python main.py -m b -t wpPlease cite our work when using the codes:
Wang Y., Hu R. (2021) A Prompt-Independent and Interpretable Automated Essay Scoring Method for Chinese Second Language Writing. In: Li S. et al. (eds) Chinese Computational Linguistics. CCL 2021. Lecture Notes in Computer Science, vol 12869. Springer, Cham. https://doi.org/10.1007/978-3-030-84186-7_30
@inproceedings{wang2021prompt,
title={A Prompt-Independent and Interpretable Automated Essay Scoring Method for Chinese Second Language Writing},
author={Wang, Yupei and Hu, Renfen},
booktitle={China National Conference on Chinese Computational Linguistics},
pages={450--470},
year={2021},
organization={Springer}
}
More modules of L2C-rater coming soon!