substring-language-detection

Substring language detection

Only available ['MS', 'EN', 'OTHERS', 'CAPITAL', 'NOT_LANG'].

download

1. https://huggingface.co/datasets/mesolitica/substring-language-detection/resolve/main/en-substrings.json
2. https://huggingface.co/datasets/mesolitica/substring-language-detection/resolve/main/ms-substrings.json
3. https://huggingface.co/datasets/mesolitica/substring-language-detection/resolve/main/en-ms-substrings.json
4. https://huggingface.co/datasets/mesolitica/substring-language-detection/resolve/main/en-ms-substrings-v2.json
5. https://huggingface.co/datasets/mesolitica/substring-language-detection/resolve/main/ms-en-substrings.json
6. https://huggingface.co/datasets/mesolitica/substring-language-detection/resolve/main/ms-en-substrings-v2.json

Citation

@misc{Malay-Dataset, We gather Bahasa Malaysia corpus!, Substring language detection,
  author = {Husein, Zolkepli},
  title = {Malay-Dataset},
  year = {2018},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/huseinzol05/malay-dataset/tree/master/corpus/substring-language-detection}}
}