Replies: 8 comments 38 replies
-
I think it depends on the approach. I don't know of any popular "small" language models (<7B) that support multiple languages right now. We would need to have a new foundation model trained from the ground-up in the new language. After that you would need to translate this dataset into that new language as well in order for the fine-tuning to be effective. With a much larger model, it should have multiple languages in it's base training data and could potentially "transfer" the fine-tuning from English examples to another language. I believe Mixtral-8x7b has shown some particularly good capabilities with performing tasks in multiple languages as well as English, but that is a huge model that is still fairly difficult to run on consumer hardware (reasonable quantizations are still 32GB of VRAM) |
Beta Was this translation helpful? Give feedback.
-
Wanted to add an update on this: As I have done more digging, it looks like there might be some resources to put together a multi-language version of the model. There is a translation of the alpaca dataset that I used into 132 languages: https://huggingface.co/datasets/saillab/taco-datasets This should help with the fine-tuning step. The biggest hurdle would be to translate this projects dataset into other languages. Not sure if I'm a fan of using machine translation on it since I don't have the ability to rate the accuracy of its output whatsoever. |
Beta Was this translation helpful? Give feedback.
-
After doing some quick experimenting with
To me, this is promising, and I think having proper fine-tuning examples in the desired language should significantly boost the model's performance up closer to the English performance will be. The next step is to translate the fine-tuning dataset to other languages. If you want to have this project usable in your language, please take a look at: Adding a column to these CSV files for your language would be very helpful so we don't need to use machine translation which will inevitably lead to lower quality data. |
Beta Was this translation helpful? Give feedback.
-
I have fine-tuned a version of StableLM-Zephyr-3B and it has surprisingly good multi-language understanding given the model is not trained on non-english data. It is available here: https://huggingface.co/acon96/Home-3B-v3-GGUF I was able to get it to respond to English, German, Spanish, and French |
Beta Was this translation helpful? Give feedback.
-
Hello, you've worked hard. How is the Chinese performance of this model? |
Beta Was this translation helpful? Give feedback.
-
The english version is awesome, nice work! Any updates on implementing the Dutch language? Would love to speak to the AI in Dutch. |
Beta Was this translation helpful? Give feedback.
-
Hi, I see this topic was dead for couple of months now. I see that there are currently translations to three languages other than English. I would very much like to help in translation to Polish. This week I will try to complete translation of the two data files to polish. Just let me know what would be the next steps once that's done. Regards |
Beta Was this translation helpful? Give feedback.
-
An update for those following this thread: I've uploaded a model I trained a little while ago but never got around to fully finalizing. It is a full fine-tuning of StableLM Zephyr 3B (instead of just a LoRA) using 3 languages (German, French, & Spanish) in addition to English. It is available here: https://huggingface.co/acon96/stablehome-multilingual-experimental Any feedback on how the model performs in languages other than English would be appreciated. I want to do another training run (it involves cloud compute instead of my workstation) but want to iterate on the translated dataset a bit before attempting to do that. Make sure you install the |
Beta Was this translation helpful? Give feedback.
-
Hi, what would be the way forward on multilanguage support? Just another basemodel or is this more complicated?
Beta Was this translation helpful? Give feedback.
All reactions