How to train a model similar to this one?

[leon-rgb]

Hi everyone,
I'm currently doing my master's thesis in Software Enginnering. My topic is similar to what this repository does. I should develop a chatbot that is able to perform actions and answer questions in a given Smart Home environment (not home assistant).

I am curious how to do this with an LLM. Are there some resources you can share to learn about the process? I know a bit about cusotmizing models like adding a System Message or adjusting to temperature of the model and I also trained an LLM to generate software requiremts before. But I don't know how to train a model to perform an action like turning a smart device on or off.

Maybe you could even provide me with an explanation how the models for home assistant were trained exactly.

Thanks in advance!
Leon

[acon96]

Are you asking more about the theory about how Supervised Fine Tuning works or are you just looking for how to run the scripts to reproduce the model?

[leon-rgb]

Both actually. I mean I really want to understand / learn how everything works and at the same time apply what you did to the actions my model need to be capable of.

[acon96]

Sorry, been busy and haven't had the time to respond to this properly.

First, building a model like this is actually more about building the dataset. The dataset represents what the model is capable of. The largest part of this project has been building this dataset that contains thousands of examples of different scenarios of a person asking a personal assistant to perform an action in Home Assistant. Nothing suitable existed beforehand so I generated a bunch of pieces of an example and then wrote a script to put those pieces together to produce many different examples. The examples in this dataset are formatted in a specific way that includes all of the information that the model might need in order to respond and perform the action that we requested. The idea is that it will piece together the different parts of the context and produce the correct answer when given a scenario that it hasn't seen before. An example looks something like this (the chat/instruct format will be different depending on the model you want to fine tune):

<|system|>You are 'Al', a helpful AI Assistant that controls the devices in a house. Complete the following task as instructed or answer the following question with the information provided only.
The current time and date is 08:12 AM on Thursday March 14, 2024
Services: light.turn_off(), light.turn_on(rgb_color,brightness), fan.turn_on(), fan.turn_off()
Devices:
light.office 'Office Light' = on;80%
fan.office 'Office fan' = off
light.kitchen 'Kitchen Light' = on;80%;red
light.bedroom 'Bedroom Light' = off<|endoftext|>
<|user|>
please turn on the lights in the kitchen now<|endoftext|>
<|assistant|>I'll turn the lights on for you right way
```homeassistant
{ "service": "light.turn_on", "target_device": "light.kitchen" }
```<|endoftext|>

By providing enough different variations of this scenario to the model during training, it is able to learn to complete the users request by picking the name of the device in the context as well as the service that it should call based on what the user is saying. After training, when you present the model with only the first half of the problem (i.e. the user request) and ask the model to complete the sequence, then it should produce a quick text response and then the JSON object inside of the markdown style code block. The integration works by using the model to produce the output, and then find those two parts in the response, and then returns the text to the user while executing the service in Home Assistant.

When it comes to training the model, I am just doing normal instruct fine-tuning using the HuggingFace Trainer scripts. Make sure you match any instruct or chat format that the model uses and mask out the context so you aren't training the model to reproduce the system prompt; only the assistant response. A good place to start out is making sure you have the correct learning rate, training schedule, as well as other hyper-parameters. To do this, it is very useful to have an evaluation framework that can determine the model's accuracy and let you see more than just training loss so that you can have a real benchmark to compare your training runs against each other.