Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Support for bundled LoRA/embedding files #3659

Open
efhosci opened this issue Jun 5, 2024 · 0 comments
Open

[Feature Request] Support for bundled LoRA/embedding files #3659

efhosci opened this issue Jun 5, 2024 · 0 comments
Labels
Feature New feature or request

Comments

@efhosci
Copy link

efhosci commented Jun 5, 2024

Onetrainer recently added an option to simultaneously train embeddings and LoRAs (I think this is called pivotal tuning?) and bundle them into a single safetensor file. I believe the format for this was actually created a while ago (see the first link). I experimented a bit with training using this method and got good results, and it seems like it can be an effective method for multi-concept LoRAs.

In A1111/web UI these work as expected, just include "lora:loraname:1 embeddingname" in the prompt. However I can't find a way to activate the embedding in a bundled file in Stable Swarm or the ComfyUI backend, only the LoRA. Separate embedding files work when loaded with embed:embeddingname but bundled files are a lot more convenient for large multi-concept LoRAs. Would support for this be something that can be implemented by ComfyUI?

Ideally, bundled embeddings should only be "accessible" when the associated LoRA is active, which is how web UI appears to work from my testing. Or there may be a better way to avoid conflicts between identically named embeddings in multiple bundle files.

Some links for reference:
Original web UI pull request for the bundle format
OneTrainer pull request
Relevant section of the OneTrainer code

@mcmonkey4eva mcmonkey4eva added the Feature New feature or request label Jun 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants