Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Text encoder, unet control don't working? #24

Open
MetroByte opened this issue Oct 3, 2023 · 6 comments
Open

Text encoder, unet control don't working? #24

MetroByte opened this issue Oct 3, 2023 · 6 comments

Comments

@MetroByte
Copy link

MetroByte commented Oct 3, 2023

<lora:name:te=0@0,1@1:unet=1> (the same as mentioned in readme.md with pixelart robot) gives the same result as <lora:name:te=0:unet=1>

Unet progression seems to be working as intended. However, simple progression like <lora:name:0@0,1@1> works fine.

I can't figure out what is the problem. I have extension for LoRA's (LoRA Block Weight) but that is disabled.

Latest version of SD-webui and this extension (at the moment of issue creating)

@cheald
Copy link
Owner

cheald commented Oct 3, 2023

I'm unable to replicate this. Separate te/unet control seems to be working as expected here:

<lora:pixel:te=0@0,1@1:unet=1@0,0@1>

image

<lora:pixel:te=0@0,1@1:unet=1>

image

The "simple progression" is just setting that schedule on both the te and unet together:

image

When you capture plots of the weights (with the checkbox in the extension options), do you get a plot that reflects what you expect? It's possible that you're working with a Lora that's heavily dependent on the text encoder, so starting at 0 weight is basically starting it outside of "lora space" and it has trouble getting back into it. Perhaps try starting the TE at some value larger than 0.

@MetroByte
Copy link
Author

MetroByte commented Oct 3, 2023

Plotting works fine. I've even added prints to network_patch.py functions to check if calculation works. And everything works as intended, I guess. At least calculation.
And once again, I've done example setting right from readme text, last example (only with different LoRA). It is no difference what to put after , in my case. It seems like it is taking the first te number and keeps it?
Maybe another extension conflicts with this one
xyz_grid-0002-123456

@cheald
Copy link
Owner

cheald commented Oct 3, 2023

I think there might be something funny happening with the webui itself. After a totally fresh boot of the app, I get:

image

But then when I run a second generation without changing anything:

image

That's unexpected, to say the least. It might be that some weights might not be getting unwound properly, or something. I'll dig around a bit.

@MetroByte
Copy link
Author

; semicolon?

@cheald
Copy link
Owner

cheald commented Oct 3, 2023

Commas and semicolons are both valid delimiters. I just use them because I always forget to quote the individual chunks in the S/R matrix.

@MetroByte
Copy link
Author

MetroByte commented Oct 3, 2023

Okay, I got it.
I disabled another extensions and re-launched SD-webui. No changes even after fresh re-launch, same results.

Unet progression works.
xyz_grid-0007-123456

another edit:
Actually, after another fresh restart, te progression is working. And just once. Then stops working. Really odd.
Also, after this, images generated with te=1:unet=1@0,0@2 are equal to 1@0,0@2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants