[Kolors] Add PAG

[asomoza]

What does this PR do?

Adds PAG to kolors

Note: Kolors with PAG behaves different than with SDXL, I had to use a lower scale and different layers to get good results.

How to test

Txt2Img

import torch

from diffusers import AutoPipelineForText2Image, DPMSolverMultistepScheduler


pipe = AutoPipelineForText2Image.from_pretrained(
    "Kwai-Kolors/Kolors-diffusers",
    torch_dtype=torch.float16,
    variant="fp16",
    enable_pag=True,
    pag_applied_layers=["down.blocks.2.attentions.1", "up.blocks.0.attentions.1"],
).to("cuda")
pipe.scheduler = DPMSolverMultistepScheduler.from_config(pipe.scheduler.config, use_karras_sigmas=True)

prompt = "cinematic film still of a pirate ship is navigating the milk inside of a steaming cup of coffee. The ship’s wooden hull glistens with droplets of coffee, and tiny pirates can be seen on deck, adjusting sails and peering through miniature telescopes. The tilt-shift effect adds a magical touch, high budget hollywood movie, cinemascope, epic, film grain"

image = pipe(
    prompt=prompt,
    negative_prompt="",
    guidance_scale=5.5,
    num_inference_steps=25,
    pag_scale=1.5,
).images[0]

W/O	PAG

After the PAG refactor, the image changed a bit, not sure why:

before	after

Who can review?

@yiyixuxu @a-r-r-o-w @sunovivid @sayakpaul

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[yiyixuxu]

thanks!
add a test?

[asomoza]

Added the fast tests and I fixed the old ones because I needed to get the original Kolors Pipeline to test the AutoPipeline. In short, I had to add all the missing tests I had as TODO for kolors.

@yiyixuxu @sayakpaul should I add the slow tests for PAG? this one has a really big text encoder which is not in transformers yet.

[sayakpaul]

@yiyixuxu @sayakpaul should I add the slow tests for PAG? this one has a really big text encoder which is not in transformers yet.

I would say, it'd be okay skipping the SLOW tests for now.

[a-r-r-o-w]

We should try and merge this before next release, no?

[sayakpaul]

LGTM. Let's add the pipeline to the list of pipelines we support in the docs?

[sayakpaul]

Seems like these should have been done in a separate PR?

[asomoza]

if I don't do this, the tests fails (throws AttributeError), I'm ok with doing this in a separate PR but then I'll have to skip the tests in this PR

[sayakpaul]

Ah okay. Then no worries.

[sayakpaul]

Same. Probably should have been done in a separate PR.