LCM cpp demo #101

likholat · 2023-12-21T20:25:30Z

ToDo:

Add Readme
Add test
IR from optimum

Only static model is supported for now

image_generation/lcm_dreamshaper_v7/cpp/scripts/convert_model.py

image_generation/common/diffusers/CMakeLists.txt

Wovchena

I haven't tried to run it yet

image_generation/lcm_dreamshaper_v7/cpp/CMakeLists.txt

image_generation/lcm_dreamshaper_v7/cpp/README.md

Wovchena · 2023-12-25T09:36:41Z

.github/workflows/lcm_dreamshaper_cpp.yml

I'm ok with master target branch, but you may want to retarget to releases

image_generation/lcm_dreamshaper_v7/cpp/README.md

image_generation/common/diffusers/src/scheduler_lcm.cpp

image_generation/lcm_dreamshaper_v7/cpp/scripts/convert_model.py

image_generation/lcm_dreamshaper_v7/cpp/src/main.cpp

Wovchena · 2023-12-25T12:59:48Z

image_generation/lcm_dreamshaper_v7/cpp/scripts/convert_model.py

py scripts\convert_model.py -lcm "SimianLuo/LCM_Dreamshaper_v7" -t FP16 printed

Exception ignored in: <finalize object at 0x1fa1050bd20; dead> Traceback (most recent call last): File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\weakref.py", line 590, in __call__ return info.func(*info.args, **(info.kwargs or {})) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 894, in _cleanup cls._rmtree(name, ignore_errors=ignore_errors) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 881, in onerror cls._rmtree(path, ignore_errors=ignore_errors) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 603, in _rmtree_unsafe onerror(os.scandir, path, sys.exc_info()) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 600, in _rmtree_unsafe with os.scandir(path) as scandir_it: ^^^^^^^^^^^^^^^^ NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\vzlobin\\AppData\\Local\\Temp\\tmp0btk8gby\\text_encoder\\openvino_model.bin'

Although it didn't fail. There's the same issue for stable_diffusion_1_5, but maybe you know what to do about it?

I haven't tested it for Windows yet: https://github.com/likholat/openvino.genai/blob/lcm_support/image_generation/lcm_dreamshaper_v7/cpp/README.md#:~:text=This%20demo%20has%20been%20tested%20for%20Linux%20platform%20only

py scripts\convert_model.py -lcm "SimianLuo/LCM_Dreamshaper_v7" -t FP16 printed

Exception ignored in: <finalize object at 0x1fa1050bd20; dead> Traceback (most recent call last): File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\weakref.py", line 590, in __call__ return info.func(*info.args, **(info.kwargs or {})) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 894, in _cleanup cls._rmtree(name, ignore_errors=ignore_errors) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 617, in _rmtree_unsafe _rmtree_unsafe(fullname, onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 622, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 881, in onerror cls._rmtree(path, ignore_errors=ignore_errors) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\tempfile.py", line 890, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 759, in rmtree return _rmtree_unsafe(path, onerror) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 603, in _rmtree_unsafe onerror(os.scandir, path, sys.exc_info()) File "C:\Users\vzlobin\AppData\Local\Programs\Python\Python311\Lib\shutil.py", line 600, in _rmtree_unsafe with os.scandir(path) as scandir_it: ^^^^^^^^^^^^^^^^ NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\\Users\\vzlobin\\AppData\\Local\\Temp\\tmp0btk8gby\\text_encoder\\openvino_model.bin'

Although it didn't fail. There's the same issue for stable_diffusion_1_5, but maybe you know what to do about it?

it looks like optimum issue

image_generation/lcm_dreamshaper_v7/cpp/README.md

Wovchena · 2023-12-29T11:04:21Z

image_generation/common/diffusers/include/scheduler_lcm.hpp

+#include <list>
+


Suggested change

#include <list>

image_generation/lcm_dreamshaper_v7/cpp/README.md

image_generation/common/diffusers/include/scheduler_lcm.hpp

likholat · 2023-12-29T14:24:26Z

@Wovchena @ilya-lavrenov thanks for review :)

ilya-lavrenov · 2024-01-08T18:22:48Z

image_generation/lcm_dreamshaper_v7/cpp/scripts/convert_model.py

+model = OVLatentConsistencyModelPipeline.from_pretrained(args.lcm_weights,trust_remote_code=True, export=True, compile=False)
+if args.type == "FP16":
+    model.half()
+    model.reshape(1, 512, 512, 1)


what if we don't use FP16? Looks like the model will be saved with dynamic shapes

Updated here: #135

image_generation/lcm_dreamshaper_v7/cpp/src/main.cpp

ilya-lavrenov · 2024-01-08T18:26:29Z

image_generation/lcm_dreamshaper_v7/cpp/src/main.cpp

+
+    ov::Tensor text_embeddings(ov::element::f32, {1, MAX_LENGTH, HIDDEN_SIZE});
+
+    compute_text_embeddings(pos_prompt, ov::Tensor(text_embeddings, {0, 0, 0}, {1, MAX_LENGTH, HIDDEN_SIZE}));


looks like we can embed (remove) this lambda since it's called only once

ilya-lavrenov · 2024-01-08T18:28:32Z

image_generation/lcm_dreamshaper_v7/cpp/src/main.cpp

+
+    std::vector<float> res_vec(half_dim), emb_cos(half_dim);
+    std::transform(emb_vec.begin(), emb_vec.end(), res_vec.begin(), [](float x){return std::sin(x);});
+    std::transform(emb_vec.begin(), emb_vec.end(), emb_cos.begin(), [](float x){return std::cos(x);});


looks like we can remove this multiple std::transform calls with simple for loop, which performs several operations at once

ilya-lavrenov · 2024-01-08T18:29:28Z

image_generation/lcm_dreamshaper_v7/cpp/src/main.cpp

+        ov::Tensor latent_model_input = randn_tensor(height, width, read_np_latent, seed);
+
+        float guidance_scale = 8.0;
+        ov::Tensor guidance_scale_embedding = get_w_embedding(guidance_scale, 256);


guidance_scale_embedding creation can be moved outside of the num_images loop

ilya-lavrenov · 2024-01-08T18:30:47Z

image_generation/lcm_dreamshaper_v7/cpp/README.md

+* `-s, --seed arg`      Number of random seed to generate latent (default: 42)
+* `--num arg`           Number of image output(default: 1)
+* `--height arg`        Height of output image (default: 512)
+* `--width arg`         Width of output image (default: 512)


it's stated that Only static model is currently supported for this sample. So, do we need height and width parameters then?

likholat · 2024-01-09T16:28:21Z

#135

ilya-lavrenov reviewed Dec 21, 2023

View reviewed changes

image_generation/lcm_dreamshaper_v7/cpp/scripts/convert_model.py Show resolved Hide resolved

Wovchena reviewed Dec 22, 2023

View reviewed changes

image_generation/common/diffusers/CMakeLists.txt Outdated Show resolved Hide resolved

likholat force-pushed the lcm_support branch from abf6da4 to 58826e1 Compare December 22, 2023 18:19

likholat marked this pull request as ready for review December 22, 2023 18:20

Wovchena requested changes Dec 25, 2023

View reviewed changes

Wovchena requested changes Dec 26, 2023

View reviewed changes

likholat force-pushed the lcm_support branch 2 times, most recently from 557f851 to 3756a06 Compare December 28, 2023 15:56

likholat added 9 commits December 28, 2023 16:58

added LCM scheduler and sample

f4cf7d8

text_embeddings, noise generation

0c2c1ec

model from optimum, readme, ci test

44c1580

fix comments

dd615ff

added images

eb557cb

fixed diffusers version

832778f

rm extra files

e69dee3

add dependency

5b9a375

tested with lora

3756a06

likholat requested review from Wovchena and ilya-lavrenov December 28, 2023 16:14

Wovchena requested changes Dec 29, 2023

View reviewed changes

fix comments

b270bff

likholat requested a review from Wovchena December 29, 2023 13:29

Wovchena approved these changes Dec 29, 2023

View reviewed changes

Wovchena merged commit 4116365 into openvinotoolkit:master Dec 29, 2023
2 checks passed

ilya-lavrenov reviewed Jan 8, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

LCM cpp demo #101

LCM cpp demo #101

likholat commented Dec 21, 2023 •

edited

Loading

Wovchena left a comment

Wovchena Dec 25, 2023

Wovchena Dec 25, 2023

likholat Dec 28, 2023

ilya-lavrenov Jan 10, 2024

Wovchena Dec 29, 2023

likholat commented Dec 29, 2023

ilya-lavrenov Jan 8, 2024

likholat Jan 9, 2024

ilya-lavrenov Jan 8, 2024

ilya-lavrenov Jan 8, 2024

ilya-lavrenov Jan 8, 2024

ilya-lavrenov Jan 8, 2024

likholat commented Jan 9, 2024


		ov::Tensor text_embeddings(ov::element::f32, {1, MAX_LENGTH, HIDDEN_SIZE});

		compute_text_embeddings(pos_prompt, ov::Tensor(text_embeddings, {0, 0, 0}, {1, MAX_LENGTH, HIDDEN_SIZE}));

LCM cpp demo #101

LCM cpp demo #101

Conversation

likholat commented Dec 21, 2023 • edited Loading

Wovchena left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

likholat commented Dec 29, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

likholat commented Jan 9, 2024

likholat commented Dec 21, 2023 •

edited

Loading