LoRA Adapter

[slyalin]

TODO:

• Eliminate additional constructors in LLMPipeline with explicit AdapterConfig arguments, use properties instead like it is available in generate.
• Add an additional constructor to LLMPipeline that accepts a variadic list of properties. It is more convenient than AnyMap with properties, and aligned with generate. A trivial attempt to do that failed due to a specific implementation of Property::Forward helper class that confuses the compiler if multiple such arguments are passed. Requires a more accurate definition of AdatperConfig constructors.
• Avoid two-stage weight preparation in MODE_FUSE when multiple LoRAs are combined. Now there are two separately compiled CPU models: one is for LoRAs concatenation and another one is for main weight fusion. These two helper models can be merged into one.
• Python API
• User friendly comments for publicly exposed classes
• Functional tests
• Benchmark
• One selected variant for demonstration in lora_greedy_causal_lm sample instead of a series of different modes.
• Part of FIXMEs that can be addressed without waiting for fixes at the plugins side