Feature: Optional Lazy Loading of Transitive Dependencies

[Sewer56]

Problem

Hello, it's been a while since my last contribution #111 but I figured I'd swing around again.

Forever thirsty for performance, I've been trying to optimise startup times in my own application; not because it's strictly necessary but because I figured I can. Maybe it's just the perfectionist inside me.

After some investigation it occurred to me that startup has always been mostly I/O bottlenecked (as expected). After some minor experimentation, I figured that it is possible to partially work around that and came up with an idea to heavily reduce this bottleneck.

Solution

Load dependencies of exported plugin assemblies lazily (just in time as they are needed!) rather than pre-loading them all ahead of time.

This PR adds an optional feature IsLazyLoaded in PluginConfig which achieves just that.
Relevant patches are made in AssemblyLoadContextBuilder.PreferDefaultLoadContextAssembly and ManagedLoadContext.Load.

Use Cases

Main use case for this feature would be extracting small amounts of data from the plugins and/or loading many plugins at once.

For the former, sometimes you might only want to extract a small snippet of embedded information and as such loading all dependencies is not necessary to do ahead of time.

For the latter, consider situations such as applications with 20, 40, 60, 80+ etc. plugins being cold loaded after a PC restart. On a conventional hard drive, we might be saving seconds here.

Example: UI Application & Configurable Plugins

Plugins might have an interface which can be used for exporting configurations. These configurations might then be edited using a generic interface such as a PropertyGrid in WinForms.

Currently selecting an individual plugin (especially cold loads) in UI applications may cause a noticeable stutter/lag spike due to the aforementioned I/O bottleneck.

Risks

The intended/main usage of the library, host sharing a contract in the form of a known interface at compile time with the plugins should be unaffected by this optional feature. There should not be any functional difference introduced by not loading ahead of time.

Transitive dependencies may come to mind, specifically dependencies of the plugin that the host is also aware or has a copy of. These function just the same way as if they were loaded ahead of time; I added a test for this just in case.

I have not managed to identify any cases in which the introduction of this feature would break existing functionality.

Tests

As for the test project; I wasn't really sure if this feature needed any special specific tests outside of ensuring transitive assemblies are still correctly resolved; so I added an additional variation of TransitiveAssembliesOfSharedTypesAreResolved in SharedTypesTests that uses the new setting to ensure correct resolution.

I also tested this on real software and a few random existing tests just in case (see below), everything seems to be working properly.

Results (Real Software)

People tend to often showcase the best case scenario for changes; I figured I'd go the other way around and show the worst case scenario instead.

Notably; we are working around an I/O bottleneck here so the worst case condition is to perform a hot load (i.e. plugins are either cached by the OS in memory or by hardware). This was performed by loading 5 times in a row and taking the best time before and after.

Before:

After:

This is Reloaded II, using a sample of 16 plugins.

Specifically benchmarked is the process described in #60 (comment) , where Loading Assembly Metadata refers to step 2 (for all plugins); and initialising refers to step 3; both performed in parallel to maximise performance (this is an I/O bottleneck after all).

Real world performance improvement for "worst case" tends to be in the range of 2.5x-3.0x considering the numbers also include finding the entry point etc. I haven't tested cold loads (best case scenario as this is an I/O bottleneck) but I'd expect the difference to be more significant.

Edit:
I took a quick cold boot test after all.
Total time saved for this highly optimised concurrent load scenario was 300ms, as opposed to 100ms in cold boot on a standard SATA 3 SSD.
Savings weren't as large as expected but still fairly noticeable nonetheless; especially when executing non-concurrently.

Other Miscellaneous Changes (Performance)

• PluginLoader.CreateFromAssemblyFile(string, Type[], Action<PluginConfig>)

Instead of adding the assembly for each type, I considered that there are end users (unfamiliar to inner workings) who may pass multiple types belonging to the same assembly to the method. With that in mind, I made the library remove duplicates before they are added to config.SharedAssemblies.

The main reason for this is down the road, the loader will call AssemblyLoadContextBuilder.PreferDefaultLoadContextAssembly; this is an expensive operation when lazy loading is not enabled and it does not need to be executed multiple times with the same assembly name.

[Sewer56]

Closing due to branch rename. Will open new PR (with same commits) soon.