Framework should support plugging new providers in from Tanzu Distributions

[joshrosso]

This issue was sourced from the TCE repo: vmware-tanzu/community-edition#2513

The ask is for framework to provide a way where distributions can bring their own provider, such that it does not need to propagate in all distributions using framework.

Describe the feature request

From @dims:

There are several CAPI providers in the community already and there are many downstream/third-party/custom CAPI providers as well. Since most of the upstream tooling/utilities do support these, we need to find a way to document these scenarios and make pluggable as needed in TCE.

Let's take an example of the CAPG provider in the community for which a feature request has already been added:
vmware-tanzu/community-edition#2165

There should be a way for folks to use TCE with GCP easily, it's quite possible that there are things in TCE that needs to be extensible to support these external CAPI providers. These may include changes here in this repository as well as the tanzu-framework.

Ideally one should be able to add a external CAPI provider without any changes in TCE, but we may be far away from the ideal state. So could we please use the need for CAPG as a way to experiment how to support this scenario and document the steps needed right now (while we are working through it) and use this experience to remove the rough edges over time to get to the ideal state mentioned above over time.

[blc1996]

Hi @vuil , could you help triage this issue?

[swalner-vmware]

@vuil plans to flesh this out more, because area/lcm is not an exact match.

[vuil]

Thanks @joshrosso, allowing integration of third party providers with minimal friction is definitely something we should look into. Today that area of responsibility probably falls between how distributions expresses what the providers are available, and how plugins like cluster/management-cluster will leverage that information.

Here's an idea of a more tactical solution and the changes we may have to do to get there:

Today the providers used by the CLI is driven by a single OCI image referenced in the BOM for a said distribution of CLI. Said image contains provider specific data like CRDs, templates, ytt overlays, plan definitions. There is an assumption that one and only one such image will be use to deploy these data client-side today.
An obvious way to modify existing or introduce additional providers is to be able to publish an new version of this image and a BOM to reference it.
A more practical approach may to allow the flexibility for other provider vendors to independently publish their own provider images, introduce a means in the BOM publishing process to reference them or introduce a difference means to install them separately. To account for this flexibility, there will be changes needed in the lcm plugin code to work with multiple sets of provider data.
One area that also needs further consideration is the expansion of the number of supported providers will increase the likelihood of needing to support multiple CAPI providers in the same management cluster deployment. We are mostly there, but will need to file issues to audit what additional changes and testing are required to validate we have full support of multiple providers.

[codegold79]

Part of the acceptance criteria for the PR that resolves this issue should be to include tests that prove that a CAPI implementation is compatible with TCE.

See more information in #1779, which as been closed in favor of implementing the tests here. If tests seem outside the scope of this issue, please re-open #1779 as it should be done.