Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup pluginmanager of unused feature flag code #484

Merged

Conversation

marckhouzam
Copy link
Contributor

@marckhouzam marckhouzam commented Sep 19, 2023

What this PR does / why we need it

This commit removes all the unused code around feature flags from the manager.go file.

Code cleanup details:

  1. Remove ValidatePlugin() which was never used. There is a separate copy of this function in the tanzu-plugin-runtime repository.
  2. Remove discoverPlugins() and directly use discoverSpecificPlugins()
  3. Remove discoverServerPluginsBasedOnCurrentServer which is no longer used.
  4. Include the logic of discoverServerPluginsBasedOnAllCurrentContexts() directly into DiscoverServerPlugins() which had become a one-liner.
  5. Remove the following functions that were no longer used:
    - AvailablePlugins()
    - AvailablePluginsFromLocalSource()
    - availablePlugins()
    - combineDuplicatePlugins()
    - getInstalledButNotDiscoveredStandalonePlugins()
    - DiscoveredFromPlugininfo()
    - availablePluginsFromStandaloneAndServerPlugins()
    - pluginIndexForName()
    - legacyPluginInstall()
    - GetRecommendedVersionOfPlugin()

Completely remove the feature flag constant: FeatureDisableCentralRepositoryForTesting

Update the unit tests for the plugin manager.

This commit also improves our ability to test the pluginmanager.
It does this using two techniques:

  1. Creating a test plugin inventory DB in the cache and requesting the inventory code to always use the cache (without checking the digest). To force the use of the cache the commit Introduces a test variable TANZU_CLI_USE_DB_CACHE_ONLY. This approach allows to discover plugins and groups without needing a real OCI registry.

  2. Creating "fake" plugin binaries in the plugin binary cache. This allows installation of plugins to find the binaries in the cache and therefore never have to go to an OCI registry to download them.

With these two techniques, the unit tests can discovery and install plugins, and can also do so using groups. The plugin sync can also be tested with this.

Which issue(s) this PR fixes

Part of #477

Describe testing done for PR

# Remove the existing plugin DB
$ \rm -rf ~/.cache/tanzu/plugin_inventory
# Confirm the DB gets downloaded
$ tz plugin search --name apps
[i] Reading plugin inventory for "projects.registry.vmware.com/tanzu_cli/plugins/plugin-inventory:latest", this will take a few seconds.
  NAME  DESCRIPTION                 TARGET      LATEST
  apps  Applications on Kubernetes  kubernetes  v0.12.1
# Confirm the DB does not get downloaded again
$ tz plugin search --name isolated-cluster
  NAME              DESCRIPTION                                                       TARGET  LATEST
  isolated-cluster  Prepopulating images/bundle for internet-restricted environments  global  v0.30.1
$ tz plugin group search --name vmware-tkg/default
  GROUP               DESCRIPTION      LATEST
  vmware-tkg/default  Plugins for TKG  v2.3.0

# Remove the DB again
$ \rm -rf ~/.cache/tanzu/plugin_inventory

# Confirm the DB gets downloaded for groups
$ tz plugin group search --name vmware-tkg/default
[i] Reading plugin inventory for "projects.registry.vmware.com/tanzu_cli/plugins/plugin-inventory:latest", this will take a few seconds.
  GROUP               DESCRIPTION      LATEST
  vmware-tkg/default  Plugins for TKG  v2.3.0
# Confirm the DB does not get downloaded again
$ tz plugin group search --name vmware-tkg/default
  GROUP               DESCRIPTION      LATEST
  vmware-tkg/default  Plugins for TKG  v2.3.0

# Start with no plugins installed
$ rm ~/.cache/tanzu/catalog.yaml

# Install one plugin and also see the essential plugins get installed
$ tz plugin install secret
[i] The tanzu cli essential plugins have not been installed and are being installed now. The install may take a few seconds.

[x] : unable to uniquely identify plugin 'secret'. Please specify the target (kubernetes[k8s]/mission-control[tmc]/global) of the plugin using the `--target` flag
$ tz plugin install secret --target k8s
[i] Installing plugin 'secret:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'secret:v0.30.1' found in cache
[ok] successfully installed 'secret' plugin
$ tz plugin list
Standalone Plugins
  NAME       DESCRIPTION                                                 TARGET      VERSION  STATUS
  telemetry  configure cluster-wide settings for vmware tanzu telemetry  global      v1.1.0   installed
  secret     Tanzu secret management                                     kubernetes  v0.30.1  installed

# Install a specific plugin version
$ tz plugin install apps --version v0.11.1
[i] Installing plugin 'apps:v0.11.1' with target 'kubernetes'
[i] Plugin binary for 'apps:v0.11.1' found in cache
[ok] successfully installed 'apps' plugin

# Install plugins from a group
$ tz plugin install --group vmware-tkg/default
[i] Installing plugin 'isolated-cluster:v0.30.1' with target 'global'
[i] Plugin binary for 'isolated-cluster:v0.30.1' found in cache
[i] Installing plugin 'management-cluster:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'management-cluster:v0.30.1' found in cache
[i] Installing plugin 'package:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'package:v0.30.1' found in cache
[i] Installing plugin 'pinniped-auth:v0.30.1' with target 'global'
[i] Plugin binary for 'pinniped-auth:v0.30.1' found in cache
[i] Installing plugin 'secret:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'secret:v0.30.1' found in cache
[i] Installing plugin 'telemetry:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'telemetry:v0.30.1' found in cache
[ok] successfully installed all plugins from group 'vmware-tkg/default:v2.3.0'

# Look for plugin group versions
$ tz plugin group search --name vmware-tkg/default --show-details
name: vmware-tkg/default
description: Plugins for TKG
latest: v2.3.0
versions:
    - v1.6.0
    - v1.6.1
    - v2.1.0
    - v2.1.1
    - v2.2.0
    - v2.3.0

# Install plugins from a specific group version
$ tz plugin install --group vmware-tkg/default:v2.2.0
[i] Installing plugin 'isolated-cluster:v0.29.0' with target 'global'
[i] Installing plugin 'management-cluster:v0.29.0' with target 'kubernetes'
[i] Installing plugin 'package:v0.29.0' with target 'kubernetes'
[i] Installing plugin 'pinniped-auth:v0.29.0' with target 'global'
[i] Installing plugin 'secret:v0.29.0' with target 'kubernetes'
[i] Installing plugin 'telemetry:v0.29.0' with target 'kubernetes'
[ok] successfully installed all plugins from group 'vmware-tkg/default:v2.2.0'

# Do some installation through contexts
$ tz context list
Target:  kubernetes
  NAME  ISACTIVE  ENDPOINT  KUBECONFIGPATH                          KUBECONTEXT
  tkg1  false               /Users/kmarc/.k3d/kubeconfig-tkg1.yaml  k3d-tkg1
Target:  mission-control
  NAME  ISACTIVE  ENDPOINT
  tmc   false     unstable.tmc-dev.cloud.vmware.com:443
$ tz context use tkg1
[i] Checking for required plugins...
[i] Installing plugin 'cluster:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'cluster:v0.30.1' found in cache
[i] Installing plugin 'kubernetes-release:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'kubernetes-release:v0.30.1' found in cache
[i] Installing plugin 'feature:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'feature:v0.30.1' found in cache
[i] Successfully installed all required plugins

$ tz plugin list
Standalone Plugins
  NAME                DESCRIPTION                                                        TARGET      VERSION  STATUS
  isolated-cluster    Prepopulating images/bundle for internet-restricted environments   global      v0.29.0  installed
  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  global      v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         global      v1.1.0   installed
  apps                Applications on Kubernetes                                         kubernetes  v0.11.1  installed
  management-cluster  Kubernetes management cluster operations                           kubernetes  v0.29.0  installed
  package             Tanzu package management                                           kubernetes  v0.29.0  installed
  secret              Tanzu secret management                                            kubernetes  v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         kubernetes  v0.29.0  installed

Plugins from Context:  tkg1
  NAME                DESCRIPTION                           TARGET      VERSION  STATUS
  cluster             Kubernetes cluster operations         kubernetes  v0.30.1  installed
  feature             Operate on features and featuregates  kubernetes  v0.30.1  installed
  kubernetes-release  Kubernetes release operations         kubernetes  v0.30.1  installed

# Check that the installation happens (but then I interrupted because it is very long)
$ tz context use tmc
[i] Checking for required plugins...
[i] Installing plugin 'management-cluster:v0.1.2' with target 'mission-control'
^C
$ tz plugin list
Standalone Plugins
  NAME                DESCRIPTION                                                        TARGET      VERSION  STATUS
  isolated-cluster    Prepopulating images/bundle for internet-restricted environments   global      v0.29.0  installed
  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  global      v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         global      v1.1.0   installed
  apps                Applications on Kubernetes                                         kubernetes  v0.11.1  installed
  management-cluster  Kubernetes management cluster operations                           kubernetes  v0.29.0  installed
  package             Tanzu package management                                           kubernetes  v0.29.0  installed
  secret              Tanzu secret management                                            kubernetes  v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         kubernetes  v0.29.0  installed

Plugins from Context:  tkg1
  NAME                DESCRIPTION                           TARGET      VERSION  STATUS
  cluster             Kubernetes cluster operations         kubernetes  v0.30.1  installed
  feature             Operate on features and featuregates  kubernetes  v0.30.1  installed
  kubernetes-release  Kubernetes release operations         kubernetes  v0.30.1  installed

Plugins from Context:  tmc
  NAME                  DESCRIPTION                                                     TARGET           VERSION  STATUS
  account               Account for tmc resources                                       mission-control  v0.1.4   not installed
  agentartifacts        helm for tmc resources                                          mission-control  v0.1.4   not installed
  aks-cluster           Manage and unmanage tmc aks clusters                            mission-control  v0.1.7   not installed
  apply                 Create/Update a resource with a resource file                   mission-control  v0.3.1   not installed
  audit                 Run an audit request on an org                                  mission-control  v0.1.4   not installed
  cluster                                                                               mission-control  v0.1.6   not installed
  clustergroup          A group of Kubernetes clusters                                  mission-control  v0.1.4   not installed
  continuousdelivery    Continuousdelivery for tmc resources                            mission-control  v0.1.4   not installed
  data-protection       Data protection for tmc resources                               mission-control  v0.1.4   not installed
  ekscluster                                                                            mission-control  v0.1.4   not installed
  events                Events for any meaningful user activity or system state change  mission-control  v0.1.4   not installed
  helm                  helm for tmc resources                                          mission-control  v0.1.4   not installed
  iam                   IAM Policies for tmc resources                                  mission-control  v0.1.4   not installed
  inspection            Inspection for tmc resources                                    mission-control  v0.1.4   not installed
  integration           Get available integrations and their information from registry  mission-control  v0.1.4   not installed
  management-cluster    A TMC registered Management cluster                             mission-control  v0.1.2   not installed
  policy                Policy for tmc resources                                        mission-control  v0.1.4   not installed
  provider-aks-cluster  Manage and unmanage tmc provider aks clusters                   mission-control  v0.1.4   not installed
  provider-eks-cluster                                                                  mission-control  v0.1.4   not installed
  secret                secret for tmc resources                                        mission-control  v0.1.4   not installed
  setting               Setting plugin for tmc resources                                mission-control  v0.2.2   not installed
  tanzupackage          Tanzupackage for tmc resources                                  mission-control  v0.2.3   not installed
  workspace             A group of Kubernetes namespaces                                mission-control  v0.1.6   not installed

[!] As shown above, some recommended plugins have not been installed or are outdated. To install them please run 'tanzu plugin sync'.

# Try the other plugin operations
$ tz plugin delete cluster
Deleting Plugin 'cluster' for target 'kubernetes'. Are you sure? [y/N]: y
[ok] successfully deleted plugin 'cluster'
$ tz plugin list
Standalone Plugins
  NAME                DESCRIPTION                                                        TARGET      VERSION  STATUS
  isolated-cluster    Prepopulating images/bundle for internet-restricted environments   global      v0.29.0  installed
  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  global      v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         global      v1.1.0   installed
  apps                Applications on Kubernetes                                         kubernetes  v0.11.1  installed
  management-cluster  Kubernetes management cluster operations                           kubernetes  v0.29.0  installed
  package             Tanzu package management                                           kubernetes  v0.29.0  installed
  secret              Tanzu secret management                                            kubernetes  v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         kubernetes  v0.29.0  installed

Plugins from Context:  tkg1
  NAME                DESCRIPTION                           TARGET      VERSION  STATUS
  cluster             Kubernetes cluster operations         kubernetes  v0.30.1  not installed
  feature             Operate on features and featuregates  kubernetes  v0.30.1  installed
  kubernetes-release  Kubernetes release operations         kubernetes  v0.30.1  installed

Plugins from Context:  tmc
  NAME                  DESCRIPTION                                                     TARGET           VERSION  STATUS
  account               Account for tmc resources                                       mission-control  v0.1.4   not installed
  agentartifacts        helm for tmc resources                                          mission-control  v0.1.4   not installed
  aks-cluster           Manage and unmanage tmc aks clusters                            mission-control  v0.1.7   not installed
  apply                 Create/Update a resource with a resource file                   mission-control  v0.3.1   not installed
  audit                 Run an audit request on an org                                  mission-control  v0.1.4   not installed
  cluster                                                                               mission-control  v0.1.6   not installed
  clustergroup          A group of Kubernetes clusters                                  mission-control  v0.1.4   not installed
  continuousdelivery    Continuousdelivery for tmc resources                            mission-control  v0.1.4   not installed
  data-protection       Data protection for tmc resources                               mission-control  v0.1.4   not installed
  ekscluster                                                                            mission-control  v0.1.4   not installed
  events                Events for any meaningful user activity or system state change  mission-control  v0.1.4   not installed
  helm                  helm for tmc resources                                          mission-control  v0.1.4   not installed
  iam                   IAM Policies for tmc resources                                  mission-control  v0.1.4   not installed
  inspection            Inspection for tmc resources                                    mission-control  v0.1.4   not installed
  integration           Get available integrations and their information from registry  mission-control  v0.1.4   not installed
  management-cluster    A TMC registered Management cluster                             mission-control  v0.1.2   not installed
  policy                Policy for tmc resources                                        mission-control  v0.1.4   not installed
  provider-aks-cluster  Manage and unmanage tmc provider aks clusters                   mission-control  v0.1.4   not installed
  provider-eks-cluster                                                                  mission-control  v0.1.4   not installed
  secret                secret for tmc resources                                        mission-control  v0.1.4   not installed
  setting               Setting plugin for tmc resources                                mission-control  v0.2.2   not installed
  tanzupackage          Tanzupackage for tmc resources                                  mission-control  v0.2.3   not installed
  workspace             A group of Kubernetes namespaces                                mission-control  v0.1.6   not installed

[!] As shown above, some recommended plugins have not been installed or are outdated. To install them please run 'tanzu plugin sync'.
$ tz plugin describe cluster
[x] : unable to find plugin 'cluster'
$ tz plugin describe isolated-cluster
  NAME              VERSION  STATUS     TARGET  DESCRIPTION                                                       INSTALLATIONPATH
  isolated-cluster  v0.29.0  installed  global  Prepopulating images/bundle for internet-restricted environments  /Users/kmarc/Library/Application
                                                                                                                  Support/tanzu-cli/isolated-cluster/v0.29.0_a5daf2abfcad79a73eebd952bf0623b0a93e1eb6c76f417fb0c8d31726f1b483_global
$ tz plugin upgrade package
[i] Installing plugin 'package:v0.30.1' with target 'kubernetes'
[i] Plugin binary for 'package:v0.30.1' found in cache
[ok] successfully upgraded plugin 'package'
$ tz plugin list
Standalone Plugins
  NAME                DESCRIPTION                                                        TARGET      VERSION  STATUS
  isolated-cluster    Prepopulating images/bundle for internet-restricted environments   global      v0.29.0  installed
  pinniped-auth       Pinniped authentication operations (usually not directly invoked)  global      v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         global      v1.1.0   installed
  apps                Applications on Kubernetes                                         kubernetes  v0.11.1  installed
  management-cluster  Kubernetes management cluster operations                           kubernetes  v0.29.0  installed
  package             Tanzu package management                                           kubernetes  v0.30.1  installed
  secret              Tanzu secret management                                            kubernetes  v0.29.0  installed
  telemetry           configure cluster-wide settings for vmware tanzu telemetry         kubernetes  v0.29.0  installed

Plugins from Context:  tkg1
  NAME                DESCRIPTION                           TARGET      VERSION  STATUS
  cluster             Kubernetes cluster operations         kubernetes  v0.30.1  not installed
  feature             Operate on features and featuregates  kubernetes  v0.30.1  installed
  kubernetes-release  Kubernetes release operations         kubernetes  v0.30.1  installed

Plugins from Context:  tmc
  NAME                  DESCRIPTION                                                     TARGET           VERSION  STATUS
  account               Account for tmc resources                                       mission-control  v0.1.4   not installed
  agentartifacts        helm for tmc resources                                          mission-control  v0.1.4   not installed
  aks-cluster           Manage and unmanage tmc aks clusters                            mission-control  v0.1.7   not installed
  apply                 Create/Update a resource with a resource file                   mission-control  v0.3.1   not installed
  audit                 Run an audit request on an org                                  mission-control  v0.1.4   not installed
  cluster                                                                               mission-control  v0.1.6   not installed
  clustergroup          A group of Kubernetes clusters                                  mission-control  v0.1.4   not installed
  continuousdelivery    Continuousdelivery for tmc resources                            mission-control  v0.1.4   not installed
  data-protection       Data protection for tmc resources                               mission-control  v0.1.4   not installed
  ekscluster                                                                            mission-control  v0.1.4   not installed
  events                Events for any meaningful user activity or system state change  mission-control  v0.1.4   not installed
  helm                  helm for tmc resources                                          mission-control  v0.1.4   not installed
  iam                   IAM Policies for tmc resources                                  mission-control  v0.1.4   not installed
  inspection            Inspection for tmc resources                                    mission-control  v0.1.4   not installed
  integration           Get available integrations and their information from registry  mission-control  v0.1.4   not installed
  management-cluster    A TMC registered Management cluster                             mission-control  v0.1.2   not installed
  policy                Policy for tmc resources                                        mission-control  v0.1.4   not installed
  provider-aks-cluster  Manage and unmanage tmc provider aks clusters                   mission-control  v0.1.4   not installed
  provider-eks-cluster                                                                  mission-control  v0.1.4   not installed
  secret                secret for tmc resources                                        mission-control  v0.1.4   not installed
  setting               Setting plugin for tmc resources                                mission-control  v0.2.2   not installed
  tanzupackage          Tanzupackage for tmc resources                                  mission-control  v0.2.3   not installed
  workspace             A group of Kubernetes namespaces                                mission-control  v0.1.6   not installed

[!] As shown above, some recommended plugins have not been installed or are outdated. To install them please run 'tanzu plugin sync'.

Release note

Improve unit testing for plugin lifecycle and remove unused code around the previous use of feature flags.

Additional information

Special notes for your reviewer

A new test variable is introduced: TEST_TANZU_CLI_USE_DB_CACHE_ONLY. When set to 1 (or any other enabled boolean value) it forces the DBBackedOCIDiscovery to only use the cache. Using this variable, tests now setup an sqlite
DB file in the cache, and the tests will use it without having to access a registry. Note that we have a DiscoveryOptions WithUseLocalCacheOnly() which could have been used for some tests, specifically the tests discovering plugins. However, in other tests such as the ones installing plugins, using only the cache is not normally supported so this variable is a way to short-circuit that.

The test coverage from the manager.go has gone from 73.6% to 76.5%. This may not seem like a big deal, but after the cleanup the test coverage had actually dropped to 68.6%, so getting back to over 75% required some effort.

More unit tests would benefit from moving to the new approach of testing instead of relying on local plugin installation. However such an improvement will need to be done in a future effort.

@marckhouzam marckhouzam requested a review from a team as a code owner September 19, 2023 14:44
@marckhouzam marckhouzam force-pushed the feat/cleanupPluginManager branch 4 times, most recently from b3bfacf to 5e4fc78 Compare September 19, 2023 15:37
This commit removes all the unused code around feature flags from the
`manager.go` file.  Those changes are:
1. Remove `ValidatePlugin()` which was never used.  There is a
   separate copy of this function in the tanzu-plugin-runtime repository.
2. Remove `discoverPlugins()` and directly use `discoverSpecificPlugins()`
3. Remove `discoverServerPluginsBasedOnCurrentServer` which is no longer
   used.
4. Include the logic of `discoverServerPluginsBasedOnAllCurrentContexts()`
   directly into `DiscoverServerPlugins()` which had become a one-liner.
5. Remove the following functions that were no longer used:
      - `AvailablePlugins()`
      - `AvailablePluginsFromLocalSource()`
      - `availablePlugins()`
      - `combineDuplicatePlugins()`
      - `getInstalledButNotDiscoveredStandalonePlugins()`
      - `DiscoveredFromPlugininfo()`
      - `availablePluginsFromStandaloneAndServerPlugins()`
      - `pluginIndexForName()`
      - `legacyPluginInstall()`
      - `GetRecommendedVersionOfPlugin()`

Completely remove the feature flag constant:
  FeatureDisableCentralRepositoryForTesting

* Update the unit tests for the plugin manager.

This commit improves our ability to test the pluginmanager.
It does this using two techniques:

1. Creating a test plugin inventory DB in the cache and requesting the
   inventory code to always use the cache (without checking the digest).
   To force the use of the cache the commit introduces a test variable
   `TEST_TANZU_CLI_USE_DB_CACHE_ONLY`.
   This approach allows to discover plugins and groups without needing
   a real OCI registry.

2. Creating "fake" plugin binaries in the plugin binary cache.  This
   allows installation of plugins to find the binaries in the cache and
   therefore never have to go to an OCI registry to download them.

With these two techniques, the unit tests can discovery and install
plugins, and can also do so using groups.  The plugin sync can also be
tested with this.

Note: the unit tests would still benefit from some tests moving to this
approach of testing, but this will need to come in a future effort.

Signed-off-by: Marc Khouzam <kmarc@vmware.com>
Copy link
Contributor

@anujc25 anujc25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for doing this. This will help a lot going forward as it removes some complex unused code. Also appreciate you adding unit tests as well.

@marckhouzam marckhouzam merged commit c0e6210 into vmware-tanzu:main Sep 20, 2023
4 checks passed
@marckhouzam marckhouzam deleted the feat/cleanupPluginManager branch September 20, 2023 01:48
@marckhouzam marckhouzam added this to the 1.1.0 milestone Oct 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants