Expose detected config deprecations to plugins

[joshdover]

In order to support an enhanced upgrade experience, users should be able to see any detected deprecated configs in the Upgrade Assistant (#54702).

While Core currently collects these deprecations, it does not current expose any detected deprecations to plugins. We should add an API to the ConfigService so that this can be reported on.

Steps:

• Add new fields to config deprecation API
• Update all existing config deprecation implementations for new fields
  • If this turns out difficult, we can involve other teams
• Expose new API of config deprecations to upgrade assistant

[joshdover]

@jloleysens Is this still something that we need before 8.0?

[jloleysens]

I would say yes. We have primarily been discussing surfacing stack-wide deprecations, config should form part of that for sure.

For reference, this seems to be roughly the same as what I proposed here: #54702 but driving it from core sounds much better as there will not be potential dependency issues. @pugnascotia also opened an issue regarding this https://github.com/elastic/dev/issues/1426 at a stack-wide level.

[joshdover]

Thinking about the API here a bit, one tricky aspect is that our config deprecation mechanism is very generic and allows plugins to apply any modifications they'd like to the resulting config. Without changing how this mechanism works, it will be hard to provide structured data for the deprecations. The simplest option would be to only expose the log messages that were added for deprecated config keys:

interface ConfigService {
  /* ... */

  /** Returns an Observable of all current configuration deprecation warnings */
  deprecationWarnings$(): Observable<string[]>;

  /* ... */
}

The strings in this simple case would read like:

• "${fullOldPath}" is deprecated and has been replaced by "${fullNewPath}"
• "${fullOldPath}" is deprecated and has been replaced by "${fullNewPath}". However both keys are present, ignoring "${fullOldPath}"
• ${fullPath} is deprecated and is no longer used

If we would like more structured output (documentation URLs, whether or not the config key was renamed, removed, etc.) in the UI, we would need to make some more significant changes to the config deprecation mechanism. With such changes, we could provide an API like:

interface ConfigDeprecationWarning {
  configKey: string;
  message: string;
  correctiveAction: 'rename' | 'remove' | 'other';
  documenationUrl?: string;
}

interface ConfigService {
  /* ... */

  /** Returns an Observable of all current configuration deprecation warnings */
  deprecationWarnings$(): Observable<ConfigDeprecationWarning[]>;

  /* ... */
}

@alisonelizabeth Any thoughts on the level of detail we need here?

[alisonelizabeth]

@joshdover sorry for the late reply here. The scope of what we will display in UA and how is still under discussion. I think ideally it would be preferred if we could have access to a more structured output, but I will keep you posted as we get more clarity around this.

[alisonelizabeth]

@joshdover to follow up on our conversation earlier - I think the fields you defined above will satisfy what we need in UA. I think it might also be beneficial to have a level field. Let me know what you think.

interface ConfigDeprecationWarning {
  configKey: string;
  message: string;
  correctiveAction: 'rename' | 'remove' | 'other';
  documentationUrl?: string;
  level: 'critical' | 'warning';
}

[Bamieh]

Side-question: Do we want to have those messages internationalized?

Yes: localized content for better user experience.
No: Consistent with ES deprecation logs since they have no i18n. It would also be easier to search the web and find answers with a non localized string.

At the moment We dont have a strict convension about localizing logs/errors coming from the server.

[alisonelizabeth]

@Bamieh that's a good question. I agree it is a better UX. However, for now, based on the other reasons you pointed out (ES deprecations are not translated, easier to search, no conventions currently in place), maybe we use a non-localized string.

[Bamieh]

@alisonelizabeth +1 on not using i18n for the deprecation messages. Thanks!

[rudolf]

Since there is already a stack-wide destination for deprecations logs https://github.com/elastic/dev/issues/1426 I think we shouldn't introduce a new Core deprecationWarnings$(): Observable<string[]>; API. Instead, core should just write config deprecations into the deprecation logs index directly.

Upgrade assistant can then consume these logs like any other deprecation log #79208

We will not be sending "static" deprecations like config deprecation to the deprecation logs index as it makes it harder to know when a deprecation has been successfully resolved.

[rudolf]

There seems to be only a few plugins with config deprecations so it shouldn't be a lot of work to coordinate a breaking API change.

Search results for ConfigDeprecationFactory (ConfigDeprecation was also only used in these plugins).

x-pack/plugins/monitoring/server/deprecations.ts
x-pack/plugins/monitoring/target/types/server/deprecations.d.ts
x-pack/plugins/reporting/server/config/index.test.ts
x-pack/plugins/security/server/config_deprecations.test.ts
x-pack/plugins/spaces/server/config.test.ts
x-pack/plugins/task_manager/server/index.test.ts

[Bamieh]

I believe this is safe to close now.

Steps:

• Add new fields to config deprecation API: Update all existing config deprecation implementations for new fields

Config deprecations now automatically provide manual steps for the users to fix those deprecations. We also provide a onDerpecation hook to config.deprecations to allow customizing the deprecation messaging and corrective action.

I believe we need to make sure all kibana plugin owners are aware of this introduced capablity so they can customize their deprecated configs as needed. (cc @joshdover ) A separate issue for this might be a good start?

• Expose new API of config deprecations to upgrade assistant

[joshdover]

Agreed, we can close this now!

I believe we need to make sure all kibana plugin owners are aware of this introduced capablity so they can customize their deprecated configs as needed. (cc @joshdover ) A separate issue for this might be a good start?

A separate issue would be good with a checklist of existing deprecations that need more detailed info. We can follow a similar format to #83910. Would you mind creating that?

[Bamieh]

Follow up: We've agreed not to create a meta issue since we dont have actionable items for teams to follow up on.
We just need to increase visiblity of the feature which we are utilizing other channels to do so.