[Fleet] Add installed template components to installation object and remove them #72959
Assignees
Labels
Team:Fleet
Team label for Observability Data Collection Fleet team
technical debt
Improvement of the software architecture and operational architecture
Comments
neptunian
added
the
Team:Fleet
|
Pinging @elastic/ingest-management (Team:Ingest Management) |
ph
added
technical debt
jen-huang
changed the title
|
@jfsiii Assigning to you as it sounds like you have already made some progress here. |
2 tasks
jfsiii
pushed a commit
that referenced
this issue
Jun 23, 2021
## Summary Closes #90454 Closes #72959 * Rename the component templates which are [installed for some packages](https://github.com/elastic/kibana/blob/master/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/install.ts#L197-L213) from `${templateName}-mappings` and `${templateName}-settings` to `${templateName}@mappings` and `${templateName}@settings` * When any package is installed, add a component template named `${templateName}@custom` * Any of above templates also include a `_meta` property with `{ package: { name: packageName } }` * On package installation, add any installed component templates to the `installed_es` property of the `epm-packages` saved object * On package removal, remove any installed component templates from the `installed_es` property of the `epm-packages` saved object <details><summary>Kibana logs showing component templates added for package</summary> ``` │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.file@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.registry@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [.logs-endpoint.diagnostic.collection@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.library@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.security@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.network@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.alerts@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metrics@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.process@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.policy@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metadata@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.registry@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [.logs-endpoint.diagnostic.collection@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.security@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.file@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.library@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.network@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.alerts@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metrics@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.policy@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.process@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metadata@custom] ``` </details> <details><summary>screenshot - component templates are editable in the Stack Management UI</summary> <img width="1342" alt="Screen Shot 2021-06-17 at 4 06 24 PM" src="https://user-images.githubusercontent.com/57655/122465421-1502bb80-cf86-11eb-94f4-9880cb3ea844.png"> </details> ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Jun 23, 2021
## Summary Closes elastic#90454 Closes elastic#72959 * Rename the component templates which are [installed for some packages](https://github.com/elastic/kibana/blob/master/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/install.ts#L197-L213) from `${templateName}-mappings` and `${templateName}-settings` to `${templateName}@mappings` and `${templateName}@settings` * When any package is installed, add a component template named `${templateName}@custom` * Any of above templates also include a `_meta` property with `{ package: { name: packageName } }` * On package installation, add any installed component templates to the `installed_es` property of the `epm-packages` saved object * On package removal, remove any installed component templates from the `installed_es` property of the `epm-packages` saved object <details><summary>Kibana logs showing component templates added for package</summary> ``` │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.file@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.registry@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [.logs-endpoint.diagnostic.collection@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.library@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.security@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.network@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.alerts@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metrics@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.process@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.policy@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metadata@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.registry@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [.logs-endpoint.diagnostic.collection@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.security@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.file@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.library@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.network@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.alerts@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metrics@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.policy@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.process@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metadata@custom] ``` </details> <details><summary>screenshot - component templates are editable in the Stack Management UI</summary> <img width="1342" alt="Screen Shot 2021-06-17 at 4 06 24 PM" src="https://user-images.githubusercontent.com/57655/122465421-1502bb80-cf86-11eb-94f4-9880cb3ea844.png"> </details> ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios
kibanamachine
added a commit
that referenced
this issue
Jun 23, 2021
…3126) ## Summary Closes #90454 Closes #72959 * Rename the component templates which are [installed for some packages](https://github.com/elastic/kibana/blob/master/x-pack/plugins/fleet/server/services/epm/elasticsearch/template/install.ts#L197-L213) from `${templateName}-mappings` and `${templateName}-settings` to `${templateName}@mappings` and `${templateName}@settings` * When any package is installed, add a component template named `${templateName}@custom` * Any of above templates also include a `_meta` property with `{ package: { name: packageName } }` * On package installation, add any installed component templates to the `installed_es` property of the `epm-packages` saved object * On package removal, remove any installed component templates from the `installed_es` property of the `epm-packages` saved object <details><summary>Kibana logs showing component templates added for package</summary> ``` │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.file@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.registry@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [.logs-endpoint.diagnostic.collection@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.library@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.security@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.network@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.alerts@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metrics@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.process@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.policy@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metadata@mappings] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.registry@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [.logs-endpoint.diagnostic.collection@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.security@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.file@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.library@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.network@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.alerts@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metrics@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.policy@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [logs-endpoint.events.process@custom] │ info [o.e.c.m.MetadataIndexTemplateService] [JFSIII.local] adding component template [metrics-endpoint.metadata@custom] ``` </details> <details><summary>screenshot - component templates are editable in the Stack Management UI</summary> <img width="1342" alt="Screen Shot 2021-06-17 at 4 06 24 PM" src="https://user-images.githubusercontent.com/57655/122465421-1502bb80-cf86-11eb-94f4-9880cb3ea844.png"> </details> ### Checklist Delete any items that are not applicable to this PR. - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios Co-authored-by: John Schulz <john.schulz@elastic.co>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As part of elastic/package-registry#497 and #68380 implemented #70517, component templates belonging to dataset index templates should be tracked as part of a package installation and added to the
epm-packageinstallation saved object. When a package is removed, the components should also be removed. Currently, AFAIK onlyendpointis using component templates and it is not a removable package, but we ought to get this done to support removable packages using component templates.The text was updated successfully, but these errors were encountered: