[ML] add new flag exclude_generated that removes generated fields in GET config APIs
#63899
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
When exporting and cloning ml configurations in a cluster it can be frustrating to remove all the fields that were generated by the plugin. Especially as the number of these fields change from version to version. This flag, remove_generated, allows the GET config APIs to return configurations with these generated fields removed. relates to elastic#63055
|
Pinging @elastic/ml-core (:ml) |
davidkyle
reviewed
Oct 20, 2020
| @@ -67,7 +67,9 @@ Specifies whether the included model definition should be returned as a JSON map | |||
|
|
|||
| `for_export`:: | |||
| (Optional, boolean) | |||
| include::{es-repo-dir}/ml/ml-shared.asciidoc[tag=for-export] | |||
| Indicates if certain fields should be removed from the model configuration on | |||
There was a problem hiding this comment.
Same comment as #63093 (comment)
There was a problem hiding this comment.
Can the ml-shared.asciidoc snippit be written in a general enough way to be used here too?
Shouldn't for_export be deprecated for trained models and switch to the exclude_generated flag? Or it that for another PR
There was a problem hiding this comment.
@davidkyle I think keeping it for_export makes sense for now as its use case is really only for export (no clone makes sense for trained models)
There was a problem hiding this comment.
I'd much prefer the same flag was used for the same purpose in all the APIs. Trained models will look like an oddity when transforms, AD jobs and DFA all use exclude_generated
The use case shouldn't define the parameter name, in future there may be another reason to get the model without the generated fields e.g. copy and update the model with further training. I like exclude_generated as it describes the action that takes place rather than one possible use case.
|
run elasticsearch-ci/1 |
…ag-for-clone-export
benwtrent
added a commit
to benwtrent/elasticsearch
that referenced
this pull request
Oct 20, 2020
…in GET config APIs (elastic#63899) When exporting and cloning ml configurations in a cluster it can be frustrating to remove all the fields that were generated by the plugin. Especially as the number of these fields change from version to version. This flag, exclude_generated, allows the GET config APIs to return configurations with these generated fields removed. APIs supporting this flag: - GET _ml/anomaly_detection/<job_id> - GET _ml/datafeeds/<datafeed_id> - GET _ml/data_frame/analytics/<analytics_id> The following fields are not returned in the objects: - any field that is not user settable (e.g. version, create_time) - any field that is a calculated default value (e.g. datafeed chunking_config) - any field that is automatically set via another Elastic stack process (e.g. anomaly job custom_settings.created_by) relates to elastic#63055
benwtrent
added a commit
that referenced
this pull request
Oct 20, 2020
…ields in GET config APIs (#63899)(#63092) (#63177) * [ML] adding for_export flag for ml plugin GET resource APIs (#63092) This adds the new `for_export` flag to the following APIs: - GET _ml/anomaly_detection/<job_id> - GET _ml/datafeeds/<datafeed_id> - GET _ml/data_frame/analytics/<analytics_id> The flag is designed for cloning or exporting configuration objects to later be put into the same cluster or a separate cluster. The following fields are not returned in the objects: - any field that is not user settable (e.g. version, create_time) - any field that is a calculated default value (e.g. datafeed chunking_config) - any field that would effectively require changing to be of use (e.g. datafeed job_id) - any field that is automatically set via another Elastic stack process (e.g. anomaly job custom_settings.created_by) closes #63055 * [ML] adding new flag exclude_generated that removes generated fields in GET config APIs (#63899) When exporting and cloning ml configurations in a cluster it can be frustrating to remove all the fields that were generated by the plugin. Especially as the number of these fields change from version to version. This flag, exclude_generated, allows the GET config APIs to return configurations with these generated fields removed. APIs supporting this flag: - GET _ml/anomaly_detection/<job_id> - GET _ml/datafeeds/<datafeed_id> - GET _ml/data_frame/analytics/<analytics_id> The following fields are not returned in the objects: - any field that is not user settable (e.g. version, create_time) - any field that is a calculated default value (e.g. datafeed chunking_config) - any field that is automatically set via another Elastic stack process (e.g. anomaly job custom_settings.created_by) relates to #63055
pugnascotia
pushed a commit
to pugnascotia/elasticsearch
that referenced
this pull request
Oct 21, 2020
…in GET config APIs (elastic#63899) When exporting and cloning ml configurations in a cluster it can be frustrating to remove all the fields that were generated by the plugin. Especially as the number of these fields change from version to version. This flag, exclude_generated, allows the GET config APIs to return configurations with these generated fields removed. APIs supporting this flag: - GET _ml/anomaly_detection/<job_id> - GET _ml/datafeeds/<datafeed_id> - GET _ml/data_frame/analytics/<analytics_id> The following fields are not returned in the objects: - any field that is not user settable (e.g. version, create_time) - any field that is a calculated default value (e.g. datafeed chunking_config) - any field that is automatically set via another Elastic stack process (e.g. anomaly job custom_settings.created_by) relates to elastic#63055
pugnascotia
changed the title
exclude_generated that removes generated fields in GET config APIs
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
When exporting and cloning ml configurations in a cluster it can be
frustrating to remove all the fields that were generated by
the plugin. Especially as the number of these fields change
from version to version.
This flag, exclude_generated, allows the GET config APIs to return
configurations with these generated fields removed.
APIs supporting this flag:
The following fields are not returned in the objects:
relates to #63055