[ML] AIOps: Adds dip support to log rate analysis in ML AIOps Labs #163100

walterra · 2023-08-03T16:44:24Z

Summary

This updates log rate analysis to be able to auto-detect whether the selected deviation is a spike or dip compared to the baseline time range. To achieve this, we compare the median bucket size of the two selections. If a dip gets detected, the analysis will then switch the window parameters sent to the API endpoint to run the analysis.

An info callout points out the auto-selected analysis type and explains to which time range the analysis results refer to. We need to do this to make it clear that for dip analysis the significant terms and their doc counts refer to the baseline time range and vice versa for spike analysis.

Log rate spike

Log rate dip

Functional tests

The artificial logs dataset generator for functional tests was updated to be able to also produce a dataset with a dip. Functional tests have been added to make use of that:

Observability Alert Details Page

In Observability, since we now auto-detect the analysis type, we no longer need to pass on the analysis type in the alert details page from the alert context. Instead, the analysis type will be part of the onAnalysisComplete() callback. The prompt for the AI assistant was updated to include the information about spike/dip that's also present in the in callout to users.

Checklist

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
Unit or functional tests were updated or added to match the most common scenarios
This was checked for breaking API changes and was labeled appropriately

elasticmachine · 2023-08-04T14:04:01Z

Pinging @elastic/ml-ui (:ml)

peteharverson

Overall looks good - just tested inside the ML AIOps Labs page. All the examples I ran correctly detected if it was a dip or a spike.

Left a few comments, mostly related to the text.

x-pack/plugins/aiops/public/components/log_rate_analysis/log_rate_analysis_results.tsx

...alerting/log_threshold/components/alert_details_app_section/components/log_rate_analysis.tsx

peteharverson · 2023-08-07T09:29:24Z

x-pack/plugins/aiops/public/components/log_rate_analysis/log_rate_analysis_results.tsx

+                  })
+                : i18n.translate('xpack.aiops.analysis.analysisTypeDipCallOutContent', {
+                    defaultMessage:
+                      'The median log rate in the selected deviation time range is lower than the baseline. Therefore, the analysis results table shows statistically significant items within the baseline time range that are less present or missing within the deviation time range. The "doc count" column refers to the amount of documents in the baseline time range.',


less in number better than less present? Any thoughts @szabosteve ?

Updated in 4f8c71a.

Sorry, I'm late to the party! Yes, less in number is a clearer solution. Thanks for updating!

...alerting/log_threshold/components/alert_details_app_section/components/log_rate_analysis.tsx

peteharverson · 2023-08-07T09:38:22Z

x-pack/packages/ml/aiops_utils/log_rate_histogram_item.ts

+   */
+  time: number | string;
+  /**
+   * Number of doc count for that time bucket


Are we able to calculate how the doc count (per bucket) for the deviation compares to the baseline? I end up wanting to know how the counts compare, rather than just a single count number for the baseline / deviation.

Yes I thought about that too, it would be good to have both a baseline and deviation column in the table. To make the numbers comparable it would be good to show median per bucket (to use the same measure we use to define if it's spike or dip). I added an item to the meta issue, I'd like to add that in a separate PR: #160247

weltenwort

deferring to the @elastic/actionable-observability team for review of the alerting-related changes

x-pack/packages/ml/aiops_components/src/document_count_chart/document_count_chart.tsx

peteharverson

Latest edits LGTM

qn895 · 2023-08-08T14:34:18Z

LGTM 🎉

kibana-ci · 2023-08-08T14:45:05Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: f289fc8

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #53 / Machine Learning modules get_module lists all modules
[job] [logs] FTR Configs #56 / spaces api with security resolve copy to spaces conflicts rbac user with all globally from the default space single-namespace types "before each" hook for "should return 200 when not overwriting, with references"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`aiops`	439	441	+2
`dataVisualizer`	538	540	+2
`infra`	1385	1386	+1
total			+5

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
`@kbn/aiops-components`	6	0	-6

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`aiops`	540.3KB	542.3KB	+2.0KB
`dataVisualizer`	604.6KB	605.0KB	+394.0B
`infra`	2.0MB	2.0MB	+691.0B
total			+3.1KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
`@kbn/aiops-components`	1	0	-1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`aiops`	6.2KB	6.0KB	-206.0B

Unknown metric groups

API count

id	before	after	diff
`@kbn/aiops-components`	30	33	+3
`@kbn/aiops-utils`	12	20	+8
`aiops`	60	57	-3
total			+8

History

💚 Build #148092 succeeded 4f8c71a
💚 Build #147829 succeeded 7a248a1
💔 Build #147807 failed 4c646a4
💔 Build #147774 failed e4eaafe
💚 Build #147535 succeeded 4e314f7
💚 Build #147482 succeeded eaa2330

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @walterra

benakansara

LGTM

benakansara · 2023-08-08T19:46:24Z

...alerting/log_threshold/components/alert_details_app_section/components/log_rate_analysis.tsx

@@ -60,11 +58,9 @@ export const LogRateAnalysis: FC<AlertDetailsLogRateAnalysisSectionProps> = ({ r
  const [dataView, setDataView] = useState<DataView | undefined>();
  const [esSearchQuery, setEsSearchQuery] = useState<QueryDslQueryContainer | undefined>();
  const [logRateAnalysisParams, setLogRateAnalysisParams] = useState<
-    { significantFieldValues: SignificantFieldValue[] } | undefined
+    | { logRateAnalysisType: LogRateAnalysisType; significantFieldValues: SignificantFieldValue[] }


nit: There is an extra | here.

That's caused by our linting rules because the first item starts at a new line.

elastic#163100) This updates log rate analysis to be able to auto-detect whether the selected deviation is a spike or dip compared to the baseline time range. To achieve this, we compare the median bucket size of the two selections. If a dip gets detected, the analysis will then switch the window parameters sent to the API endpoint to run the analysis. An info callout points out the auto-selected analysis type and explains to which time range the analysis results refer to. We need to do this to make it clear that for dip analysis the significant terms and their doc counts refer to the baseline time range and vice versa for spike analysis.

walterra self-assigned this Aug 3, 2023

walterra force-pushed the 161832-ml-aiops-detect-spike-or-dip branch 3 times, most recently from 34691f9 to c7d80fe Compare August 3, 2023 20:15

walterra added 9 commits August 3, 2023 22:35

auto-detect if spike or dip selection

46cccc6

fix types

91097d5

break out getLogRateAnalysisType into own file and add unit tests

fdb93e5

adds functional test with artificial logs dataset with dip

e2b4ab7

replace plain string

df6002a

assert the analysis type once the analysis completes

53f0ef4

fix missing comments/exports

883f197

linting scripts

226af34

fix circular dependency

2223676

walterra force-pushed the 161832-ml-aiops-detect-spike-or-dip branch from 3297e7e to 2223676 Compare August 3, 2023 20:37

walterra added 4 commits August 4, 2023 13:37

fix jsdoc

71aa5d0

switch to deep import

eaa2330

more granular code org to avoid bundle bloat

543661a

update ai assistant prompt for spike/dip

4e314f7

walterra added release_note:enhancement :ml Feature:ML/AIOps ML AIOps features: Change Point Detection, Log Pattern Analysis, Log Rate Analysis v8.10.0 labels Aug 4, 2023

walterra mentioned this pull request Aug 1, 2023

[ML] AIOps: Support both spikes and dips for log rate analysis. #161832

Closed

3 tasks

walterra marked this pull request as ready for review August 4, 2023 14:03

walterra requested review from a team as code owners August 4, 2023 14:03

walterra requested review from alvarezmelissa87 and peteharverson August 4, 2023 14:04

walterra requested a review from qn895 August 4, 2023 14:04

walterra added 2 commits August 7, 2023 08:15

Merge branch 'main' into 161832-ml-aiops-detect-spike-or-dip

e4eaafe

Merge branch 'main' into 161832-ml-aiops-detect-spike-or-dip

4c646a4

peteharverson mentioned this pull request Aug 7, 2023

[ML] Increase Test Coverage 8.10.0 #160712

Closed

13 tasks

Merge branch 'main' into 161832-ml-aiops-detect-spike-or-dip

7a248a1

peteharverson reviewed Aug 7, 2023

View reviewed changes

weltenwort approved these changes Aug 7, 2023

View reviewed changes

qn895 reviewed Aug 7, 2023

View reviewed changes

x-pack/packages/ml/aiops_components/src/document_count_chart/document_count_chart.tsx Outdated Show resolved Hide resolved

qn895 reviewed Aug 7, 2023

View reviewed changes

x-pack/packages/ml/aiops_components/src/document_count_chart/document_count_chart.tsx Outdated Show resolved Hide resolved

walterra added 3 commits August 7, 2023 20:22

Merge branch 'main' into 161832-ml-aiops-detect-spike-or-dip

83039e6

BrushSelectionUpdateHandler

f638002

text tweaks

4f8c71a

peteharverson approved these changes Aug 8, 2023

View reviewed changes

walterra added 2 commits August 8, 2023 15:30

Merge branch 'main' into 161832-ml-aiops-detect-spike-or-dip

b907633

fix timestamp creation for artificial datasets

f289fc8

qn895 approved these changes Aug 8, 2023

View reviewed changes

benakansara approved these changes Aug 8, 2023

View reviewed changes

walterra merged commit da0fb1d into elastic:main Aug 9, 2023

walterra deleted the 161832-ml-aiops-detect-spike-or-dip branch August 9, 2023 06:05

walterra mentioned this pull request Aug 9, 2023

[ML] AIOps Log Rate Analysis: Allow the baseline selection window to be set after the deviation window. #154229

Closed

peteharverson changed the title ~~[ML] AIOps: Auto-detect if spike or dip selected in log rate analysis.~~ [ML] AIOps: Adds dip support log rate analysis in ML AIOps Labs Aug 22, 2023

peteharverson changed the title ~~[ML] AIOps: Adds dip support log rate analysis in ML AIOps Labs~~ [ML] AIOps: Adds dip support to log rate analysis in ML AIOps Labs Aug 22, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ML] AIOps: Adds dip support to log rate analysis in ML AIOps Labs #163100

[ML] AIOps: Adds dip support to log rate analysis in ML AIOps Labs #163100

walterra commented Aug 3, 2023 •

edited

Loading

elasticmachine commented Aug 4, 2023

peteharverson left a comment

peteharverson Aug 7, 2023

walterra Aug 7, 2023

szabosteve Aug 8, 2023

peteharverson Aug 7, 2023

walterra Aug 7, 2023

weltenwort left a comment

peteharverson left a comment

qn895 commented Aug 8, 2023

kibana-ci commented Aug 8, 2023

API count

benakansara left a comment

benakansara Aug 8, 2023

walterra Aug 9, 2023

[ML] AIOps: Adds dip support to log rate analysis in ML AIOps Labs #163100

[ML] AIOps: Adds dip support to log rate analysis in ML AIOps Labs #163100

Conversation

walterra commented Aug 3, 2023 • edited Loading

Summary

Log rate spike

Log rate dip

Functional tests

Observability Alert Details Page

Checklist

elasticmachine commented Aug 4, 2023

peteharverson left a comment

Choose a reason for hiding this comment

peteharverson Aug 7, 2023

Choose a reason for hiding this comment

walterra Aug 7, 2023

Choose a reason for hiding this comment

szabosteve Aug 8, 2023

Choose a reason for hiding this comment

peteharverson Aug 7, 2023

Choose a reason for hiding this comment

walterra Aug 7, 2023

Choose a reason for hiding this comment

weltenwort left a comment

Choose a reason for hiding this comment

peteharverson left a comment

Choose a reason for hiding this comment

qn895 commented Aug 8, 2023

kibana-ci commented Aug 8, 2023

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

Module Count

Public APIs missing comments

Async chunks

Public APIs missing exports

Page load bundle

API count

History

benakansara left a comment

Choose a reason for hiding this comment

benakansara Aug 8, 2023

Choose a reason for hiding this comment

walterra Aug 9, 2023

Choose a reason for hiding this comment

walterra commented Aug 3, 2023 •

edited

Loading