Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When a rule is running, the run history displays an empty row #129126

Closed
mikecote opened this issue Mar 31, 2022 · 3 comments · Fixed by #129151
Closed

When a rule is running, the run history displays an empty row #129126

mikecote opened this issue Mar 31, 2022 · 3 comments · Fixed by #129151
Assignees
Labels
Feature:Alerting/RulesManagement Issues related to the Rules Management UX Feature:EventLog Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)

Comments

@mikecote
Copy link
Contributor

Whenever a rule is running, an empty row is displayed at the end of the run history (last page).

Steps to reproduce:

  • Add a slowdown loop within a rule type
  • Wait for the rule to run
  • Navigate to the run history tab on the last page
  • Notice an empty row

Code to slow down rule type runs (paste within rule type runner)

console.log(new Date(), '*** waiting 30s');
await new Promise((resolve) => setTimeout(resolve, 30000));

Screenshot:
Screen Shot 2022-03-31 at 2 52 23 PM

@mikecote mikecote added Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams) Feature:EventLog Feature:Alerting/RulesManagement Issues related to the Rules Management UX labels Mar 31, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/response-ops (Team:ResponseOps)

@ymao1
Copy link
Contributor

ymao1 commented Apr 1, 2022

Nice find! This is happening because the execute event is not written until rule execution finishes but there is an execute-start event that is logged with an execution UUID when rule execution starts. Because we're grouping by UUID but getting most of our information (including timestamp) from the execute event, we're returning empty information, with just a UUID

Example response:

{
    "id": "1227a277-738c-4303-991f-ba2bc5066d2b",
    "timestamp": "",
    "duration_ms": 0,
    "status": "",
    "message": "",
    "num_active_alerts": 0,
    "num_new_alerts": 0,
    "num_recovered_alerts": 0,
    "num_triggered_actions": 0,
    "num_scheduled_actions": 0,
    "num_succeeded_actions": 0,
    "num_errored_actions": 0,
    "total_search_duration_ms": 0,
    "es_search_duration_ms": 0,
    "schedule_delay_ms": 0,
    "timed_out": false
}

We have a few options:

  1. Filter out these rows with empty timestamp in the response (either client or server side). This would make the pagination a little wonky
  2. Update the aggregation to exclude execute-start events. These events don't provide any information that is not available in the execute event so we wouldn't be losing any information. OTOH, it may be useful to know that there are execute-start events with no matching execute event because that means something catastrophic happened during rule execution (or more likely Kibana was restarted).
  3. Update the response to return a start_time field that represents the timestamp from the execute-start event. Then the response would have a start_time and an empty timestamp. I think this is only worth doing if we show something in the UI for these rows. And likely, we would want to show something different if the newest row entry has this format (like rule is currently running vs something happened during rule execution

I think for a bug fix, we should consider doing #2 and then perhaps add #3 as a feature later

@mikecote @XavierM @JiaweiWu

@mikecote
Copy link
Contributor Author

mikecote commented Apr 1, 2022

I'm +1 to go with option 2 for now 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Alerting/RulesManagement Issues related to the Rules Management UX Feature:EventLog Team:ResponseOps Label for the ResponseOps team (formerly the Cases and Alerting teams)
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

3 participants