[Reporting] Add "warning" status as an alternate type of completed job #63498

tsullivan · 2020-04-14T17:46:30Z

Summary

This PR has 2 purposes:

Promote "completed_with_warnings" as a new kind of status. This will stop a bug before it happens since the warning state doesn't have to be derived by looking at multiple fields in the source document.
Reporting jobs that have warnings can now be counted by doing a terms aggregation on the job status, so the count of reports with warnings now appears in Telemetry data:

"reporting": {
  "_all": 30,
  "status": {
    "completed": 26,
    "failed": 3,
    "completed_with_warnings": 1
  },
  "printable_pdf": {
    "available": true,
    "total": 11,
    "app": {
      "dashboard": 0,
      "visualization": 4,
      "canvas workpad": 7
    },
    "layout": {
      "preserve_layout": 11,
      "print": 0
    }
  },
  "csv": {
    "available": true,
    "total": 0
  },
  "PNG": {
    "available": true,
    "total": 19
  }
}

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
~~[ ] Documentation was added for features that require explanation or tutorials~~
Unit or functional tests were updated or added to match the most common scenarios

elasticmachine · 2020-04-14T17:47:17Z

Pinging @elastic/kibana-reporting-services (Team:Reporting Services)

x-pack/legacy/plugins/reporting/server/lib/esqueue/worker.js

tsullivan · 2020-04-14T21:32:41Z

x-pack/plugins/reporting/public/components/report_listing.test.tsx

+      { _id: 'k9065q3s1d4wcbae0c00fxlh', _index: '.reporting-2020.04.12', _score: null, _source: { attempts: 1, browser_type: 'chromium', completed_at: '2020-04-14T17:18:03.910Z', created_at: '2020-04-14T17:17:47.752Z', created_by: 'elastic', jobtype: 'printable_pdf', kibana_id: '5b2de169-2785-441b-ae8c-186a1936b17d', kibana_name: 'spicy.local', max_attempts: 1, meta: { layout: 'preserve_layout', objectType: 'canvas workpad', }, output: { content_type: 'application/pdf', size: 80262, }, payload: { basePath: '/kbn', browserTimezone: 'America/Phoenix', forceNow: '2020-04-14T17:17:47.750Z', layout: { dimensions: { height: 720, width: 1080, }, id: 'preserve_layout', }, objectType: 'canvas workpad', relativeUrls: [ '/s/hsyjklk/app/canvas#/export/workpad/pdf/workpad-53d38306-eda4-410f-b5e7-efbeca1a8c63/page/1', ], title: 'My Canvas Workpad', }, priority: 10, process_expiration: '2020-04-14T17:22:50.379Z', started_at: '2020-04-14T17:17:50.379Z', status: 'completed', timeout: 300000, }, sort: [1586884667752], },
+      { _id: 'k905zdw11d34cbae0c3y6tzh', _index: '.reporting-2020.04.12', _score: null, _source: { attempts: 1, browser_type: 'chromium', completed_at: '2020-04-14T17:13:03.719Z', created_at: '2020-04-14T17:12:51.985Z', created_by: 'elastic', jobtype: 'printable_pdf', kibana_id: '5b2de169-2785-441b-ae8c-186a1936b17d', kibana_name: 'spicy.local', max_attempts: 1, meta: { layout: 'preserve_layout', objectType: 'canvas workpad', }, output: { content_type: 'application/pdf', size: 80262, }, payload: { basePath: '/kbn', browserTimezone: 'America/Phoenix', forceNow: '2020-04-14T17:12:51.984Z', layout: { dimensions: { height: 720, width: 1080, }, id: 'preserve_layout', }, objectType: 'canvas workpad', relativeUrls: [ '/s/hsyjklk/app/canvas#/export/workpad/pdf/workpad-53d38306-eda4-410f-b5e7-efbeca1a8c63/page/1', ], title: 'My Canvas Workpad', }, priority: 10, process_expiration: '2020-04-14T17:17:52.431Z', started_at: '2020-04-14T17:12:52.431Z', status: 'completed', timeout: 300000, }, sort: [1586884371985], },
+      { _id: 'k8t4ylcb07mi9d006214ifyg', _index: '.reporting-2020.04.05', _score: null, _source: { attempts: 1, browser_type: 'chromium', completed_at: '2020-04-09T19:10:10.049Z', created_at: '2020-04-09T19:09:52.139Z', created_by: 'elastic', jobtype: 'PNG', kibana_id: 'f2e59b4e-f79b-4a48-8a7d-6d50a3c1d914', kibana_name: 'spicy.local', max_attempts: 1, meta: { layout: 'png', objectType: 'visualization', }, output: { content_type: 'image/png', }, payload: { basePath: '/kbn', browserTimezone: 'America/Phoenix', forceNow: '2020-04-09T19:09:52.137Z', layout: { dimensions: { height: 1575, width: 1423, }, id: 'png', }, objectType: 'visualization', relativeUrl: "/s/hsyjklk/app/kibana#/visualize/edit/94d1fe40-7a94-11ea-b373-0749f92ad295?_a=(filters:!(),linked:!f,query:(language:kuery,query:''),uiState:(),vis:(aggs:!((enabled:!t,id:'1',params:(),schema:metric,type:count)),params:(addLegend:!f,addTooltip:!t,metric:(colorSchema:'Green%20to%20Red',colorsRange:!((from:0,to:10000)),invertColors:!f,labels:(show:!t),metricColorMode:None,percentageMode:!f,style:(bgColor:!f,bgFill:%23000,fontSize:60,labelColor:!f,subText:''),useRanges:!f),type:metric),title:count,type:metric))&_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15y,to:now))&indexPattern=d81752b0-7434-11ea-be36-1f978cda44d4&type=metric", title: 'count', }, priority: 10, process_expiration: '2020-04-09T19:14:54.570Z', started_at: '2020-04-09T19:09:54.570Z', status: 'completed', timeout: 300000, }, sort: [1586459392139], },
+    ]), // prettier-ignore


I noticed the snapshot files didn't have to be updated after I changed all the job rows, which means there is a gap in this Enzyme test, and it is not testing everything that it should. But I am not addressing that in this PR

x-pack/legacy/plugins/reporting/server/lib/esqueue/worker.js

joelgriffith · 2020-04-14T22:40:00Z

x-pack/legacy/plugins/reporting/server/routes/lib/get_document_payload.ts

@@ -9,6 +9,7 @@ import contentDisposition from 'content-disposition';
 import * as _ from 'lodash';
 import { CSV_JOB_TYPE } from '../../../common/constants';
 import { ExportTypeDefinition, ExportTypesRegistry, JobDocOutput, JobSource } from '../../../types';
+import { statuses } from '../../lib/esqueue/constants/statuses';


x-pack/plugins/reporting/index.d.ts

joelgriffith

Few small nits, happy to go either way. Having the types in our root type file would be nice.

kibanamachine · 2020-04-15T20:36:32Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 74b48d9

History

💚 Build #40813 succeeded 3d1b36c
💔 Build #40745 failed 978f951

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

elastic#63498) * [Reporting] Add "warning" as a status * test * fix warning status handling * Simplify logic * fix syntax * more different statuses * fix warning * feedbacks

#63498) (#63647) * [Reporting] Add "warning" as a status * test * fix warning status handling * Simplify logic * fix syntax * more different statuses * fix warning * feedbacks

* master: (56 commits) [i18n] Update CODEOWNERS (elastic#63354) add platform team definition of done (elastic#59993) [SIEM] move away from Joi for importing/exporting timeline (elastic#62125) Fix discover preserve url (elastic#63580) [alerting] Adds an alertServices mock and uses it in siem, monitoring and uptime (elastic#63489) Closes elastic#63109 for Service Map by resetting edges styles for the selected node (elastic#63655) MIgrated index_header to react (elastic#63490) Index pattern management UI -> TypeScript and New Platform Ready (indexed_fields_table) (elastic#63364) [SIEM] [Cases] Insert timeline and reporters/tags in table bug fixes (elastic#63642) [Reporting] Make usable default element positions (elastic#63191) [Reporting] Switch Serverside Config Wrapper to NP (elastic#62500) [Reporting] Add "warning" status as an alternate type of completed job (elastic#63498) Split action types into own page (elastic#63516) [Lens] Only show copy on save for previously saved docs (elastic#63535) Update README.md (elastic#63622) Bugfix clear saved query crashes kibana on Discover in some cases (elastic#63554) Add uptime CODEOWNER entries. (elastic#63616) [ML] Extract apiDoc params from the schema definitions (elastic#62933) Fix alerting documentation encryption key requirement (elastic#63512) Fix CODEOWNERS and sass lint paths (elastic#63552) ...

tsullivan added 3 commits April 13, 2020 17:24

[Reporting] Add "warning" as a status

70d0933

test

7852569

fix warning status handling

978f951

tsullivan added release_note:skip Skip the PR/issue when compiling release notes v7.8.0 v8.0.0 Team:Reporting Services labels Apr 14, 2020

tsullivan marked this pull request as draft April 14, 2020 17:47

joelgriffith reviewed Apr 14, 2020

View reviewed changes

x-pack/legacy/plugins/reporting/server/lib/esqueue/worker.js Outdated Show resolved Hide resolved

joelgriffith mentioned this pull request Apr 14, 2020

[Reporting] Capture a screenshot when PDFs are near timeouts #63294

Closed

tsullivan added 4 commits April 14, 2020 13:26

Simplify logic

1c356b6

fix syntax

23064ef

more different statuses

362f44f

fix warning

3d1b36c

tsullivan marked this pull request as ready for review April 14, 2020 21:16

tsullivan commented Apr 14, 2020

View reviewed changes

joelgriffith reviewed Apr 14, 2020

View reviewed changes

x-pack/legacy/plugins/reporting/server/lib/esqueue/worker.js Outdated Show resolved Hide resolved

joelgriffith reviewed Apr 14, 2020

View reviewed changes

x-pack/plugins/reporting/index.d.ts Show resolved Hide resolved

joelgriffith suggested changes Apr 14, 2020

View reviewed changes

tsullivan added 2 commits April 15, 2020 11:36

Merge branch 'master' into reporting/job-warning-status

6ac297d

feedbacks

74b48d9

joelgriffith approved these changes Apr 15, 2020

View reviewed changes

tsullivan merged commit 23e3f1a into elastic:master Apr 15, 2020

tsullivan mentioned this pull request Apr 15, 2020

[7.x] [Reporting] Add "warning" status as an alternate type of completed job (#63498) #63647

Merged

tsullivan deleted the reporting/job-warning-status branch June 25, 2020 18:09

sophiec20 added the (Deprecated) Feature:Reporting Use Reporting:Screenshot, Reporting:CSV, or Reporting:Framework instead label Aug 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Reporting] Add "warning" status as an alternate type of completed job #63498

[Reporting] Add "warning" status as an alternate type of completed job #63498

tsullivan commented Apr 14, 2020

elasticmachine commented Apr 14, 2020

tsullivan Apr 14, 2020

joelgriffith Apr 14, 2020

joelgriffith left a comment

kibanamachine commented Apr 15, 2020

[Reporting] Add "warning" status as an alternate type of completed job #63498

[Reporting] Add "warning" status as an alternate type of completed job #63498

Conversation

tsullivan commented Apr 14, 2020

Summary

Checklist

elasticmachine commented Apr 14, 2020

tsullivan Apr 14, 2020

Choose a reason for hiding this comment

joelgriffith Apr 14, 2020

Choose a reason for hiding this comment

joelgriffith left a comment

Choose a reason for hiding this comment

kibanamachine commented Apr 15, 2020

💚 Build Succeeded

History