Skip to content

Commit

Permalink
Migrate rollup client side code (#63227) (#63869)
Browse files Browse the repository at this point in the history
  • Loading branch information
flash1293 authored Apr 20, 2020
1 parent 1718df6 commit bbad5ca
Show file tree
Hide file tree
Showing 143 changed files with 288 additions and 254 deletions.
2 changes: 1 addition & 1 deletion x-pack/.i18nrc.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
"xpack.remoteClusters": "plugins/remote_clusters",
"xpack.painlessLab": "plugins/painless_lab",
"xpack.reporting": ["plugins/reporting", "legacy/plugins/reporting"],
"xpack.rollupJobs": "legacy/plugins/rollup",
"xpack.rollupJobs": ["legacy/plugins/rollup", "plugins/rollup"],
"xpack.searchProfiler": "plugins/searchprofiler",
"xpack.security": ["legacy/plugins/security", "plugins/security"],
"xpack.server": "legacy/server",
Expand Down
12 changes: 6 additions & 6 deletions x-pack/legacy/plugins/rollup/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,30 +14,30 @@ The rest of this doc dives into the implementation details of each of the above

## Create and manage rollup jobs

The most straight forward part of this plugin! A new app called Rollup Jobs is registered in the Management section and follows a typical CRUD UI pattern. This app allows users to create, start, stop, clone, and delete rollup jobs. There is no way to edit an existing rollup job; instead, the UI offers a cloning ability. The client-side portion of this app lives [here](public/crud_app) and uses endpoints registered [here](server/routes/api/jobs.js).
The most straight forward part of this plugin! A new app called Rollup Jobs is registered in the Management section and follows a typical CRUD UI pattern. This app allows users to create, start, stop, clone, and delete rollup jobs. There is no way to edit an existing rollup job; instead, the UI offers a cloning ability. The client-side portion of this app lives [here](../../../plugins/rollup/public/crud_app) and uses endpoints registered [here](server/routes/api/jobs.js).

Refer to the [Elasticsearch documentation](https://www.elastic.co/guide/en/elasticsearch/reference/current/rollup-getting-started.html) to understand rollup indices and how to create rollup jobs.

## Create rollup index patterns

Kibana uses index patterns to consume and visualize rollup indices. Typically, Kibana can inspect the indices captured by an index pattern, identify its aggregations and fields, and determine how to consume the data. Rollup indices don't contain this type of information, so we predefine how to consume a rollup index pattern with the type and typeMeta fields on the index pattern saved object. All rollup index patterns have `type` defined as "rollup" and `typeMeta` defined as an object of the index pattern's capabilities.

In the Index Pattern app, the "Create index pattern" button includes a context menu when a rollup index is detected. This menu offers items for creating a standard index pattern and a rollup index pattern. A [rollup config is registered to index pattern creation extension point](public/index_pattern_creation/rollup_index_pattern_creation_config.js). The context menu behavior in particular uses the `getIndexPatternCreationOption()` method. When the user chooses to create a rollup index pattern, this config changes the behavior of the index pattern creation wizard:
In the Index Pattern app, the "Create index pattern" button includes a context menu when a rollup index is detected. This menu offers items for creating a standard index pattern and a rollup index pattern. A [rollup config is registered to index pattern creation extension point](../../../plugins/rollup/public/index_pattern_creation/rollup_index_pattern_creation_config.js). The context menu behavior in particular uses the `getIndexPatternCreationOption()` method. When the user chooses to create a rollup index pattern, this config changes the behavior of the index pattern creation wizard:

1. Adds a `Rollup` badge to rollup indices using `getIndexTags()`.
2. Enforces index pattern rules using `checkIndicesForErrors()`. Rollup index patterns must match **one** rollup index, and optionally, any number of regular indices. A rollup index pattern configured with one or more regular indices is known as a "hybrid" index pattern. This allows the user to visualize historical (rollup) data and live (regular) data in the same visualization.
3. Routes to this plugin's [rollup `_fields_for_wildcard` endpoint](server/routes/api/index_patterns.js), instead of the standard one, using `getFetchForWildcardOptions()`, so that the internal rollup data field names are mapped to the original field names.
4. Writes additional information about aggregations, fields, histogram interval, and date histogram interval and timezone to the rollup index pattern saved object using `getIndexPatternMappings()`. This collection of information is referred to as its "capabilities".

Once a rollup index pattern is created, it is tagged with `Rollup` in the list of index patterns, and its details page displays capabilities information. This is done by registering [yet another config for the index pattern list](public/index_pattern_list/rollup_index_pattern_list_config.js) extension points.
Once a rollup index pattern is created, it is tagged with `Rollup` in the list of index patterns, and its details page displays capabilities information. This is done by registering [yet another config for the index pattern list](../../../plugins/rollup/public/index_pattern_list/rollup_index_pattern_list_config.js) extension points.

## Create visualizations from rollup index patterns

This plugin enables the user to create visualizations from rollup data using the Visualize app, excluding TSVB, Vega, and Timelion. When Visualize sends search requests, this plugin routes the requests to the [Elasticsearch rollup search endpoint](https://www.elastic.co/guide/en/elasticsearch/reference/current/rollup-search.html), which searches the special document structure within rollup indices. The visualization options available to users are based on the capabilities of the rollup index pattern they're visualizing.

Routing to the Elasticsearch rollup search endpoint is done by creating an extension point in Courier, effectively allowing multiple "search strategies" to be registered. A [rollup search strategy](public/search/register.js) is registered by this plugin that queries [this plugin's rollup search endpoint](server/routes/api/search.js).
Routing to the Elasticsearch rollup search endpoint is done by creating an extension point in Courier, effectively allowing multiple "search strategies" to be registered. A [rollup search strategy](../../../plugins/rollup/public/search/register.js) is registered by this plugin that queries [this plugin's rollup search endpoint](server/routes/api/search.js).

Limiting visualization editor options is done by [registering configs](public/visualize/index.js) to various vis extension points. These configs use information stored on the rollup index pattern to limit:
Limiting visualization editor options is done by [registering configs](../../../plugins/rollup/public/visualize/index.js) to various vis extension points. These configs use information stored on the rollup index pattern to limit:
* Available aggregation types
* Available fields for a particular aggregation
* Default and base interval for histogram aggregation
Expand All @@ -47,6 +47,6 @@ Limiting visualization editor options is done by [registering configs](public/vi

In Index Management, similar to system indices, rollup indices are hidden by default. A toggle is provided to show rollup indices and add a badge to the table rows. This is done by using Index Management's extension points.

The toggle and badge are registered on client-side [here](public/extend_index_management/index.js).
The toggle and badge are registered on client-side [here](../../../plugins/rollup/public/extend_index_management/index.js).

Additional data needed to filter rollup indices in Index Management is provided with a [data enricher](rollup_data_enricher.js).
22 changes: 1 addition & 21 deletions x-pack/legacy/plugins/rollup/common/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,4 @@ export const PLUGIN = {
},
};

export const CONFIG_ROLLUPS = 'rollups:enableIndexPatterns';

export const API_BASE_PATH = '/api/rollup';

export {
UIM_APP_NAME,
UIM_APP_LOAD,
UIM_JOB_CREATE,
UIM_JOB_DELETE,
UIM_JOB_DELETE_MANY,
UIM_JOB_START,
UIM_JOB_START_MANY,
UIM_JOB_STOP,
UIM_JOB_STOP_MANY,
UIM_SHOW_DETAILS_CLICK,
UIM_DETAIL_PANEL_SUMMARY_TAB_CLICK,
UIM_DETAIL_PANEL_TERMS_TAB_CLICK,
UIM_DETAIL_PANEL_HISTOGRAM_TAB_CLICK,
UIM_DETAIL_PANEL_METRICS_TAB_CLICK,
UIM_DETAIL_PANEL_JSON_TAB_CLICK,
} from './ui_metric';
export * from '../../../../plugins/rollup/common';
26 changes: 1 addition & 25 deletions x-pack/legacy/plugins/rollup/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,40 +4,16 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { resolve } from 'path';
import { i18n } from '@kbn/i18n';
import { PluginInitializerContext } from 'src/core/server';
import { RollupSetup } from '../../../plugins/rollup/server';
import { PLUGIN, CONFIG_ROLLUPS } from './common';
import { PLUGIN } from './common';
import { plugin } from './server';

export function rollup(kibana: any) {
return new kibana.Plugin({
id: PLUGIN.ID,
configPrefix: 'xpack.rollup',
publicDir: resolve(__dirname, 'public'),
require: ['kibana', 'elasticsearch', 'xpack_main'],
uiExports: {
styleSheetPaths: resolve(__dirname, 'public/index.scss'),
managementSections: ['plugins/rollup/legacy'],
uiSettingDefaults: {
[CONFIG_ROLLUPS]: {
name: i18n.translate('xpack.rollupJobs.rollupIndexPatternsTitle', {
defaultMessage: 'Enable rollup index patterns',
}),
value: true,
description: i18n.translate('xpack.rollupJobs.rollupIndexPatternsDescription', {
defaultMessage: `Enable the creation of index patterns which capture rollup indices,
which in turn enable visualizations based on rollup data. Refresh
the page to apply the changes.`,
}),
category: ['rollups'],
},
},
indexManagement: ['plugins/rollup/legacy'],
visualize: ['plugins/rollup/legacy'],
search: ['plugins/rollup/legacy'],
},
init(server: any) {
const { core: coreSetup, plugins } = server.newPlatform.setup;
const { usageCollection, visTypeTimeseries, indexManagement } = plugins;
Expand Down
13 changes: 0 additions & 13 deletions x-pack/legacy/plugins/rollup/public/legacy.ts

This file was deleted.

27 changes: 27 additions & 0 deletions x-pack/plugins/rollup/common/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export const CONFIG_ROLLUPS = 'rollups:enableIndexPatterns';

export const API_BASE_PATH = '/api/rollup';

export {
UIM_APP_NAME,
UIM_APP_LOAD,
UIM_JOB_CREATE,
UIM_JOB_DELETE,
UIM_JOB_DELETE_MANY,
UIM_JOB_START,
UIM_JOB_START_MANY,
UIM_JOB_STOP,
UIM_JOB_STOP_MANY,
UIM_SHOW_DETAILS_CLICK,
UIM_DETAIL_PANEL_SUMMARY_TAB_CLICK,
UIM_DETAIL_PANEL_TERMS_TAB_CLICK,
UIM_DETAIL_PANEL_HISTOGRAM_TAB_CLICK,
UIM_DETAIL_PANEL_METRICS_TAB_CLICK,
UIM_DETAIL_PANEL_JSON_TAB_CLICK,
} from './ui_metric';
File renamed without changes.
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { getRandomString } from '../../../../test_utils';
import { getRandomString } from '../../../test_utils';

const initialValues = {
dateHistogramField: 'timestamp',
Expand Down
5 changes: 4 additions & 1 deletion x-pack/plugins/rollup/kibana.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,8 @@
"id": "rollup",
"version": "8.0.0",
"kibanaVersion": "kibana",
"server": true
"server": true,
"ui": true,
"optionalPlugins": ["home", "indexManagement", "indexPatternManagement", "usageCollection"],
"requiredPlugins": ["management", "data"]
}
Original file line number Diff line number Diff line change
Expand Up @@ -5,15 +5,17 @@
*/

import React from 'react';
import { ChromeBreadcrumb, CoreSetup } from 'kibana/public';
import { render, unmountComponentAtNode } from 'react-dom';
import { Provider } from 'react-redux';
import { KibanaContextProvider } from '../../../../../src/plugins/kibana_react/public';
import { ChromeBreadcrumb, CoreSetup } from '../../../../../src/core/public';
import { KibanaContextProvider } from '../../../../src/plugins/kibana_react/public';
// @ts-ignore
import { rollupJobsStore } from './crud_app/store';
// @ts-ignore
import { App } from './crud_app/app';

import './index.scss';

/**
* This module will be loaded asynchronously to reduce the bundle size of your plugin's main bundle.
*/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ import { HashRouter, Switch, Route, Redirect, withRouter } from 'react-router-do

import { UIM_APP_LOAD } from '../../common';
import { CRUD_APP_BASE_PATH } from './constants';
import { registerRouter, setUserHasLeftApp, trackUiMetric, METRIC_TYPE } from './services';
import { registerRouter, setUserHasLeftApp, METRIC_TYPE } from './services';
import { trackUiMetric } from '../kibana_services';
import { JobList, JobCreate } from './sections';

class ShareRouterComponent extends Component {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import first from 'lodash/array/first';
import { i18n } from '@kbn/i18n';
import { FormattedMessage } from '@kbn/i18n/react';

import { withKibana } from '../../../../../../../../src/plugins/kibana_react/public/';
import { withKibana } from '../../../../../../../src/plugins/kibana_react/public';

import {
EuiCallOut,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ import {
EuiTitle,
} from '@elastic/eui';

import { search } from '../../../../../../../../../src/plugins/data/public';
import { search } from '../../../../../../../../src/plugins/data/public';
const { parseEsInterval } = search.aggs;

import { getDateHistogramDetailsUrl, getDateHistogramAggregationUrl } from '../../../services';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ import {
EuiTitle,
} from '@elastic/eui';

import { CronEditor } from '../../../../../../../../../src/plugins/es_ui_shared/public';
import { indexPatterns } from '../../../../../../../../../src/plugins/data/public';
import { CronEditor } from '../../../../../../../../src/plugins/es_ui_shared/public';
import { indexPatterns } from '../../../../../../../../src/plugins/data/public';

import { indices } from '../../../../shared_imports';
import { getLogisticalDetailsUrl, getCronUrl } from '../../../services';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import cloneDeep from 'lodash/lang/cloneDeep';
import get from 'lodash/object/get';
import pick from 'lodash/object/pick';

import { WEEK } from '../../../../../../../../../src/plugins/es_ui_shared/public';
import { WEEK } from '../../../../../../../../src/plugins/es_ui_shared/public';

import { validateId } from './validate_id';
import { validateIndexPattern } from './validate_index_pattern';
Expand Down Expand Up @@ -42,7 +42,7 @@ export const stepIds = [
* 1. getDefaultFields: (overrides) => object
* 2. fieldValidations
*
* See x-pack/legacy/plugins/rollup/public/crud_app/services/jobs.js for more information on override's shape
* See x-pack/plugins/rollup/public/crud_app/services/jobs.js for more information on override's shape
*/
export const stepIdToStepConfigMap = {
[STEP_LOGISTICS]: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import React from 'react';
import { FormattedMessage } from '@kbn/i18n/react';
import { search } from '../../../../../../../../../src/plugins/data/public';
import { search } from '../../../../../../../../src/plugins/data/public';
const {
InvalidEsIntervalFormatError,
InvalidEsCalendarIntervalError,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
import React from 'react';
import { FormattedMessage } from '@kbn/i18n/react';

import { indexPatterns } from '../../../../../../../../../src/plugins/data/public';
import { indexPatterns } from '../../../../../../../../src/plugins/data/public';
export function validateIndexPattern(indexPattern, rollupIndex) {
if (!indexPattern || !indexPattern.trim()) {
return [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

import React from 'react';
import { FormattedMessage } from '@kbn/i18n/react';
import { search } from '../../../../../../../../../src/plugins/data/public';
import { search } from '../../../../../../../../src/plugins/data/public';
const {
InvalidEsIntervalFormatError,
InvalidEsCalendarIntervalError,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,8 @@ import {
UIM_DETAIL_PANEL_METRICS_TAB_CLICK,
UIM_DETAIL_PANEL_JSON_TAB_CLICK,
} from '../../../../../common';
import { trackUiMetric, METRIC_TYPE } from '../../../services';
import { METRIC_TYPE } from '../../../services';
import { trackUiMetric } from '../../../../kibana_services';

import {
JobActionMenu,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { registerTestBed } from '../../../../../../../../test_utils';
import { registerTestBed } from '../../../../../../../test_utils';
import { getJob } from '../../../../../fixtures';
import { rollupJobsStore } from '../../../store';
import { DetailPanel } from './detail_panel';
Expand All @@ -17,9 +17,8 @@ import {
tabToHumanizedMap,
} from '../../components';

jest.mock('ui/new_platform');
jest.mock('../../../services', () => {
const services = require.requireActual('../../../services');
jest.mock('../../../../kibana_services', () => {
const services = require.requireActual('../../../../kibana_services');
return {
...services,
trackUiMetric: jest.fn(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import {
EuiCallOut,
} from '@elastic/eui';

import { withKibana } from '../../../../../../../../src/plugins/kibana_react/public/';
import { withKibana } from '../../../../../../../src/plugins/kibana_react/public';

import { CRUD_APP_BASE_PATH } from '../../constants';
import { getRouterLinkProps, extractQueryParams, listBreadcrumb } from '../../services';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,14 @@
*/

import React from 'react';
import { registerTestBed } from '../../../../../../../test_utils';
import { registerTestBed } from '../../../../../../test_utils';
import { rollupJobsStore } from '../../store';
import { JobList } from './job_list';

import { KibanaContextProvider } from '../../../../../../../../src/plugins/kibana_react/public';
import { coreMock } from '../../../../../../../../src/core/public/mocks';
import { KibanaContextProvider } from '../../../../../../../src/plugins/kibana_react/public';
import { coreMock } from '../../../../../../../src/core/public/mocks';
const startMock = coreMock.createStart();

jest.mock('ui/new_platform');

jest.mock('../../services', () => {
const services = require.requireActual('../../services');
return {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ import {
} from '@elastic/eui';

import { UIM_SHOW_DETAILS_CLICK } from '../../../../../common';
import { trackUiMetric, METRIC_TYPE } from '../../../services';
import { METRIC_TYPE } from '../../../services';
import { trackUiMetric } from '../../../../kibana_services';
import { JobActionMenu, JobStatus } from '../../components';

const COLUMNS = [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,13 @@

import { Pager } from '@elastic/eui';

import { registerTestBed } from '../../../../../../../../test_utils';
import { registerTestBed } from '../../../../../../../test_utils';
import { getJobs, jobCount } from '../../../../../fixtures';
import { rollupJobsStore } from '../../../store';
import { JobTable } from './job_table';

jest.mock('ui/new_platform');
jest.mock('../../../services', () => {
const services = require.requireActual('../../../services');
jest.mock('../../../../kibana_services', () => {
const services = require.requireActual('../../../../kibana_services');
return {
...services,
trackUiMetric: jest.fn(),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { HttpStart } from 'src/core/public';
import { HttpStart } from 'kibana/public';

let _http: HttpStart | null = null;

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,4 +47,4 @@ export { sortTable } from './sort_table';

export { retypeMetrics } from './retype_metrics';

export { trackUiMetric, METRIC_TYPE } from './track_ui_metric';
export { METRIC_TYPE } from './track_ui_metric';
Loading

0 comments on commit bbad5ca

Please sign in to comment.