Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rollups] Server NP migration #55606

Merged
merged 19 commits into from
Feb 7, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions x-pack/legacy/plugins/rollup/common/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,22 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { LICENSE_TYPE_BASIC, LicenseType } from '../../../common/constants';

export const PLUGIN = {
ID: 'rollup',
MINIMUM_LICENSE_REQUIRED: LICENSE_TYPE_BASIC as LicenseType,
getI18nName: (i18n: any): string => {
return i18n.translate('xpack.rollupJobs.appName', {
defaultMessage: 'Rollup jobs',
});
},
};

export const CONFIG_ROLLUPS = 'rollups:enableIndexPatterns';

export const API_BASE_PATH = '/api/rollup';

export {
UIM_APP_NAME,
UIM_APP_LOAD,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,13 @@
*/

import { resolve } from 'path';
import { PLUGIN, CONFIG_ROLLUPS } from './common';
import { registerLicenseChecker } from './server/lib/register_license_checker';
import { rollupDataEnricher } from './rollup_data_enricher';
import { registerRollupSearchStrategy } from './server/lib/search_strategies';
import {
registerIndicesRoute,
registerFieldsForWildcardRoute,
registerSearchRoute,
registerJobsRoute,
} from './server/routes/api';
import { registerRollupUsageCollector } from './server/usage';
import { i18n } from '@kbn/i18n';
import { PluginInitializerContext } from 'src/core/server';
import { RollupSetup } from '../../../plugins/rollup/server';
import { PLUGIN, CONFIG_ROLLUPS } from './common';
import { plugin } from './server';

export function rollup(kibana) {
export function rollup(kibana: any) {
return new kibana.Plugin({
id: PLUGIN.ID,
configPrefix: 'xpack.rollup',
Expand All @@ -45,22 +38,30 @@ export function rollup(kibana) {
visualize: ['plugins/rollup/legacy'],
search: ['plugins/rollup/legacy'],
},
init: function(server) {
const { usageCollection } = server.newPlatform.setup.plugins;
registerLicenseChecker(server);
registerIndicesRoute(server);
registerFieldsForWildcardRoute(server);
registerSearchRoute(server);
registerJobsRoute(server);
registerRollupUsageCollector(usageCollection, server);
if (
server.plugins.index_management &&
server.plugins.index_management.addIndexManagementDataEnricher
) {
server.plugins.index_management.addIndexManagementDataEnricher(rollupDataEnricher);
}
init(server: any) {
const { core: coreSetup, plugins } = server.newPlatform.setup;
const { usageCollection, metrics } = plugins;

const rollupSetup = (plugins.rollup as unknown) as RollupSetup;

registerRollupSearchStrategy(this.kbnServer);
const initContext = ({
config: rollupSetup.__legacy.config,
logger: rollupSetup.__legacy.logger,
} as unknown) as PluginInitializerContext;

const rollupPluginInstance = plugin(initContext);

rollupPluginInstance.setup(coreSetup, {
usageCollection,
metrics,
__LEGACY: {
plugins: {
xpack_main: server.plugins.xpack_main,
rollup: server.plugins[PLUGIN.ID],
index_management: server.plugins.index_management,
},
},
});
},
});
}
14 changes: 14 additions & 0 deletions x-pack/legacy/plugins/rollup/kibana.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"id": "rollup",
"version": "kibana",
"requiredPlugins": [
"home",
"index_management",
"metrics"
],
"optionalPlugins": [
"usageCollection"
],
"server": true,
"ui": false
}
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
* you may not use this file except in compliance with the Elastic License.
*/

export const elasticsearchJsPlugin = (Client, config, components) => {
export const elasticsearchJsPlugin = (Client: any, config: any, components: any) => {
const ca = components.clientAction.factory;

Client.prototype.rollup = components.clientAction.namespaceFactory();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@
* you may not use this file except in compliance with the Elastic License.
*/

export { isEsErrorFactory } from './is_es_error_factory';
export { registerRollupUsageCollector } from './register';
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,31 @@
*/

import { get } from 'lodash';
import { UsageCollectionSetup } from 'src/plugins/usage_collection/server';
import { CallCluster } from 'src/legacy/core_plugins/elasticsearch';

interface IdToFlagMap {
[key: string]: boolean;
}

const ROLLUP_USAGE_TYPE = 'rollups';

// elasticsearch index.max_result_window default value
const ES_MAX_RESULT_WINDOW_DEFAULT_VALUE = 1000;

function getIdFromSavedObjectId(savedObjectId) {
function getIdFromSavedObjectId(savedObjectId: string) {
// The saved object ID is formatted `{TYPE}:{ID}`.
return savedObjectId.split(':')[1];
}

function createIdToFlagMap(ids) {
function createIdToFlagMap(ids: string[]) {
return ids.reduce((map, id) => {
map[id] = true;
return map;
}, {});
}, {} as any);
}

async function fetchRollupIndexPatterns(kibanaIndex, callCluster) {
async function fetchRollupIndexPatterns(kibanaIndex: string, callCluster: CallCluster) {
const searchParams = {
size: ES_MAX_RESULT_WINDOW_DEFAULT_VALUE,
index: kibanaIndex,
Expand All @@ -50,7 +56,11 @@ async function fetchRollupIndexPatterns(kibanaIndex, callCluster) {
});
}

async function fetchRollupSavedSearches(kibanaIndex, callCluster, rollupIndexPatternToFlagMap) {
async function fetchRollupSavedSearches(
kibanaIndex: string,
callCluster: CallCluster,
rollupIndexPatternToFlagMap: IdToFlagMap
) {
const searchParams = {
size: ES_MAX_RESULT_WINDOW_DEFAULT_VALUE,
index: kibanaIndex,
Expand Down Expand Up @@ -86,19 +96,19 @@ async function fetchRollupSavedSearches(kibanaIndex, callCluster, rollupIndexPat
const searchSource = JSON.parse(searchSourceJSON);

if (rollupIndexPatternToFlagMap[searchSource.index]) {
const id = getIdFromSavedObjectId(savedObjectId);
const id = getIdFromSavedObjectId(savedObjectId) as string;
rollupSavedSearches.push(id);
}

return rollupSavedSearches;
}, []);
}, [] as string[]);
}

async function fetchRollupVisualizations(
kibanaIndex,
callCluster,
rollupIndexPatternToFlagMap,
rollupSavedSearchesToFlagMap
kibanaIndex: string,
callCluster: CallCluster,
rollupIndexPatternToFlagMap: IdToFlagMap,
rollupSavedSearchesToFlagMap: IdToFlagMap
) {
const searchParams = {
size: ES_MAX_RESULT_WINDOW_DEFAULT_VALUE,
Expand Down Expand Up @@ -135,7 +145,7 @@ async function fetchRollupVisualizations(
savedSearchRefName,
kibanaSavedObjectMeta: { searchSourceJSON },
},
references = [],
references = [] as any[],
},
} = visualization;

Expand Down Expand Up @@ -164,13 +174,14 @@ async function fetchRollupVisualizations(
};
}

export function registerRollupUsageCollector(usageCollection, server) {
const kibanaIndex = server.config().get('kibana.index');

export function registerRollupUsageCollector(
usageCollection: UsageCollectionSetup,
kibanaIndex: string
): void {
const collector = usageCollection.makeUsageCollector({
type: ROLLUP_USAGE_TYPE,
isReady: () => true,
fetch: async callCluster => {
fetch: async (callCluster: CallCluster) => {
const rollupIndexPatterns = await fetchRollupIndexPatterns(kibanaIndex, callCluster);
const rollupIndexPatternToFlagMap = createIdToFlagMap(rollupIndexPatterns);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { PluginInitializerContext } from 'src/core/server';
import { RollupsServerPlugin } from './plugin';

export { wrapCustomError } from './wrap_custom_error';
export { wrapEsError } from './wrap_es_error';
export { wrapUnknownError } from './wrap_unknown_error';
export const plugin = (ctx: PluginInitializerContext) => new RollupsServerPlugin(ctx);

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { ElasticsearchServiceSetup } from 'kibana/server';
import { once } from 'lodash';
import { elasticsearchJsPlugin } from '../../client/elasticsearch_rollup';

const callWithRequest = once((elasticsearchService: ElasticsearchServiceSetup) => {
const config = { plugins: [elasticsearchJsPlugin] };
return elasticsearchService.createClient('rollup', config);
});

export const callWithRequestFactory = (
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this factory? In my PR, I've used it directly from the router context

https://github.com/elastic/kibana/pull/56829/files#diff-178c0726cd4b1e7299f257e5069b4a42R28

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for sharing! I'm going to leave as is for now, but will consider changing this in my next PR.

elasticsearchService: ElasticsearchServiceSetup,
request: any
) => {
return (...args: any[]) => {
return (
callWithRequest(elasticsearchService)
.asScoped(request)
// @ts-ignore
.callAsCurrentUser(...args)
);
};
};
Loading