Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Index management] Server-side NP ready #56829

Merged
merged 32 commits into from
Feb 11, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
b22e9ab
Move server code to "np_ready" folder
sebelga Feb 3, 2020
ee474e8
Add reload() api route
sebelga Feb 3, 2020
88192c5
Add indexClose() & indexOpen() api route
sebelga Feb 3, 2020
65f1ec2
Add indexRefresh() api route
sebelga Feb 3, 2020
bd74ed2
Add freeze() & unfreeze() api route
sebelga Feb 3, 2020
a69e7f8
Add flush() & forceMerge() api route
sebelga Feb 3, 2020
ba2c389
Add deleteIndex() api route
sebelga Feb 3, 2020
55a1c78
Add indexMapping() api route
sebelga Feb 3, 2020
7e53791
Add index settings Api routes
sebelga Feb 3, 2020
09542fe
Add indexStats() api route
sebelga Feb 4, 2020
8425911
Add get template(s) api routes
sebelga Feb 4, 2020
c212847
Clean-up server plugin dependencies
sebelga Feb 4, 2020
59ac5da
Add create & update template api routes
sebelga Feb 4, 2020
d5e6177
Use data client instead of admin client
sebelga Feb 4, 2020
3f7ffe5
Add delete template api routes and bring in wrapEsError helper
sebelga Feb 4, 2020
d4616bb
Use class to declare ApiRoutes
sebelga Feb 4, 2020
38e038c
Create License service to check license on routes
sebelga Feb 4, 2020
12c3723
Add route guard to License service
sebelga Feb 4, 2020
81410f1
Guard all api route with license checker
sebelga Feb 4, 2020
77188c8
Remove constructor from License and move to setup() call
sebelga Feb 4, 2020
3aa9d3b
Move IndexDataEnricher to its own class
sebelga Feb 5, 2020
a6e227e
Add missing license route guard
sebelga Feb 5, 2020
fec69db
Add missing "rollover_alias" prop to ilmPolicy schema object
sebelga Feb 5, 2020
78340e6
Refactor var name plugin public api
sebelga Feb 5, 2020
5e1b5e7
Remove "np_ready" folder for server side
sebelga Feb 5, 2020
7f8d305
Add readonly to Server plugin class
sebelga Feb 5, 2020
5861ab0
Add lib helpers in route dependencies
sebelga Feb 7, 2020
5da8c81
Wrap all server api response in try catch and return ES errors
sebelga Feb 7, 2020
30a7494
Fix API integration tests
sebelga Feb 10, 2020
fc96fed
Merge branch 'master' into np-migration/index-management
sebelga Feb 10, 2020
167f075
Fix API integration test (2)
sebelga Feb 11, 2020
1e902eb
Merge remote-tracking branch 'upstream/master' into np-migration/inde…
sebelga Feb 11, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions x-pack/legacy/plugins/cross_cluster_replication/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import { PLUGIN } from './common/constants';
import { registerLicenseChecker } from './server/lib/register_license_checker';
import { registerRoutes } from './server/routes/register_routes';
import { ccrDataEnricher } from './cross_cluster_replication_data';
import { addIndexManagementDataEnricher } from '../index_management/server/index_management_data';

export function crossClusterReplication(kibana) {
return new kibana.Plugin({
id: PLUGIN.ID,
Expand Down Expand Up @@ -49,8 +49,13 @@ export function crossClusterReplication(kibana) {
init: function initCcrPlugin(server) {
registerLicenseChecker(server);
registerRoutes(server);
if (server.config().get('xpack.ccr.ui.enabled')) {
addIndexManagementDataEnricher(ccrDataEnricher);

if (
server.config().get('xpack.ccr.ui.enabled') &&
server.plugins.index_management &&
server.plugins.index_management.addIndexManagementDataEnricher
) {
server.plugins.index_management.addIndexManagementDataEnricher(ccrDataEnricher);
}
},
});
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,15 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { LICENSE_TYPE_BASIC } from '../../../../common/constants';
import { LicenseType } from '../../../../../plugins/licensing/common/types';

const basicLicense: LicenseType = 'basic';

export const PLUGIN = {
ID: 'index_management',
id: 'index_management',
minimumLicenseType: basicLicense,
getI18nName: (i18n: any): string =>
i18n.translate('xpack.idxMgmt.appTitle', {
defaultMessage: 'Index Management',
}),
MINIMUM_LICENSE_REQUIRED: LICENSE_TYPE_BASIC,
};
7 changes: 7 additions & 0 deletions x-pack/legacy/plugins/index_management/common/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

export { PLUGIN, API_BASE_PATH } from './constants';
39 changes: 9 additions & 30 deletions x-pack/legacy/plugins/index_management/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,15 @@
*/

import { resolve } from 'path';
import { i18n } from '@kbn/i18n';
import { Legacy } from 'kibana';
import { createRouter } from '../../server/lib/create_router';
import { registerLicenseChecker } from '../../server/lib/register_license_checker';
import { PLUGIN, API_BASE_PATH } from './common/constants';
import { LegacySetup } from './server/plugin';
import { plugin as initServerPlugin } from './server';
import { PLUGIN } from './common/constants';
import { plugin as initServerPlugin, Dependencies } from './server';

export type ServerFacade = Legacy.Server;

export function indexManagement(kibana: any) {
return new kibana.Plugin({
id: PLUGIN.ID,
id: PLUGIN.id,
configPrefix: 'xpack.index_management',
publicDir: resolve(__dirname, 'public'),
require: ['kibana', 'elasticsearch', 'xpack_main'],
Expand All @@ -29,32 +25,15 @@ export function indexManagement(kibana: any) {

init(server: ServerFacade) {
const coreSetup = server.newPlatform.setup.core;

const pluginsSetup = {};

const __LEGACY: LegacySetup = {
router: createRouter(server, PLUGIN.ID, `${API_BASE_PATH}/`),
plugins: {
license: {
registerLicenseChecker: registerLicenseChecker.bind(
null,
server,
PLUGIN.ID,
PLUGIN.getI18nName(i18n),
PLUGIN.MINIMUM_LICENSE_REQUIRED as 'basic'
),
},
elasticsearch: server.plugins.elasticsearch,
},
const coreInitializerContext = server.newPlatform.coreContext;
const pluginsSetup: Dependencies = {
licensing: server.newPlatform.setup.plugins.licensing as any,
};

const serverPlugin = initServerPlugin();
const indexMgmtSetup = serverPlugin.setup(coreSetup, pluginsSetup, __LEGACY);
const serverPlugin = initServerPlugin(coreInitializerContext as any);
const serverPublicApi = serverPlugin.setup(coreSetup, pluginsSetup);

server.expose(
'addIndexManagementDataEnricher',
indexMgmtSetup.addIndexManagementDataEnricher
);
server.expose('addIndexManagementDataEnricher', serverPublicApi.indexDataEnricher.add);
},
});
}
10 changes: 6 additions & 4 deletions x-pack/legacy/plugins/index_management/server/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { IndexMgmtPlugin } from './plugin';

export function plugin() {
return new IndexMgmtPlugin();
}
import { PluginInitializerContext } from 'src/core/server';
import { IndexMgmtServerPlugin } from './plugin';

export const plugin = (ctx: PluginInitializerContext) => new IndexMgmtServerPlugin(ctx);

export { Dependencies } from './types';

This file was deleted.

41 changes: 16 additions & 25 deletions x-pack/legacy/plugins/index_management/server/lib/fetch_indices.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { IndexDataEnricher } from '../services';
import { Index, CallAsCurrentUser } from '../types';
import { fetchAliases } from './fetch_aliases';
import { getIndexManagementDataEnrichers } from '../index_management_data';

interface Hit {
health: string;
status: string;
Expand All @@ -27,22 +29,7 @@ interface Params {
index?: string[];
}

const enrichResponse = async (response: any, callWithRequest: any) => {
let enrichedResponse = response;
const dataEnrichers = getIndexManagementDataEnrichers();
for (let i = 0; i < dataEnrichers.length; i++) {
const dataEnricher = dataEnrichers[i];
try {
const dataEnricherResponse = await dataEnricher(enrichedResponse, callWithRequest);
enrichedResponse = dataEnricherResponse;
} catch (e) {
// silently swallow enricher response errors
}
}
return enrichedResponse;
};

function formatHits(hits: Hit[], aliases: Aliases) {
function formatHits(hits: Hit[], aliases: Aliases): Index[] {
return hits.map((hit: Hit) => {
return {
health: hit.health,
Expand All @@ -59,7 +46,7 @@ function formatHits(hits: Hit[], aliases: Aliases) {
});
}

async function fetchIndicesCall(callWithRequest: any, indexNames?: string[]) {
async function fetchIndicesCall(callAsCurrentUser: CallAsCurrentUser, indexNames?: string[]) {
const params: Params = {
format: 'json',
h: 'health,status,index,uuid,pri,rep,docs.count,sth,store.size',
Expand All @@ -69,13 +56,17 @@ async function fetchIndicesCall(callWithRequest: any, indexNames?: string[]) {
params.index = indexNames;
}

return await callWithRequest('cat.indices', params);
return await callAsCurrentUser('cat.indices', params);
}

export const fetchIndices = async (callWithRequest: any, indexNames?: string[]) => {
const aliases = await fetchAliases(callWithRequest);
const hits = await fetchIndicesCall(callWithRequest, indexNames);
let response = formatHits(hits, aliases);
response = await enrichResponse(response, callWithRequest);
return response;
export const fetchIndices = async (
callAsCurrentUser: CallAsCurrentUser,
indexDataEnricher: IndexDataEnricher,
indexNames?: string[]
) => {
const aliases = await fetchAliases(callAsCurrentUser);
const hits = await fetchIndicesCall(callAsCurrentUser, indexNames);
const indices = formatHits(hits, aliases);

return await indexDataEnricher.enrichIndices(indices, callAsCurrentUser);
};
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@
// Cloud has its own system for managing templates and we want to make
// this clear in the UI when a template is used in a Cloud deployment.
export const getManagedTemplatePrefix = async (
callWithInternalUser: any
callAsCurrentUser: any
): Promise<string | undefined> => {
try {
const { persistent, transient, defaults } = await callWithInternalUser('cluster.getSettings', {
const { persistent, transient, defaults } = await callAsCurrentUser('cluster.getSettings', {
filterPath: '*.*managed_index_templates',
flatSettings: true,
includeDefaults: true,
Expand Down
13 changes: 13 additions & 0 deletions x-pack/legacy/plugins/index_management/server/lib/is_es_error.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import * as legacyElasticsearch from 'elasticsearch';

const esErrorsParent = legacyElasticsearch.errors._Abstract;

export function isEsError(err: Error) {
return err instanceof esErrorsParent;
}
87 changes: 53 additions & 34 deletions x-pack/legacy/plugins/index_management/server/plugin.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,48 +3,67 @@
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/
import { CoreSetup } from 'src/core/server';
import { ElasticsearchPlugin } from 'src/legacy/core_plugins/elasticsearch';
import { Router } from '../../../server/lib/create_router';
import { addIndexManagementDataEnricher } from './index_management_data';
import { registerIndicesRoutes } from './routes/api/indices';
import { registerTemplateRoutes } from './routes/api/templates';
import { registerMappingRoute } from './routes/api/mapping';
import { registerSettingsRoutes } from './routes/api/settings';
import { registerStatsRoute } from './routes/api/stats';

export interface LegacySetup {
router: Router;
plugins: {
elasticsearch: ElasticsearchPlugin;
license: {
registerLicenseChecker: () => void;
};
};
}
import { i18n } from '@kbn/i18n';
import { CoreSetup, Plugin, Logger, PluginInitializerContext } from 'src/core/server';

import { PLUGIN } from '../common';
import { Dependencies } from './types';
import { ApiRoutes } from './routes';
import { License, IndexDataEnricher } from './services';
import { isEsError } from './lib/is_es_error';

export interface IndexMgmtSetup {
addIndexManagementDataEnricher: (enricher: any) => void;
indexDataEnricher: {
add: IndexDataEnricher['add'];
};
}

export class IndexMgmtPlugin {
public setup(core: CoreSetup, plugins: {}, __LEGACY: LegacySetup): IndexMgmtSetup {
const serverFacade = {
plugins: {
elasticsearch: __LEGACY.plugins.elasticsearch,
},
};
export class IndexMgmtServerPlugin implements Plugin<IndexMgmtSetup, void, any, any> {
private readonly apiRoutes: ApiRoutes;
private readonly license: License;
private readonly logger: Logger;
private readonly indexDataEnricher: IndexDataEnricher;

__LEGACY.plugins.license.registerLicenseChecker();
constructor({ logger }: PluginInitializerContext) {
this.logger = logger.get();
this.apiRoutes = new ApiRoutes();
this.license = new License();
this.indexDataEnricher = new IndexDataEnricher();
}

setup({ http }: CoreSetup, { licensing }: Dependencies): IndexMgmtSetup {
const router = http.createRouter();

registerIndicesRoutes(__LEGACY.router);
registerTemplateRoutes(__LEGACY.router, serverFacade);
registerSettingsRoutes(__LEGACY.router);
registerStatsRoute(__LEGACY.router);
registerMappingRoute(__LEGACY.router);
this.license.setup(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I like this idea.

It's nice to see how the different plugin phases can be tagged in services so we build on the established plugin pattern.

My question is; what benefits do we get from have instances of the services live on the plugin as opposed to constructing them at setup time and passing them into our other components? Is it mainly for naming or do we see these services often also having start phase concerns?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seemed more logical to me to instantiate them outside, no real benefit. But as you said, this patterns allows us to call .start() and .stop() if needed.

{
pluginId: PLUGIN.id,
minimumLicenseType: PLUGIN.minimumLicenseType,
defaultErrorMessage: i18n.translate('xpack.idxMgmt.licenseCheckErrorMessage', {
defaultMessage: 'License check failed',
}),
},
{
licensing,
logger: this.logger,
}
);

this.apiRoutes.setup({
router,
license: this.license,
indexDataEnricher: this.indexDataEnricher,
lib: {
isEsError,
},
});

return {
addIndexManagementDataEnricher,
indexDataEnricher: {
add: this.indexDataEnricher.add.bind(this.indexDataEnricher),
},
};
}

start() {}
stop() {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License;
* you may not use this file except in compliance with the Elastic License.
*/

import { API_BASE_PATH } from '../../../common';

export const addBasePath = (uri: string): string => API_BASE_PATH + uri;
Loading