Skip to content

Commit

Permalink
[SIEM][Detection Engine] Disambiguates signals, rules, alerts, and de…
Browse files Browse the repository at this point in the history
…tection engine by renaming them (#51684) (#51729)

## Summary

* Renames `signals -> rules` when it is specific about rules
* Renames `signals -> detection engine` when is generically talking about both rules and signals
* Renames `signals -> alerts` in a few spots when it is talking specifically about alerting plugins
* Keeps the name of signal when it involves the signals output index or a source input index for potential signals to be generated from
* Did a `git mv <file_1> <file_2>` for everything
* Updated local variables as well per rules above.

### Checklist

Use ~~strikethroughs~~ to remove checklist items you don't feel are applicable to this PR.

~~- [ ] This was checked for cross-browser compatibility, [including a check against IE11](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility)~~

~~- [ ] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)~~

~~- [ ] [Documentation](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#writing-documentation) was added for features that require explanation or tutorials~~

- [ ] [Unit or functional tests](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#cross-browser-compatibility) were updated or added to match the most common scenarios

~~- [ ] This was checked for [keyboard-only and screenreader accessibility](https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Accessibility#Accessibility_testing_checklist)~~

### For maintainers

~~- [ ] This was checked for breaking API changes and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)~~

- [ ] This includes a feature addition or change that requires a release note and was [labeled appropriately](https://github.com/elastic/kibana/blob/master/CONTRIBUTING.md#release-notes-process)
  • Loading branch information
FrankHassanabad authored Nov 26, 2019
1 parent 31b20f4 commit 4b7631c
Show file tree
Hide file tree
Showing 66 changed files with 630 additions and 651 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,22 @@ const path = require('path');

/*
* This script is used to parse a set of saved searches on a file system
* and output signal data compatible json files.
* and output rule data compatible json files.
* Example:
* node saved_query_to_signals.js ${HOME}/saved_searches ${HOME}/saved_signals
* node saved_query_to_rules.js ${HOME}/saved_searches ${HOME}/saved_rules
*
* After editing any changes in the files of ${HOME}/saved_signals/*.json
* you can then post the signals with a CURL post script such as:
* After editing any changes in the files of ${HOME}/saved_rules/*.json
* you can then post the rules with a CURL post script such as:
*
* ./post_signal.sh ${HOME}/saved_signals/*.json
* ./post_rule.sh ${HOME}/saved_rules/*.json
*
* Note: This script is recursive and but does not preserve folder structure
* when it outputs the saved signals.
* when it outputs the saved rules.
*/

// Defaults of the outputted signals since the saved KQL searches do not have
// Defaults of the outputted rules since the saved KQL searches do not have
// this type of information. You usually will want to make any hand edits after
// doing a search to KQL conversion before posting it as a signal or checking it
// doing a search to KQL conversion before posting it as a rule or checking it
// into another repository.
const INTERVAL = '5m';
const SEVERITY = 'low';
Expand All @@ -36,8 +36,8 @@ const TO = 'now';
const IMMUTABLE = true;
const RISK_SCORE = 50;
const ENABLED = false;
let allSignals = '';
const allSignalsNdJson = 'all_rules.ndjson';
let allRules = '';
const allRulesNdJson = 'all_rules.ndjson';

// For converting, if you want to use these instead of rely on the defaults then
// comment these in and use them for the script. Otherwise this is commented out
Expand Down Expand Up @@ -74,7 +74,7 @@ const cleanupFileName = file => {
async function main() {
if (process.argv.length !== 4) {
throw new Error(
'usage: saved_query_to_signals [input directory with saved searches] [output directory]'
'usage: saved_query_to_rules [input directory with saved searches] [output directory]'
);
}

Expand Down Expand Up @@ -152,11 +152,11 @@ async function main() {
`${outputDir}/${fileToWrite}.json`,
JSON.stringify(outputMessage, null, 2)
);
allSignals += `${JSON.stringify(outputMessage)}\n`;
allRules += `${JSON.stringify(outputMessage)}\n`;
}
}
);
fs.writeFileSync(`${outputDir}/${allSignalsNdJson}`, allSignals);
fs.writeFileSync(`${outputDir}/${allRulesNdJson}`, allRules);
}

if (require.main === module) {
Expand Down
26 changes: 13 additions & 13 deletions x-pack/legacy/plugins/siem/server/kibana.index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,13 @@ import {
timelineSavedObjectType,
} from './saved_objects';

import { signalsAlertType } from './lib/detection_engine/alerts/signals_alert_type';
import { rulesAlertType } from './lib/detection_engine/alerts/rules_alert_type';
import { isAlertExecutor } from './lib/detection_engine/alerts/types';
import { createSignalsRoute } from './lib/detection_engine/routes/create_signals_route';
import { readSignalsRoute } from './lib/detection_engine/routes/read_signals_route';
import { findSignalsRoute } from './lib/detection_engine/routes/find_signals_route';
import { deleteSignalsRoute } from './lib/detection_engine/routes/delete_signals_route';
import { updateSignalsRoute } from './lib/detection_engine/routes/update_signals_route';
import { createRulesRoute } from './lib/detection_engine/routes/create_rules_route';
import { readRulesRoute } from './lib/detection_engine/routes/read_rules_route';
import { findRulesRoute } from './lib/detection_engine/routes/find_rules_route';
import { deleteRulesRoute } from './lib/detection_engine/routes/delete_rules_route';
import { updateRulesRoute } from './lib/detection_engine/routes/update_rules_route';
import { ServerFacade } from './types';

const APP_ID = 'siem';
Expand All @@ -33,7 +33,7 @@ export const initServerWithKibana = (
) => {
if (kbnServer.plugins.alerting != null) {
const version = kbnServer.config().get<string>('pkg.version');
const type = signalsAlertType({ logger, version });
const type = rulesAlertType({ logger, version });
if (isAlertExecutor(type)) {
kbnServer.plugins.alerting.setup.registerType(type);
}
Expand All @@ -49,13 +49,13 @@ export const initServerWithKibana = (
kbnServer.config().has('xpack.alerting.enabled') === true
) {
logger.info(
'Detected feature flags for actions and alerting and enabling signals API endpoints'
'Detected feature flags for actions and alerting and enabling detection engine API endpoints'
);
createSignalsRoute(kbnServer);
readSignalsRoute(kbnServer);
updateSignalsRoute(kbnServer);
deleteSignalsRoute(kbnServer);
findSignalsRoute(kbnServer);
createRulesRoute(kbnServer);
readRulesRoute(kbnServer);
updateRulesRoute(kbnServer);
deleteRulesRoute(kbnServer);
findRulesRoute(kbnServer);
}

const xpackMainPlugin = kbnServer.plugins.xpack_main;
Expand Down
18 changes: 9 additions & 9 deletions x-pack/legacy/plugins/siem/server/lib/detection_engine/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ xpack.alerting.enabled: true
xpack.actions.enabled: true
```

Start Kibana and you will see these messages indicating signals is activated like so:
Start Kibana and you will see these messages indicating detection engine is activated like so:

```sh
server log [11:39:05.561] [info][siem] Detected feature flags for actions and alerting and enabling signals API endpoints
server log [11:39:05.561] [info][siem] Detected feature flags for actions and alerting and enabling detection engine API endpoints
```

If you see crashes like this:
Expand Down Expand Up @@ -98,10 +98,10 @@ server log [22:05:22.277] [info][status][plugin:alerting@8.0.0] Status chan
server log [22:05:22.270] [info][status][plugin:actions@8.0.0] Status changed from uninitialized to green - Ready
```

You should also see the SIEM detect the feature flags and start the API endpoints for signals
You should also see the SIEM detect the feature flags and start the API endpoints for detection engine

```sh
server log [11:39:05.561] [info][siem] Detected feature flags for actions and alerting and enabling signals API endpoints
server log [11:39:05.561] [info][siem] Detected feature flags for actions and alerting and enabling detection engine API endpoints
```

Go into your SIEM Advanced settings and underneath the setting of `siem:defaultSignalsIndex`, set that to the same
Expand All @@ -125,16 +125,16 @@ which will:
- Delete any existing alert tasks you have
- Delete any existing signal mapping you might have had.
- Add the latest signal index and its mappings using your settings from `SIGNALS_INDEX` environment variable.
- Posts the sample signal from `signals/root_or_admin_1.json` by replacing its `output_index` with your `SIGNALS_INDEX` environment variable
- The sample signal checks for root or admin every 5 minutes and reports that as a signal if it is a positive hit
- Posts the sample rule from `rules/root_or_admin_1.json` by replacing its `output_index` with your `SIGNALS_INDEX` environment variable
- The sample rule checks for root or admin every 5 minutes and reports that as a signal if it is a positive hit

Now you can run

```sh
./find_signals.sh
./find_rules.sh
```

You should see the new signals created like so:
You should see the new rules created like so:

```sh
{
Expand Down Expand Up @@ -184,7 +184,7 @@ Every 5 minutes if you get positive hits you will see messages on info like so:
server log [09:54:59.013] [info][plugins][siem] Total signals found from signal rule "id: a556065c-0656-4ba1-ad64-a77ca9d2013b", "ruleId: rule-1": 10000
```

Signals are space aware and default to the "default" space for these scripts if you do not export
Rules are space aware and default to the "default" space for these scripts if you do not export
the variable of SPACE_URL. For example, if you want to post rules to the space `test-space` you would
set your SPACE_URL to be:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,12 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { SignalSourceHit, SignalSearchResponse, AlertTypeParams } from '../types';
import { SignalSourceHit, SignalSearchResponse, RuleTypeParams } from '../types';

export const sampleSignalAlertParams = (
export const sampleRuleAlertParams = (
maxSignals: number | undefined,
riskScore?: number | undefined
): AlertTypeParams => ({
): RuleTypeParams => ({
ruleId: 'rule-1',
description: 'Detecting root and admin users',
falsePositives: [],
Expand Down Expand Up @@ -242,4 +242,4 @@ export const sampleDocSearchResultsWithSortId = (someUuid: string): SignalSearch
},
});

export const sampleSignalId = '04128c15-0d1b-4716-a4c5-46997ac7f3bd';
export const sampleRuleGuid = '04128c15-0d1b-4716-a4c5-46997ac7f3bd';
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
*/

import { SIGNALS_ID } from '../../../../common/constants';
import { SignalParams } from './types';
import { RuleParams } from './types';

export const createSignals = async ({
export const createRules = async ({
alertsClient,
actionsClient, // TODO: Use this actionsClient once we have actions such as email, etc...
description,
Expand All @@ -33,7 +33,7 @@ export const createSignals = async ({
to,
type,
references,
}: SignalParams) => {
}: RuleParams) => {
return alertsClient.create({
data: {
name,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,27 +4,27 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { readSignals } from './read_signals';
import { DeleteSignalParams } from './types';
import { readRules } from './read_rules';
import { DeleteRuleParams } from './types';

export const deleteSignals = async ({
export const deleteRules = async ({
alertsClient,
actionsClient, // TODO: Use this when we have actions such as email, etc...
id,
ruleId,
}: DeleteSignalParams) => {
const signal = await readSignals({ alertsClient, id, ruleId });
if (signal == null) {
}: DeleteRuleParams) => {
const rule = await readRules({ alertsClient, id, ruleId });
if (rule == null) {
return null;
}

if (ruleId != null) {
await alertsClient.delete({ id: signal.id });
return signal;
await alertsClient.delete({ id: rule.id });
return rule;
} else if (id != null) {
try {
await alertsClient.delete({ id });
return signal;
return rule;
} catch (err) {
if (err.output.statusCode === 404) {
return null;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@
* you may not use this file except in compliance with the Elastic License.
*/

import { getFilter } from './find_signals';
import { getFilter } from './find_rules';
import { SIGNALS_ID } from '../../../../common/constants';

describe('find_signals', () => {
describe('find_rules', () => {
test('it returns a full filter with an AND if sent down', () => {
expect(getFilter('alert.attributes.enabled: true')).toEqual(
`alert.attributes.alertTypeId: ${SIGNALS_ID} AND alert.attributes.enabled: true`
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
*/

import { SIGNALS_ID } from '../../../../common/constants';
import { FindSignalParams } from './types';
import { FindRuleParams } from './types';

export const getFilter = (filter: string | null | undefined) => {
if (filter == null) {
Expand All @@ -15,15 +15,15 @@ export const getFilter = (filter: string | null | undefined) => {
}
};

export const findSignals = async ({
export const findRules = async ({
alertsClient,
perPage,
page,
fields,
filter,
sortField,
sortOrder,
}: FindSignalParams) => {
}: FindRuleParams) => {
return alertsClient.find({
options: {
fields,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
*/

import { AlertServices } from '../../../../../alerting/server/types';
import { SignalAlertParams, PartialFilter } from './types';
import { RuleAlertParams, PartialFilter } from './types';
import { assertUnreachable } from '../../../utils/build_query';
import {
Query,
Expand Down Expand Up @@ -41,7 +41,7 @@ export const getQueryFilter = (
};

interface GetFilterArgs {
type: SignalAlertParams['type'];
type: RuleAlertParams['type'];
filter: Record<string, {}> | undefined | null;
filters: PartialFilter[] | undefined | null;
language: string | undefined | null;
Expand Down Expand Up @@ -86,7 +86,7 @@ export const getFilter = async ({
if (query != null && language != null && index != null) {
return getQueryFilter(query, language, filters || [], index);
} else {
// user did not give any additional fall back mechanism for generating a signal
// user did not give any additional fall back mechanism for generating a rule
// rethrow error for activity monitoring
throw err;
}
Expand Down
Loading

0 comments on commit 4b7631c

Please sign in to comment.