Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution][Elastic AI Assistant] Updates ESQL Knowledge Base docs to latest #169593

Merged
merged 3 commits into from
Oct 26, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

Original file line number Diff line number Diff line change
@@ -1,10 +1,28 @@
[[esql-processing-commands]]
== {esql} processing commands
[[esql-commands]]
=== {esql} commands

++++
<titleabbrev>Processing commands</titleabbrev>
<titleabbrev>Commands</titleabbrev>
++++

// tag::source_commands[]
==== Source commands

An {esql} source command produces a table, typically with data from {es}. An {esql} query must start with a source command.

image::images/esql/source-command.svg[A source command producing a table from {es},align="center"]

{esql} supports these source commands:

* <<esql-from>>
* <<esql-row>>
* <<esql-show>>

// end::source_command[]

// tag::proc_commands[]
==== Processing commands

{esql} processing commands change an input table by adding, removing, or changing
rows and columns.

Expand All @@ -25,6 +43,12 @@ image::images/esql/processing-command.svg[A processing command changing an input
* <<esql-stats-by>>
* <<esql-where>>

// end::proc_command[]

include::source-commands/from.asciidoc[]
include::source-commands/row.asciidoc[]
include::source-commands/show.asciidoc[]

include::processing-commands/dissect.asciidoc[]
include::processing-commands/drop.asciidoc[]
include::processing-commands/enrich.asciidoc[]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
[[esql-enrich-data]]
=== Enrich data

++++
<titleabbrev>Enrich data</titleabbrev>
++++

You can use {esql}'s <<esql-enrich>> processing command to enrich a table with
data from indices in {es}.

For example, you can use `ENRICH` to:

* Identify web services or vendors based on known IP addresses
* Add product information to retail orders based on product IDs
* Supplement contact information based on an email address

[[esql-how-enrich-works]]
==== How the `ENRICH` command works

The `ENRICH` command adds new columns to a table, with data from {es} indices.
It requires a few special components:

image::images/esql/esql-enrich.png[align="center"]

[[esql-enrich-policy]]
Enrich policy::
+
--
A set of configuration options used to add the right enrich data to the input
table.

An enrich policy contains:

include::../ingest/enrich.asciidoc[tag=enrich-policy-fields]

After <<esql-create-enrich-policy,creating a policy>>, it must be
<<esql-execute-enrich-policy,executed>> before it can be used. Executing an
enrich policy uses data from the policy's source indices to create a streamlined
system index called the _enrich index_. The `ENRICH` command uses this index to
match and enrich an input table.
--

[[esql-source-index]]
Source index::
An index which stores enrich data that the `ENRICH` command can add to input
tables. You can create and manage these indices just like a regular {es} index.
You can use multiple source indices in an enrich policy. You also can use the
same source index in multiple enrich policies.

[[esql-enrich-index]]
Enrich index::
+
--
A special system index tied to a specific enrich policy.

Directly matching rows from input tables to documents in source indices could be
slow and resource intensive. To speed things up, the `ENRICH` command uses an
enrich index.

include::../ingest/enrich.asciidoc[tag=enrich-index]
--

[[esql-set-up-enrich-policy]]
==== Set up an enrich policy

To start using `ENRICH`, follow these steps:

. Check the <<enrich-prereqs, prerequisites>>.
. <<esql-create-enrich-source-index>>.
. <<esql-create-enrich-policy>>.
. <<esql-execute-enrich-policy>>.
. <<esql-use-enrich>>

Once you have enrich policies set up, you can <<esql-update-enrich-data,update
your enrich data>> and <<esql-update-enrich-policies, update your enrich
policies>>.

[IMPORTANT]
====
The `ENRICH` command performs several operations and may impact the speed of
your query.
====

[[esql-enrich-prereqs]]
==== Prerequisites

include::{es-repo-dir}/ingest/apis/enrich/put-enrich-policy.asciidoc[tag=enrich-policy-api-prereqs]

[[esql-create-enrich-source-index]]
==== Add enrich data

include::../ingest/enrich.asciidoc[tag=create-enrich-source-index]

[[esql-create-enrich-policy]]
==== Create an enrich policy

include::../ingest/enrich.asciidoc[tag=create-enrich-policy]

[[esql-execute-enrich-policy]]
==== Execute the enrich policy

include::../ingest/enrich.asciidoc[tag=execute-enrich-policy1]

image::images/esql/esql-enrich-policy.png[align="center"]

include::../ingest/enrich.asciidoc[tag=execute-enrich-policy2]

[[esql-use-enrich]]
==== Use the enrich policy

After the policy has been executed, you can use the <<esql-enrich,`ENRICH`
command>> to enrich your data.

image::images/esql/esql-enrich-command.png[align="center",width=50%]

include::processing-commands/enrich.asciidoc[tag=examples]

[[esql-update-enrich-data]]
==== Update an enrich index

include::{es-repo-dir}/ingest/apis/enrich/execute-enrich-policy.asciidoc[tag=update-enrich-index]

[[esql-update-enrich-policies]]
==== Update an enrich policy

include::../ingest/enrich.asciidoc[tag=update-enrich-policy]
Original file line number Diff line number Diff line change
Expand Up @@ -26,16 +26,17 @@ these functions:
* <<esql-date_parse>>
* <<esql-date_trunc>>
* <<esql-e>>
* <<esql-ends_with>>
* <<esql-floor>>
* <<esql-greatest>>
* <<esql-is_finite>>
* <<esql-is_infinite>>
* <<esql-is_nan>>
* <<esql-least>>
* <<esql-left>>
* <<esql-length>>
* <<esql-log10>>
* <<esql-ltrim>>
* <<esql-rtrim>>
* <<esql-mv_avg>>
* <<esql-mv_concat>>
* <<esql-mv_count>>
Expand All @@ -47,13 +48,15 @@ these functions:
* <<esql-now>>
* <<esql-pi>>
* <<esql-pow>>
* <<esql-replace>>
* <<esql-right>>
* <<esql-round>>
* <<esql-rtrim>>
* <<esql-sin>>
* <<esql-sinh>>
* <<esql-split>>
* <<esql-starts_with>>
* <<esql-substring>>
* <<esql-left>>
* <<esql-tan>>
* <<esql-tanh>>
* <<esql-tau>>
Expand Down Expand Up @@ -88,16 +91,17 @@ include::functions/date_format.asciidoc[]
include::functions/date_parse.asciidoc[]
include::functions/date_trunc.asciidoc[]
include::functions/e.asciidoc[]
include::functions/ends_with.asciidoc[]
include::functions/floor.asciidoc[]
include::functions/greatest.asciidoc[]
include::functions/is_finite.asciidoc[]
include::functions/is_infinite.asciidoc[]
include::functions/is_nan.asciidoc[]
include::functions/least.asciidoc[]
include::functions/left.asciidoc[]
include::functions/length.asciidoc[]
include::functions/log10.asciidoc[]
include::functions/ltrim.asciidoc[]
include::functions/rtrim.asciidoc[]
include::functions/mv_avg.asciidoc[]
include::functions/mv_concat.asciidoc[]
include::functions/mv_count.asciidoc[]
Expand All @@ -109,14 +113,16 @@ include::functions/mv_sum.asciidoc[]
include::functions/now.asciidoc[]
include::functions/pi.asciidoc[]
include::functions/pow.asciidoc[]
include::functions/replace.asciidoc[]
include::functions/right.asciidoc[]
include::functions/round.asciidoc[]
include::functions/rtrim.asciidoc[]
include::functions/sin.asciidoc[]
include::functions/sinh.asciidoc[]
include::functions/split.asciidoc[]
include::functions/sqrt.asciidoc[]
include::functions/starts_with.asciidoc[]
include::functions/substring.asciidoc[]
include::functions/left.asciidoc[]
include::functions/tan.asciidoc[]
include::functions/tanh.asciidoc[]
include::functions/tau.asciidoc[]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
[[esql-functions-operators]]
=== {esql} functions and operators

++++
<titleabbrev>Functions and operators</titleabbrev>
++++

{esql} provides a comprehensive set of functions and operators for working with data.
The functions are divided into the following categories:

[[esql-functions]]
<<esql-agg-functions>>::
include::functions/aggregation-functions.asciidoc[tag=agg_list]

<<esql-math-functions>>::
include::functions/math-functions.asciidoc[tag=math_list]

<<esql-string-functions>>::
include::functions/string-functions.asciidoc[tag=string_list]

<<esql-date-time-functions>>::
include::functions/date-time-functions.asciidoc[tag=date_list]

<<esql-type-conversion-functions>>::
include::functions/type-conversion-functions.asciidoc[tag=type_list]

<<esql-conditional-functions-and-expressions>>::
include::functions/conditional-functions-and-expressions.asciidoc[tag=cond_list]

<<esql-mv-functions>>::
include::functions/mv-functions.asciidoc[tag=mv_list]

<<esql-operators>>::
include::functions/operators.asciidoc[tag=op_list]

include::functions/aggregation-functions.asciidoc[]
include::functions/math-functions.asciidoc[]
include::functions/string-functions.asciidoc[]
include::functions/date-time-functions.asciidoc[]
include::functions/type-conversion-functions.asciidoc[]
include::functions/conditional-functions-and-expressions.asciidoc[]
include::functions/mv-functions.asciidoc[]
include::functions/operators.asciidoc[]
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
[[esql-getting-started]]
== Getting started with {esql}

++++
<titleabbrev>Getting started</titleabbrev>
++++

coming::[8.11]
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
[[esql-kibana]]
== Using {esql} in {kib}

++++
<titleabbrev>Kibana</titleabbrev>
++++


Use {esql} in Discover to explore a data set. From the data view dropdown,
select *Try {esql}* to get started.

NOTE: {esql} queries in Discover and Lens are subject to the time range selected
with the time filter.


Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
[[esql-language]]
== Working with the {esql} language

++++
<titleabbrev>Working with the {esql} language</titleabbrev>
++++

Detailed information about the {esql} language:

* <<esql-syntax>>
* <<esql-commands>>
* <<esql-functions>>
* <<esql-multivalued-fields>>
* <<esql-metadata-fields>>
* <<esql-enrich-data>>

include::esql-syntax.asciidoc[]
include::esql-commands.asciidoc[]
include::esql-functions-operators.asciidoc[]
include::multivalued-fields.asciidoc[]
include::metadata-fields.asciidoc[]
include::esql-enrich-data.asciidoc[]

Loading
Loading