Merge branch 'master' into alerting/enable-by-default

elastic · Nov 21, 2019 · 42652fc · 42652fc
2 parents 6fad527 + 85568b9
commit 42652fc
Show file tree

Hide file tree

Showing 446 changed files with 7,082 additions and 4,339 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -71,6 +71,7 @@
 /x-pack/test/api_integration/apis/security/ @elastic/kibana-security
 
 # Kibana Stack Services
+/src/dev/i18n @elastic/kibana-stack-services
 /packages/kbn-analytics/ @elastic/kibana-stack-services
 /src/legacy/core_plugins/ui_metric/ @elastic/kibana-stack-services
 /x-pack/legacy/plugins/telemetry @elastic/kibana-stack-services

diff --git a/TYPESCRIPT.md b/TYPESCRIPT.md
@@ -21,7 +21,7 @@ The first thing that will probably happen when you convert a `.js` file in our s
 
 declare module '@elastic/eui' {
   // Add your types here
-  export const EuiPopoverTitle: React.SFC<EuiPopoverTitleProps>;
+  export const EuiPopoverTitle: React.FC<EuiPopoverTitleProps>;
   ...
 }
 ```
@@ -47,13 +47,13 @@ Since `@elastic/eui` already ships with a module declaration, any local addition
 // file `typings/@elastic/eui/index.d.ts`
 
 import { CommonProps } from '@elastic/eui';
-import { SFC } from 'react';
+import { FC } from 'react';
 
 declare module '@elastic/eui' {
   export type EuiNewComponentProps = CommonProps & {
     additionalProp: string;
   };
-  export const EuiNewComponent: SFC<EuiNewComponentProps>;
+  export const EuiNewComponent: FC<EuiNewComponentProps>;
 }
 ```
 

diff --git a/docs/discover/context.asciidoc b/docs/discover/context.asciidoc
@@ -3,7 +3,7 @@
 
 For certain applications it can be useful to inspect a window of documents
 surrounding a specific event. The context view enables you to do just that for
-index patterns that are configured to contain time-based events.
+<<index-patterns, index patterns>> that are configured to contain time-based events.
 
 To show the context surrounding an anchor document, click the *Expand* button
 image:images/ExpandButton.jpg[Expand Button] to the left of the document's

diff --git a/docs/discover/document-data.asciidoc b/docs/discover/document-data.asciidoc
@@ -5,7 +5,7 @@ When you submit a search query, the 500 most recent documents that match the que
 are listed in the Documents table. You can configure the number of documents shown
 in the table by setting the `discover:sampleSize` property in <<advanced-options,
 Advanced Settings>>. By default, the table shows the localized version of the time
-field configured for the selected index pattern and the document `_source`. You can
+field configured for the selected <<index-patterns, index pattern>> and the document `_source`. You can
 <<adding-columns, add fields to the Documents table>> from the Fields list.
 You can <<sorting, sort the listed documents>> by any indexed field that's included
 in the table.

diff --git a/docs/discover/field-filter.asciidoc b/docs/discover/field-filter.asciidoc
@@ -14,7 +14,8 @@ To add a filter from the Fields list:
 . Click the name of the field you want to filter on. This displays the top
 five values for that field.
 +
-image::images/filter-field.jpg[]
+[role="screenshot"]
+image::images/filter-field.png[height=317]
 . To add a positive filter, click the *Positive Filter* button
 image:images/PositiveFilter.jpg[Positive Filter].
 This includes only those documents that contain that value in the field.
@@ -43,8 +44,7 @@ field name. This includes only those documents that contain the field.
 To manually add a filter:
 
 . Click *Add Filter*. A popup will be displayed for you to create the filter.
-+
-image::images/add_filter.png[]
+
 . Choose a field to filter by. This list of fields will include fields from the
 index pattern you are currently querying against.
 +
@@ -78,26 +78,26 @@ turn off the suggestions by setting the advanced setting, `filterEditor:suggestV
 [[filter-pinning]]
 === Managing Filters
 
-To modify a filter, hover over it and click one of the action buttons.
+To modify a filter, click on it and click one of the action buttons.
 
 image::images/filter-allbuttons.png[]
 
 &nbsp;
 
-image:images/filter-enable.png[] Enable Filter :: Disable the filter without
-removing it. Click again to reenable the filter. Diagonal stripes indicate
-that a filter is disabled.
-image:images/filter-pin.png[] Pin Filter :: Pin the filter. Pinned filters
+Pin across all apps :: Pinned filters
 persist when you switch contexts in Kibana. For example, you can pin a filter
 in Discover and it remains in place when you switch to Visualize.
 Note that a filter is based on a particular index field--if the indices being
 searched don't contain the field in a pinned filter, it has no effect.
-image:images/filter-toggle.png[] Invert Filter :: Switch from a positive
-filter to a negative filter and vice-versa.
-image:images/filter-delete.png[] Remove Filter :: Remove the filter.
-image:images/filter-custom.png[] Edit Filter :: <<filter-edit, Edit the
+Edit Filter :: <<filter-edit, Edit the
 filter>> definition.  Enables you to manually update the filter and
 specify a label for the filter.
+Exclude results :: Switch from a positive
+filter to a negative filter and vice-versa.
+Temporarily disable :: Disable the filter without
+removing it. Click again to reenable the filter. Diagonal stripes indicate
+that a filter is disabled.
+Remove Filter :: Remove the filter.
 
 To apply a filter action to all of the applied filters,
 click *Actions* and select the action.

diff --git a/docs/discover/search.asciidoc b/docs/discover/search.asciidoc
@@ -1,7 +1,7 @@
 [[search]]
 == Searching your data
-You can search the indices that match the current index pattern by entering
-your search criteria in the Query bar. By default you can use Kibana's standard query language
+You can search the indices that match the current <<index-patterns, index pattern>> by entering
+your search criteria in the Query bar. By default you can use Kibana's <<kuery-query, standard query language>>
 which features autocomplete and a simple, easy to use syntax. Kibana's legacy query
 language (based on Lucene https://lucene.apache.org/core/2_9_4/queryparsersyntax.html[query syntax])
 is still available for the time being under the options menu in the Query Bar. When this

diff --git a/docs/discover/set-time-filter.asciidoc b/docs/discover/set-time-filter.asciidoc
@@ -1,7 +1,7 @@
 [[set-time-filter]]
 == Setting the time filter
 If your index contains time-based events, and a time-field is configured for the 
-selected index pattern, set a time filter that displays only the data within the 
+selected <<index-patterns, index pattern>>, set a time filter that displays only the data within the
 specified time range.
 
 You can use the time filter to change the time range, or select a specific time 

diff --git a/docs/discover/viewing-field-stats.asciidoc b/docs/discover/viewing-field-stats.asciidoc
@@ -11,4 +11,4 @@ they are available in the side bar if we uncheck "Hide missing fields".
 
 To view field data statistics, click the name of a field in the Fields list.
 
-image:images/filter-field.jpg[Field Statistics]
+image:images/filter-field.png[Field Statistics,height=317]
diff --git a/docs/images/add-bucket.png b/docs/images/add-bucket.png
diff --git a/docs/images/add_filter.png b/docs/images/add_filter.png
diff --git a/docs/images/add_filter_field.png b/docs/images/add_filter_field.png
diff --git a/docs/images/add_filter_operator.png b/docs/images/add_filter_operator.png
diff --git a/docs/images/add_filter_value.png b/docs/images/add_filter_value.png
diff --git a/docs/images/bar-terms-agg.jpg b/docs/images/bar-terms-agg.jpg
diff --git a/docs/images/bar-terms-agg.png b/docs/images/bar-terms-agg.png
diff --git a/docs/images/bar-terms-subagg.jpg b/docs/images/bar-terms-subagg.jpg
diff --git a/docs/images/bar-terms-subagg.png b/docs/images/bar-terms-subagg.png
diff --git a/docs/images/canvas-add-pages.gif b/docs/images/canvas-add-pages.gif
diff --git a/docs/images/canvas-align-elements.gif b/docs/images/canvas-align-elements.gif
diff --git a/docs/images/canvas-background-color-picker-min.gif b/docs/images/canvas-background-color-picker-min.gif
diff --git a/docs/images/canvas-create-URL-min.gif b/docs/images/canvas-create-URL-min.gif
diff --git a/docs/images/canvas-distribute-elements.gif b/docs/images/canvas-distribute-elements.gif
diff --git a/docs/images/canvas-element-select.gif b/docs/images/canvas-element-select.gif
diff --git a/docs/images/canvas-embed_workpad-min.gif b/docs/images/canvas-embed_workpad-min.gif
diff --git a/docs/images/canvas-fullscreen.gif b/docs/images/canvas-fullscreen.gif
diff --git a/docs/images/canvas-fullscreen.png b/docs/images/canvas-fullscreen.png
diff --git a/docs/images/canvas-generate-pdf-min.gif b/docs/images/canvas-generate-pdf-min.gif
diff --git a/docs/images/canvas-zoom-controls.png b/docs/images/canvas-zoom-controls.png
diff --git a/docs/images/canvas_share_autoplay_480-min.gif b/docs/images/canvas_share_autoplay_480-min.gif
diff --git a/docs/images/canvas_share_hidetoolbar_480-min.gif b/docs/images/canvas_share_hidetoolbar_480-min.gif
diff --git a/docs/images/color-picker.png b/docs/images/color-picker.png
diff --git a/docs/images/edit_filter_query.png b/docs/images/edit_filter_query.png
diff --git a/docs/images/edit_filter_query_json.png b/docs/images/edit_filter_query_json.png
diff --git a/docs/images/filter-allbuttons.png b/docs/images/filter-allbuttons.png
diff --git a/docs/images/filter-custom.png b/docs/images/filter-custom.png
diff --git a/docs/images/filter-delete.png b/docs/images/filter-delete.png
diff --git a/docs/images/filter-enable.png b/docs/images/filter-enable.png
diff --git a/docs/images/filter-field.jpg b/docs/images/filter-field.jpg
diff --git a/docs/images/filter-field.png b/docs/images/filter-field.png
diff --git a/docs/images/filter-pin.png b/docs/images/filter-pin.png
diff --git a/docs/images/filter-toggle.png b/docs/images/filter-toggle.png
diff --git a/docs/images/gauge.png b/docs/images/gauge.png
diff --git a/docs/images/goal.png b/docs/images/goal.png
diff --git a/docs/images/lens_data_info-min.gif b/docs/images/lens_data_info-min.gif
diff --git a/docs/images/lens_drag_drop-min.gif b/docs/images/lens_drag_drop-min.gif
diff --git a/docs/images/lens_suggestions-min.gif b/docs/images/lens_suggestions-min.gif
diff --git a/docs/images/time_range_per_panel-min.gif b/docs/images/time_range_per_panel-min.gif
diff --git a/docs/images/visualize-date-histogram-split-1.png b/docs/images/visualize-date-histogram-split-1.png
diff --git a/docs/images/visualize-date-histogram-split-2.png b/docs/images/visualize-date-histogram-split-2.png
diff --git a/docs/images/visualize-date-histogram.png b/docs/images/visualize-date-histogram.png
diff --git a/docs/images/visualize-drag-reorder.png b/docs/images/visualize-drag-reorder.png
diff --git a/docs/logs/images/analysis-tab-create-ml-job.png b/docs/logs/images/analysis-tab-create-ml-job.png
diff --git a/docs/logs/images/log-rate-anomalies.png b/docs/logs/images/log-rate-anomalies.png
diff --git a/docs/logs/images/log-rate-entries.png b/docs/logs/images/log-rate-entries.png
diff --git a/docs/logs/images/log-time-filter.png b/docs/logs/images/log-time-filter.png
diff --git a/docs/logs/index.asciidoc b/docs/logs/index.asciidoc
@@ -27,3 +27,5 @@ include::getting-started.asciidoc[]
 include::using.asciidoc[]
 
 include::configuring.asciidoc[]
+
+include::log-rate.asciidoc[]
diff --git a/docs/logs/log-rate.asciidoc b/docs/logs/log-rate.asciidoc
@@ -0,0 +1,94 @@
+[role="xpack"]
+[[xpack-logs-analysis]]
+== Detecting and inspecting log anomalies
+
+beta::[]
+
+When the {ml} {anomaly-detect} features are enabled,
+you can use the **Log rate** page in the Logs app.
+**Log rate** helps you to detect and inspect log anomalies and the log partitions where the log anomalies occur.
+This means you can easily spot anomalous behavior without significant human intervention --
+no more manually sampling log data, calculating rates, and determining if rates are normal.
+
+*Log rate* automatically highlights periods of time where the log rate is outside expected bounds,
+and therefore may be anomalous.
+You can use this information as a basis for further investigations.
+For example:
+
+* A significant drop in the log rate might suggest that a piece of infrastructure stopped responding,
+and thus we're serving less requests.
+* A spike in the log rate could denote a DDoS attack.
+This may lead to an investigation of IP addresses from incoming requests.
+
+You can also view log anomalies directly in the <<xpack-ml-anomalies,Machine Learning app>>.
+
+[float]
+[[logs-analysis-create-ml-job]]
+=== Enable log rate analysis and anomaly detection
+
+Create a machine learning job to enable log rate analysis and anomaly detection.
+
+[role="screenshot"]
+image::logs/images/analysis-tab-create-ml-job.png[Create machine learning job]
+
+1. To enable log rate analysis and anomaly detection,
+you must first create your own {kibana-ref}/xpack-spaces.html[space].
+2. Within a space, navigate to the Logs app and select *Log rate*.
+Here, you'll be prompted to create a machine learning job which will carry out the log rate analysis.
+3. Choose a time range for the machine learning analysis.
+4. Add the Indices that contain the logs you want to analyze.
+5. Click *Create ML job*.
+6. You're now ready to analyze your log partitions.
+
+Even though the machine learning job's time range is fixed,
+you can still use the time filter to adjust the results that are shown in your analysis.
+
+[role="screenshot"]
+image::logs/images/log-time-filter.png[Log rate time filter]
+
+[float]
+[[logs-analysis-entries-chart]]
+=== Log entries chart
+
+The log entries chart shows an overall, color-coded visualization of the log entry rate,
+partitioned according to the value of the Elastic Common Schema (ECS)
+{ecs-ref}/ecs-event.html[`event.dataset`] field.
+This chart helps you quickly spot increases or decreases in each partition's log rate.
+
+[role="screenshot"]
+image::logs/images/log-rate-entries.png[Log rate entries chart]
+
+If you have a lot of log partitions, use the following to filter your data:
+
+* Hover over a time range to see the log rate for each partition.
+* Click or hover on a partition name to show, hide, or highlight the partition values.
+
+[float]
+[[logs-analysis-anomalies-chart]]
+=== Anomalies charts
+
+The Anomalies chart shows the time range where anomalies were detected.
+The typical rate values are shown in grey, while the anomalous regions are color-coded and superimposed on top.
+
+[role="screenshot"]
+image::logs/images/log-rate-anomalies.png[Log rate entries chart]
+
+When a time range is flagged as anomalous,
+the machine learning algorithms have detected unusual log rate activity.
+This might be because:
+
+* The log rate is significantly higher than usual.
+* The log rate is significantly lower than usual.
+* Other anomalous behavior has been detected.
+For example, the log rate is within bounds, but not fluctuating when it is expected to.
+
+The level of anomaly detected in a time period is color-coded, from red, orange, yellow, to blue.
+Red indicates a critical anomaly level, while blue is a warning level.
+
+To help you further drill down into a potential anomaly,
+you can view an anomaly chart for each individual partition:
+
+Anomaly scores range from 0 (no anomalies) to 100 (critical).
+
+To analyze the anomalies in more detail, click *Analyze in ML*, which opens the
+{kibana-ref}/xpack-ml.html[Anomaly Explorer in Machine Learning].
diff --git a/docs/logs/using.asciidoc b/docs/logs/using.asciidoc
@@ -78,8 +78,19 @@ This opens the *Log event document details* fly-out that shows the fields associ
 To quickly filter the logs stream by one of the field values, in the log event details, click the *View event with filter* icon image:logs/images/logs-view-event-with-filter.png[View event icon] beside the field.
 This automatically adds a search filter to the logs stream to filter the entries by this field and value.
 
-To see other actions related to the event, in the log event details, click *Actions*.
-Depending on the event and the features you have installed and configured, you may also be able to:
+[float]
+[[view-log-anomalies]]
+=== View log anomalies
+
+When the machine learning anomaly detection features are enabled, click *Log rate*, which allows you to
+<<xpack-logs-analysis,use machine learning to detect and inspect anomalies>> in your log data.
+
+[float]
+[[logs-integrations]]
+=== Logs app integrations
+
+To see other actions related to the event, click *Actions* in the log event details.
+Depending on the event and the features you have configured, you may also be able to:
 
 * Select *View status in Uptime* to <<uptime-overview, view related uptime information>> in the *Uptime* app.
 * Select *View in APM* to <<traces, view related APM traces>> in the *APM* app.
diff --git a/docs/settings/reporting-settings.asciidoc b/docs/settings/reporting-settings.asciidoc
@@ -26,9 +26,24 @@ setting to preserve the same key across multiple restarts and multiple instances
 [[reporting-kibana-server-settings]]
 ==== Kibana server settings
 
-Reporting uses the Kibana interface to generate reports. In most cases, you don't need
-to configure Reporting to communicate with Kibana. However, if you use a reverse-proxy
-to access Kibana, you must set the proxy port, protocol, and hostname.
+Reporting opens the {kib} web interface in a server process to generate 
+screenshots of {kib} visualizations. In most cases, the default settings 
+will work and you don't need to configure Reporting to communicate with {kib}. 
+However, if your client connections must go through a reverse-proxy
+to access {kib}, Reporting configuration must have the proxy port, protocol, 
+and hostname set in the `xpack.reporting.kibanaServer.*` settings.
+
+[NOTE] 
+====
+If a reverse-proxy carries encrypted traffic from end-user 
+clients back to a {kib} server, the proxy port, protocol, and hostname 
+in Reporting settings must be valid for the encryption that the Reporting 
+browser will receive. Encrypted communications will fail if there are 
+mismatches in the host information between the request and the certificate on the server.
+
+Configuring the `xpack.reporting.kibanaServer` settings to point to a
+proxy host requires that the Kibana server has network access to the proxy.
+====
 
 `xpack.reporting.kibanaServer.port`::
 The port for accessing Kibana, if different from the `server.port` value.
@@ -39,8 +54,6 @@ The protocol for accessing Kibana, typically `http` or `https`.
 `xpack.reporting.kibanaServer.hostname`::
 The hostname for accessing {kib}, if different from the `server.host` value.
 
-NOTE: Configuring the `xpack.reporting.kibanaServer` settings to point to a
-proxy host requires that the Kibana server has network access to the proxy.
 
 [float]
 [[reporting-job-queue-settings]]

diff --git a/docs/settings/security-settings.asciidoc b/docs/settings/security-settings.asciidoc
@@ -40,6 +40,8 @@ An arbitrary string of 32 characters or more that is used to encrypt credentials
 in a cookie. It is crucial that this key is not exposed to users of {kib}. By
 default, a value is automatically generated in memory. If you use that default
 behavior, all sessions are invalidated when {kib} restarts.
+In addition, high-availability deployments of {kib} will behave unexpectedly 
+if this setting isn't the same for all instances of {kib}.
 
 `xpack.security.secureCookies`::
 Sets the `secure` flag of the session cookie. The default value is `false`. It

diff --git a/docs/user/discover.asciidoc b/docs/user/discover.asciidoc
@@ -4,7 +4,7 @@
 [partintro]
 --
 *Discover* enables you to explore your data with {kib}'s data discovery functions. 
-You have access to every document in every index that matches the selected index pattern. 
+You have access to every document in every index that matches the selected <<index-patterns, index pattern>>.
 You can submit search queries, filter the search results, and view document data.
 You can also see the number of documents that match the search query and get field value statistics. 
 If a time field is configured for the selected index pattern, the distribution of 

diff --git a/docs/user/reporting/reporting-troubleshooting.asciidoc b/docs/user/reporting/reporting-troubleshooting.asciidoc
@@ -7,18 +7,6 @@
 
 Having trouble? Here are solutions to common problems you might encounter while using Reporting.
 
-[float]
-=== Verbose logs
-{kib} server logs have a lot of useful information for troubleshooting and understanding how things work. If you're having any issues at
-all, the full logs from Reporting will be the first place to look. In `kibana.yml`:
-
-[source,yaml]
---------------------------------------------------------------------------------
-logging.verbose: true
---------------------------------------------------------------------------------
-
-For more information about logging, see <<logging-verbose,Kibana configuration settings>>.
-
 [float]
 [[reporting-troubleshooting-system-dependencies]]
 === System dependencies
@@ -98,3 +86,28 @@ the CAP_SYS_ADMIN capability.
 
 Elastic recommends that you research the feasibility of enabling unprivileged user namespaces before disabling the sandbox. An exception
 is if you are running Kibana in Docker because the container runs in a user namespace with the built-in seccomp/bpf filters.
+
+[float]
+=== Verbose logs
+{kib} server logs have a lot of useful information for troubleshooting and understanding how things work. If you're having any issues at
+all, the full logs from Reporting will be the first place to look. In `kibana.yml`:
+
+[source,yaml]
+--------------------------------------------------------------------------------
+logging.verbose: true
+--------------------------------------------------------------------------------
+
+For more information about logging, see <<logging-verbose,Kibana configuration settings>>.
+
+=== Puppeteer debug logs
+The Chromium browser that {kib} launches on the server is driven by a NodeJS library for Chromium called Puppeteer. The Puppeteer library
+has its own command-line method to generate its own debug logs, which can sometimes be helpful, particularly to figure out if a problem is
+caused by Kibana or Chromium. See more at https://github.com/GoogleChrome/puppeteer/blob/v1.19.0/README.md#debugging-tips
+
+Using Puppeteer's debug method when launching Kibana would look like:
+> Enable verbose logging - internal DevTools protocol traffic will be logged via the debug module under the puppeteer namespace.
+>     ```
+>     env DEBUG="puppeteer:*" ./bin/kibana
+>     ```
+
+The Puppeteer logs are very verbose and could possibly contain sensitive information. Handle the generated output with care.