Security usage data #110548

jportner · 2021-08-30T22:31:29Z

Adds these five fields:

stack_stats.kibana.plugins.core.config.elasticsearch.principal
stack_stats.kibana.plugins.security.auditLoggingType
stack_stats.kibana.plugins.security.sessionIdleTimeoutInMinutes
stack_stats.kibana.plugins.security.sessionExpirationInMinutes
stack_stats.kibana.plugins.security.sessionCleanupInMinutes

jportner

Author's notes for reviewers

src/core/server/core_usage_data/core_usage_data_service.ts

jportner · 2021-08-30T22:59:57Z

x-pack/plugins/security/server/usage_collector/security_usage_collector.ts

  auditLoggingEnabled: boolean;
+  auditLoggingType?: 'ecs' | 'legacy';


Since we already have auditLoggingEnabled and we can't change the existing mapping for that field, I added a second optional field for auditLoggingType. The vast majority of clusters do not have audit logging enabled.

src/core/server/core_usage_data/types.ts

legrego · 2021-08-31T17:02:51Z

x-pack/plugins/security/server/config.ts

@@ -391,11 +391,18 @@ export function createConfig(
 function getSessionConfig(session: RawConfigType['session'], providers: ProvidersConfigType) {
  return {
    cleanupInterval: session.cleanupInterval,
-    getExpirationTimeouts({ type, name }: AuthenticationProvider) {
+    getExpirationTimeouts(provider?: AuthenticationProvider) {


I think we're missing unit tests for this change in config.test.ts. The existing tests all assume that this parameter will be defined

Ah good point. I think I'll change it to provider: AuthenticationProvider | undefined too to prevent accidental misuse. We only need to use this for usage data collection.

Edit: done in a9959d1.

TinaHeiligers

Telemetry and core-related changes LGTM.

TinaHeiligers · 2021-08-31T17:52:43Z

src/plugins/telemetry/schema/oss_plugins.json

+                "principal": {
+                  "type": "keyword",
+                  "_meta": {
+                    "description": "Indicates what elasticsearch user or service account is configured, if any."


nit: It would help us a lot if we knew what was grouped under other without potentially exposing too much information in the payload. However, IIRC we strip the description out from the usage data before shipping it off to the telemetry cluster so it should be safe enough to add a little more in the description.

Okay, I'll change this description to:

Indicates how Kibana authenticates itself to Elasticsearch. If elasticsearch.username is configured, this can be any of: "elastic_user", "kibana_user", "kibana_system_user", or "other_user". Otherwise, if elasticsearch.serviceAccountToken is configured, this will be "kibana_service_account". Otherwise, this value will be "unknown", because some other principal might be used to authenticate Kibana to Elasticsearch (such as an x509 certificate), or authentication may be skipped altogether.

I think "unknown" is better than "other", plus it's less likely to be confused with "other_user".

Edit: done in 0317d33. Unfortunately I could not use a string template or string concatenation to break this up, as it caused the telemetry_check.js script to fail. So I had to write this description in one line 😅

kibanamachine · 2021-08-31T22:52:39Z

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

💔 Build #149960 failed 0317d33
💚 Build #149873 succeeded 94379a6
💚 Build #149629 succeeded 67e69a7
💔 Build #149626 failed 84234bd

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

kibanamachine · 2021-09-01T14:38:11Z

💔 Backport failed

Status	Branch	Result
❌	7.15	Commit could not be cherrypicked due to conflicts
❌	7.x	Commit could not be cherrypicked due to conflicts

To backport manually run:
node scripts/backport --pr 110548

# Conflicts: # x-pack/plugins/security/server/config.test.ts # x-pack/plugins/security/server/config.ts

* Security usage data (#110548) # Conflicts: # x-pack/plugins/security/server/config.test.ts # x-pack/plugins/security/server/config.ts * Fix unit test Messed it up when fixing the merge conflict * Fix merge error * FIX TESTS AGAIN

* Security usage data (#110548) # Conflicts: # x-pack/plugins/security/server/config.test.ts # x-pack/plugins/security/server/config.ts * Fix bad merge * FIX TESTS AGAIN Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

jportner added 2 commits August 30, 2021 17:01

Audit logging usage data

c8f7870

ES username usage data

84234bd

jportner added v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.15.0 v7.16.0 labels Aug 30, 2021

Fix mock

67e69a7

jportner commented Aug 30, 2021

View reviewed changes

jportner marked this pull request as ready for review August 30, 2021 23:01

jportner requested review from a team as code owners August 30, 2021 23:01

legrego reviewed Aug 30, 2021

View reviewed changes

src/core/server/core_usage_data/types.ts Outdated Show resolved Hide resolved

jportner added 5 commits August 31, 2021 08:26

Merge branch 'master' into issue-110532-security-usage-data

0ed956f

Change core ES usage collection

bc1842a

Add session timeout usage data

f7730fc

Rename session expiration usage data

fb0850b

Add session cleanup usage data

94379a6

jportner requested a review from legrego August 31, 2021 16:49

legrego reviewed Aug 31, 2021

View reviewed changes

TinaHeiligers approved these changes Aug 31, 2021

View reviewed changes

jportner added 3 commits August 31, 2021 15:01

Merge branch 'master' into issue-110532-security-usage-data

d8f485c

Tweak security config

a9959d1

Change elasticsearch.principal usage data/description

0317d33

jportner requested a review from legrego August 31, 2021 20:18

Fix type check

d96ced2

jportner enabled auto-merge (squash) September 1, 2021 12:06

jportner added the auto-backport Deprecated - use backport:version if exact versions are needed label Sep 1, 2021

legrego approved these changes Sep 1, 2021

View reviewed changes

jportner merged commit b17d87e into elastic:master Sep 1, 2021

jportner added a commit to jportner/kibana that referenced this pull request Sep 1, 2021

Security usage data (elastic#110548)

6997c6d

# Conflicts: # x-pack/plugins/security/server/config.test.ts # x-pack/plugins/security/server/config.ts

This was referenced Sep 1, 2021

[7.x] Security usage data (#110548) #110822

Merged

[7.15] Security usage data (#110548) #110823

Merged

jportner added a commit to jportner/kibana that referenced this pull request Sep 1, 2021

Security usage data (elastic#110548)

e187d7d

# Conflicts: # x-pack/plugins/security/server/config.test.ts # x-pack/plugins/security/server/config.ts

jportner deleted the issue-110532-security-usage-data branch September 3, 2021 19:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Security usage data #110548

Security usage data #110548

jportner commented Aug 30, 2021 •

edited

Loading

jportner left a comment

jportner Aug 30, 2021

legrego Aug 31, 2021

jportner Aug 31, 2021 •

edited

Loading

TinaHeiligers left a comment

TinaHeiligers Aug 31, 2021

jportner Aug 31, 2021 •

edited

Loading

kibanamachine commented Aug 31, 2021

kibanamachine commented Sep 1, 2021

		auditLoggingEnabled: boolean;
		auditLoggingType?: 'ecs' \| 'legacy';

Security usage data #110548

Security usage data #110548

Conversation

jportner commented Aug 30, 2021 • edited Loading

jportner left a comment

Choose a reason for hiding this comment

jportner Aug 30, 2021

Choose a reason for hiding this comment

legrego Aug 31, 2021

Choose a reason for hiding this comment

jportner Aug 31, 2021 • edited Loading

Choose a reason for hiding this comment

TinaHeiligers left a comment

Choose a reason for hiding this comment

TinaHeiligers Aug 31, 2021

Choose a reason for hiding this comment

jportner Aug 31, 2021 • edited Loading

Choose a reason for hiding this comment

kibanamachine commented Aug 31, 2021

💚 Build Succeeded

Metrics [docs]

History

kibanamachine commented Sep 1, 2021

💔 Backport failed

jportner commented Aug 30, 2021 •

edited

Loading

jportner Aug 31, 2021 •

edited

Loading

jportner Aug 31, 2021 •

edited

Loading