[Request] 8.16 Security Assistant Custom Knowledge Bases #5337
Assignees
Labels
Effort: X-Large
Issues that require several days or weeks substantial planning, research, writing, and testing
Team: GenAI
Generative AI team
v8.16.0
Comments
This was referenced Jun 6, 2024
benironside
added
Team: GenAI
8 tasks
spong
added a commit
to elastic/kibana
that referenced
this issue
Jun 14, 2024
## Summary Migrates our existing RAG pipeline to use LangGraph, and adds tools for Knowledge Base retrieval/storage. When the `assistantKnowledgeBaseByDefault` FF is enabled, a new branch, `callAssistantGraph()`, is taken in `postActionsConnectorExecuteRoute` that exercises the LangGraph implementation. This is a drop-in replacement for the existing `callAgentExecutor()` in effort to keep adoption as clean and easy as possible. The new control flow is as follows: `postActionsConnectorExecuteRoute` -> `callAssistantGraph()` -> `getDefaultAssistantGraph()` -> `isStreamingEnabled ? streamGraph() : invokeGraph()` Graph creation is isolated to `getDefaultAssistantGraph()`, and execution (streaming or not) has been extracted to `streamGraph()` and `invokeGraph()` respectively. Note: Streaming currently only works with `ChatOpenAI` models, but `SimpleChatModelStreaming` was de-risked and just need to discuss potential solutions with @stephmilovic. See [comment here](https://github.com/elastic/kibana/pull/184554/files#diff-ad87c5621b231a40810419fc1e56f28aeb4f8328e125e465dfe95ae0e1c305b8R97-R98). #### DefaultAssistantGraph To start with a predictable and piecemeal migration, our existing `agentExecutor` pipeline has been recreated in LangGraph. It consists of a single agent node, either `OpenAIFunctionsAgent`, or `StructuredChatAgent` (depending on the backing LLM), a tool executing node, and a conditional edge that routes between the two nodes until there are no more function calls chosen by the agent. This varies from our initial implementation in that multiple tool calls are now supported, so a user could ask about their alerts AND retrieve additional knowledge base information in the same response. > [!NOTE] > While `chat_history` has been plumbed into the graph, after discussing with @YulNaumenko we decided to wait to plumb the rest of persistence into the graph until #184485 is merged. I had already plumbed through the `chatTitleGeneration` node ([here](https://github.com/elastic/kibana/pull/184554/files#diff-26038489e9a3f1a14c5ea2ac2954671973d833349ef3ffaddcf9b29ce9e2b96eR33)), and so will just need to include initial conversation creation and append/update operations. #### Knowledge History & KB Tools Knowledge History is now always added in the initial prompt for any KB documents marked as `required`, and two new tools were added for creating and recalling KB entries from within the conversation, `KnowledgeBaseWriteTool` and `KnowledgeBaseRetrievalTool` respectively. All three methods of storing and retrieving KB content use the `kbDataClient` for access, and scopes all requests to the authenticatedUser that made the initial request. Additional Notes: * LangChain dependencies have been updated, and a new dependency on `LangGraph` has been added. ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind a FF, documentation to be added once feature is complete. Tracked in elastic/security-docs#5337. - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios * Test coverage in progress... --------- Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
spong
added a commit
to elastic/kibana
that referenced
this issue
Jun 19, 2024
…ng Knowledge Base Entries (#184974) ## Summary This PR adds client hooks and basic REST API's for accessing and mutating Knowledge Base Entries. This is in support of @angorayc building out the new Knowledge Base settings interface. Change set includes: - [X] Refactors existing KB client hooks from `x-pack/packages/kbn-elastic-assistant/impl/knowledge_base` to be co-located next to the API methods where we put all our other hooks: `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/knowledge_base` - [X] Refactors existing KB API calls and associated tests out of `kbn-elastic-assistant/impl/assistant/api/index.tsx` and into `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/knowledge_base/api.tsx` - [X] Adds new `find_knowledge_base_entries_route.schema.yaml` OAS for the supporting `/internal/elastic_assistant/knowledge_base/entries/_find` route - [X] Refactors `SortOrder` out of existing OAS's into the shared `schemas/common_attributes.schema.yaml` ### Client Hooks & Routes Adds new `useKnowledgeBaseEntries()` hook and corresponding `/knowledge_base/entries/_find` route for returning paginated KB Entries to populate the KB table in settings. E.g. ``` ts const { assistantFeatures: { assistantKnowledgeBaseByDefault: enableKnowledgeBaseByDefault }, http, toasts, } = useAssistantContext(); const { data: kbEntries, isLoading: isLoadingEntries } = useKnowledgeBaseEntries({ http }); ``` ###### Sample Response ``` json { "perPage": 20, "page": 1, "total": 145, "data": [ { "timestamp": "2024-06-05T21:19:56.482Z", "id": "CtBF6o8BSQy1Bdxt2FHz", "createdAt": "2024-06-05T21:19:56.482Z", "createdBy": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0", "updatedAt": "2024-06-05T21:19:56.482Z", "updatedBy": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0", "users": [ { "id": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0", "name": "elastic" } ], "metadata": { "kbResource": "security_labs", "source": "/Users/garrettspong/dev/kibana-main/x-pack/plugins/elastic_assistant/server/knowledge_base/security_labs/2022_elastic_global_threat_report_announcement.mdx", "required": false }, "namespace": "default", "text": "[Source Content Here]", "vector": { "modelId": ".elser_model_2", "tokens": { "2": 0.06595266, ... } } }, ... ] } ``` Response is the full newly created `entry`. Same format for the entry as above in the `_find` API, and the `KnowledgeBaseEntries` cache is invalidated. Adds new `useCreateKnowledgeBaseEntry()` hook and corresponding `/knowledge_base/entries` route for creating new KB Entries ``` ts const entry: KnowledgeBaseEntryCreateProps = { metadata: { kbResource: 'user', required: true, source: 'user', }, text: 'Useful information about the user', }; const { mutate: createEntry, isLoading: isCreatingEntry } = useCreateKnowledgeBaseEntry({ http, }); await createEntry(entry); ``` Adds new `useDeleteKnowledgeBaseEntries()` hook and corresponding `/knowledge_base/entries/_bulk_action` route for deleting existing KB Entries. I left a TODO to plumb through `delete_by_query` so we can add a filter bar to the table. Need to confirm if we can do pagination with similarity search as well. ``` ts const { mutate: deleteEntries, isLoading: isDeletingEntries } = useDeleteKnowledgeBaseEntries({ http, }); await deleteEntries({ ids: ['YOE_CZABSQy1BdxtAGbs'] }) ``` See `KnowledgeBaseEntryBulkCrudActionResponse` for response formats. `KnowledgeBaseEntries` cache is invalidated upon delete. ### Checklist Delete any items that are not applicable to this PR. - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] API tests will need to be rounded out as we finalize functionality behind the feature flag --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
8 tasks
seanrathier
pushed a commit
to seanrathier/kibana
that referenced
this issue
Jun 21, 2024
…ng Knowledge Base Entries (elastic#184974) ## Summary This PR adds client hooks and basic REST API's for accessing and mutating Knowledge Base Entries. This is in support of @angorayc building out the new Knowledge Base settings interface. Change set includes: - [X] Refactors existing KB client hooks from `x-pack/packages/kbn-elastic-assistant/impl/knowledge_base` to be co-located next to the API methods where we put all our other hooks: `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/knowledge_base` - [X] Refactors existing KB API calls and associated tests out of `kbn-elastic-assistant/impl/assistant/api/index.tsx` and into `x-pack/packages/kbn-elastic-assistant/impl/assistant/api/knowledge_base/api.tsx` - [X] Adds new `find_knowledge_base_entries_route.schema.yaml` OAS for the supporting `/internal/elastic_assistant/knowledge_base/entries/_find` route - [X] Refactors `SortOrder` out of existing OAS's into the shared `schemas/common_attributes.schema.yaml` ### Client Hooks & Routes Adds new `useKnowledgeBaseEntries()` hook and corresponding `/knowledge_base/entries/_find` route for returning paginated KB Entries to populate the KB table in settings. E.g. ``` ts const { assistantFeatures: { assistantKnowledgeBaseByDefault: enableKnowledgeBaseByDefault }, http, toasts, } = useAssistantContext(); const { data: kbEntries, isLoading: isLoadingEntries } = useKnowledgeBaseEntries({ http }); ``` ###### Sample Response ``` json { "perPage": 20, "page": 1, "total": 145, "data": [ { "timestamp": "2024-06-05T21:19:56.482Z", "id": "CtBF6o8BSQy1Bdxt2FHz", "createdAt": "2024-06-05T21:19:56.482Z", "createdBy": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0", "updatedAt": "2024-06-05T21:19:56.482Z", "updatedBy": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0", "users": [ { "id": "u_mGBROF_q5bmFCATbLXAcCwKa0k8JvONAwSruelyKA5E_0", "name": "elastic" } ], "metadata": { "kbResource": "security_labs", "source": "/Users/garrettspong/dev/kibana-main/x-pack/plugins/elastic_assistant/server/knowledge_base/security_labs/2022_elastic_global_threat_report_announcement.mdx", "required": false }, "namespace": "default", "text": "[Source Content Here]", "vector": { "modelId": ".elser_model_2", "tokens": { "2": 0.06595266, ... } } }, ... ] } ``` Response is the full newly created `entry`. Same format for the entry as above in the `_find` API, and the `KnowledgeBaseEntries` cache is invalidated. Adds new `useCreateKnowledgeBaseEntry()` hook and corresponding `/knowledge_base/entries` route for creating new KB Entries ``` ts const entry: KnowledgeBaseEntryCreateProps = { metadata: { kbResource: 'user', required: true, source: 'user', }, text: 'Useful information about the user', }; const { mutate: createEntry, isLoading: isCreatingEntry } = useCreateKnowledgeBaseEntry({ http, }); await createEntry(entry); ``` Adds new `useDeleteKnowledgeBaseEntries()` hook and corresponding `/knowledge_base/entries/_bulk_action` route for deleting existing KB Entries. I left a TODO to plumb through `delete_by_query` so we can add a filter bar to the table. Need to confirm if we can do pagination with similarity search as well. ``` ts const { mutate: deleteEntries, isLoading: isDeletingEntries } = useDeleteKnowledgeBaseEntries({ http, }); await deleteEntries({ ids: ['YOE_CZABSQy1BdxtAGbs'] }) ``` See `KnowledgeBaseEntryBulkCrudActionResponse` for response formats. `KnowledgeBaseEntries` cache is invalidated upon delete. ### Checklist Delete any items that are not applicable to this PR. - [ ] ~Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)~ - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] API tests will need to be rounded out as we finalize functionality behind the feature flag --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
|
I've synced with @benironside, so just an update here: only some of this new Knowledge Base work is shipping in |
spong
added a commit
to elastic/kibana
that referenced
this issue
Sep 10, 2024
…o support `IndexEntries` (#186566) ## Summary This is a follow-up to #184974 that updates the KB Entries API's (and underlying schema) to support `IndexEntries` in addition to `DocumentEntries`. `IndexEntries` are entries in the Knowledge Base that are not backed by an embedded raw text source, but rather by an entire Index or Data Stream. The user can set the data source name, the specific field to query (must be ELSER embeddings in this initial implementation), and a description for when the assistant should search this data source for Knowledge Base content. This essentially enables the user to create custom retrieval tools backed by their own data. The changes in this PR, as with the other recent KB enhancements, are behind the following feature flag: ``` xpack.securitySolution.enableExperimental: - 'assistantKnowledgeBaseByDefault' ``` however as code change is required to test the new mappings. For this you can update the `knowledgeBaseDataStream` in `x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts` to ```ts this.knowledgeBaseDataStream = this.createDataStream({ resource: 'knowledgeBase', kibanaVersion: options.kibanaVersion, fieldMap: knowledgeBaseFieldMapV2, // Update this to the V2 mapping }); ``` Change set includes: - [X] ES Knowledge Base data stream schema and OAS has been updated to support `IndexEntries`. - [X] OAS schema files have been moved to the `/entries` sub-directory - [ ] Backend KB services have been updated to support `IndexEntries` - [X] Storage methods updated - [ ] Retrieval methods updated (will round out these endpoint when working the UI next) --- With these API changes, I've also introduced a few sample `*.http` files for easier development/testing. These files are supported out of the box in JetBrains IDE's or in VSCode with the [httpyac](https://httpyac.github.io/) (and many other) extensions. Since the configuration for these files includes a `-` in the name, that's why you'll see a few @elastic/kibana-operations files updated. You can read more about `http` files [here](https://www.jetbrains.com/help/webstorm/http-client-in-product-code-editor.html) and for the spec see this repo [here](https://github.com/JetBrains/http-request-in-editor-spec/blob/master/spec.md). If we find these useful, we could add support to our [OpenAPI Generator](https://openapi-generator.tech/docs/generators/jetbrains-http-client) to create these automatically. They currently live co-located next to the OAS and generated schema files here: ``` x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/bulk_crud_knowledge_base_entries_route.http x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http ``` and the main config here: ``` x-pack/packages/kbn-elastic-assistant-common/env/http-client.env.json ``` The `x-pack/packages/kbn-elastic-assistant-common/.gitignore` has been updated to ignore `http-client.private.env.json` files locally, which is how you can override the config as you'd like. This is helpful to add variables like `basePath` as below: ``` { "dev": { "basePath": "/kbn" } } ``` To use them, just open the corresponding `*.http` for the API you want to test, and click `Send`, and the response will open in another tab. Here is what that looks like for creating one of the new `IndexEntry` KB documents that have been introduced in this PR: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/c9e70d1a-28d2-4eb3-9853-ab6d8e1c7acf" /> </p> ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
gergoabraham
pushed a commit
to gergoabraham/kibana
that referenced
this issue
Sep 13, 2024
…o support `IndexEntries` (elastic#186566) ## Summary This is a follow-up to elastic#184974 that updates the KB Entries API's (and underlying schema) to support `IndexEntries` in addition to `DocumentEntries`. `IndexEntries` are entries in the Knowledge Base that are not backed by an embedded raw text source, but rather by an entire Index or Data Stream. The user can set the data source name, the specific field to query (must be ELSER embeddings in this initial implementation), and a description for when the assistant should search this data source for Knowledge Base content. This essentially enables the user to create custom retrieval tools backed by their own data. The changes in this PR, as with the other recent KB enhancements, are behind the following feature flag: ``` xpack.securitySolution.enableExperimental: - 'assistantKnowledgeBaseByDefault' ``` however as code change is required to test the new mappings. For this you can update the `knowledgeBaseDataStream` in `x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts` to ```ts this.knowledgeBaseDataStream = this.createDataStream({ resource: 'knowledgeBase', kibanaVersion: options.kibanaVersion, fieldMap: knowledgeBaseFieldMapV2, // Update this to the V2 mapping }); ``` Change set includes: - [X] ES Knowledge Base data stream schema and OAS has been updated to support `IndexEntries`. - [X] OAS schema files have been moved to the `/entries` sub-directory - [ ] Backend KB services have been updated to support `IndexEntries` - [X] Storage methods updated - [ ] Retrieval methods updated (will round out these endpoint when working the UI next) --- With these API changes, I've also introduced a few sample `*.http` files for easier development/testing. These files are supported out of the box in JetBrains IDE's or in VSCode with the [httpyac](https://httpyac.github.io/) (and many other) extensions. Since the configuration for these files includes a `-` in the name, that's why you'll see a few @elastic/kibana-operations files updated. You can read more about `http` files [here](https://www.jetbrains.com/help/webstorm/http-client-in-product-code-editor.html) and for the spec see this repo [here](https://github.com/JetBrains/http-request-in-editor-spec/blob/master/spec.md). If we find these useful, we could add support to our [OpenAPI Generator](https://openapi-generator.tech/docs/generators/jetbrains-http-client) to create these automatically. They currently live co-located next to the OAS and generated schema files here: ``` x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/bulk_crud_knowledge_base_entries_route.http x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http ``` and the main config here: ``` x-pack/packages/kbn-elastic-assistant-common/env/http-client.env.json ``` The `x-pack/packages/kbn-elastic-assistant-common/.gitignore` has been updated to ignore `http-client.private.env.json` files locally, which is how you can override the config as you'd like. This is helpful to add variables like `basePath` as below: ``` { "dev": { "basePath": "/kbn" } } ``` To use them, just open the corresponding `*.http` for the API you want to test, and click `Send`, and the response will open in another tab. Here is what that looks like for creating one of the new `IndexEntry` KB documents that have been introduced in this PR: <p align="center"> <img width="500" src="https://github.com/user-attachments/assets/c9e70d1a-28d2-4eb3-9853-ab6d8e1c7acf" /> </p> ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
8 tasks
spong
added a commit
to elastic/kibana
that referenced
this issue
Sep 25, 2024
…192665) ## Summary This PR updates the Knowledge Base Management Settings page to use the new `entries` API introduced in #186566. Many thanks to @angorayc for her work on the Assistant Management Settings overhaul, and initial implementation of this new KB Management UI over in #186847. <p align="center"> <img width="600" src="https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa" /> </p> ### Feature Flag & Setup The changes in this PR, as with the other [recent V2 KB enhancements](#186566), are behind the following feature flag: ``` xpack.securitySolution.enableExperimental: - 'assistantKnowledgeBaseByDefault' ``` ~They also require a code change in the `AIAssistantService` to enable the new mapping (since setup happens on plugin start before FF registration), so be sure to update `fieldMap` to `knowledgeBaseFieldMapV2` below before testing:~ This is no longer the case as of [cdec104](cdec104). Just changing the above feature flag is now sufficient, just note that if upgrading and the KB was previously setup, you'll need to manually delete the data stream (`DELETE /_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or the management table will be littered with the old ESQL docs instead of being a single aggregate entry. Once configured, the new Knowledge Base Management Settings will become available in Stack Management. The old settings UI is currently still available via the Settings Modal, but will soon be removed and replaced with links to the new interface via the Assistant Settings Context Menu (replacing the existing `cog`). Please see the designs ([Security GenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0) / [Unified AI Assistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0)) for all changes. > [!IMPORTANT] > There are no migrations in place between the legacy and v2 KB mappings, so be sure to start with a clean ES data directory. ### Testing To aid with developing the UI, I took the opportunity to start fleshing out the KB Entries API integration tests. These live in [x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries), and are currently configured to only run on `@ess`, as running `tiny_elser` in serverless and MKI environments can be tricky (more on that later). To start the server and run the tests, from the `x-pack/test/security_solution_api_integration/` directory run `yarn genai_kb_entries:server:ess`, and once started, `yarn genai_kb_entries:runner:ess`. ##### Changes in support of testing In order to setup the API integration tests for use with the Knowledge Base, some functional changes needed to be made to the assistant/config: 1. Since ELSER is a heavy model to run in CI, the ML folks have created `pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()` helper off the `ml` client that we use to get the `modelld` for installing ELSER, ingest pipelines, etc, cannot be overridden ([#193633](#193633)), so we must have some other means of doing that. So to get things working in the test env, I've plumbed through an optional `modelId` override to the POST knowledge base route (`/ internal/ elastic_assistant/ knowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides the aiAssistantService `getELSER()` function [when fetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354) a `kbDataClient` using the request, which appears to be the only way to also trigger a reinitialization of the ingest pipeline (which required the `modelId`), since that usually only occurs on plugin start. If there is a cleaner way to perform this reinitialization, please let me know! 2. Turns out [`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587) can't be run in test env's with `ssl:true`, which is the default security config. You can read more about that issue in [#193477](#193477), but the current workaround is to turn off `ssl` for this specific test configuration, so that's why [`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22) looks a little different. If there's a better way to manage this config, also please let me know! ##### Additional notes We don't currently have a `securityAssistant` API client/service to use in integration tests, so I've just been creating one-off functions using `supertest` for now. I don't have the bandwidth to work this now, but perhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I did need to test multi-user and multi-space scenarios, so I ported over the same [auth helpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth) I saw used in other suites. Would be nice if these were bundled into the client as well ala how the o11y folks have done it [here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34). Perhaps this is also on the list of things for @maximpn to generate from OAS's.... 🙃 ### RBAC In plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all the places I came across that will require an RBAC check/change. This includes some of the API integration tests, which I currently have skipped as they would fail without RBAC. ### Other notable changes * There are now dedicated `legacy` and `v2` helper functions when managing persistence/retrieval of knowledge base entries. This should help with tearing out the old KB later, and better readability now. * I've tried to remove dependency on the `ElasticsearchStore` as much as possible. The store's only use should now be within tools as a retriever [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405), and in post_evaluate [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179). If we adopt the new [`naturalLanguageToESQL`](#192042) tool in `8.16` (or update our existing ESQL tool to use the `kbDataClient` for retrieval), we should be able to get rid of this entirely. * Added a [`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1) file for adding spaces, roles, users, and a sample `slackbot` index for use with [sample `IndexEntries` here](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56). ### // TODO In effort to make incremental progress and facilitate early knowledge share with @patrykkopycinski, I'm capping this PR where it's at, and so here are the remaining items to complete full integration of the new Knowledge Base Management Settings interface: - [ ] Support `Update` action - [ ] Move from `EuiInMemoryTable` - [ ] Finalize `Setup` UI - [ ] Cleanup `Save` loaders - [ ] Plumb through `{{knowledge_history}}` prompt template and include use's `required` entries All this work is behind the aforementioned feature flag and required code change, and this changeset has also been manually upgrade tested to ensure there are no issues that would impact the regularly scheduled serverless releases. This is more of a note to reviewers when testing that full functionality is not present. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
spong
added a commit
to spong/kibana
that referenced
this issue
Sep 25, 2024
…lastic#192665) ## Summary This PR updates the Knowledge Base Management Settings page to use the new `entries` API introduced in elastic#186566. Many thanks to @angorayc for her work on the Assistant Management Settings overhaul, and initial implementation of this new KB Management UI over in elastic#186847. <p align="center"> <img width="600" src="https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa" /> </p> ### Feature Flag & Setup The changes in this PR, as with the other [recent V2 KB enhancements](elastic#186566), are behind the following feature flag: ``` xpack.securitySolution.enableExperimental: - 'assistantKnowledgeBaseByDefault' ``` ~They also require a code change in the `AIAssistantService` to enable the new mapping (since setup happens on plugin start before FF registration), so be sure to update `fieldMap` to `knowledgeBaseFieldMapV2` below before testing:~ This is no longer the case as of [cdec104](elastic@cdec104). Just changing the above feature flag is now sufficient, just note that if upgrading and the KB was previously setup, you'll need to manually delete the data stream (`DELETE /_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or the management table will be littered with the old ESQL docs instead of being a single aggregate entry. Once configured, the new Knowledge Base Management Settings will become available in Stack Management. The old settings UI is currently still available via the Settings Modal, but will soon be removed and replaced with links to the new interface via the Assistant Settings Context Menu (replacing the existing `cog`). Please see the designs ([Security GenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0) / [Unified AI Assistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0)) for all changes. > [!IMPORTANT] > There are no migrations in place between the legacy and v2 KB mappings, so be sure to start with a clean ES data directory. ### Testing To aid with developing the UI, I took the opportunity to start fleshing out the KB Entries API integration tests. These live in [x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries), and are currently configured to only run on `@ess`, as running `tiny_elser` in serverless and MKI environments can be tricky (more on that later). To start the server and run the tests, from the `x-pack/test/security_solution_api_integration/` directory run `yarn genai_kb_entries:server:ess`, and once started, `yarn genai_kb_entries:runner:ess`. ##### Changes in support of testing In order to setup the API integration tests for use with the Knowledge Base, some functional changes needed to be made to the assistant/config: 1. Since ELSER is a heavy model to run in CI, the ML folks have created `pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()` helper off the `ml` client that we use to get the `modelld` for installing ELSER, ingest pipelines, etc, cannot be overridden ([elastic#193633](elastic#193633)), so we must have some other means of doing that. So to get things working in the test env, I've plumbed through an optional `modelId` override to the POST knowledge base route (`/ internal/ elastic_assistant/ knowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides the aiAssistantService `getELSER()` function [when fetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354) a `kbDataClient` using the request, which appears to be the only way to also trigger a reinitialization of the ingest pipeline (which required the `modelId`), since that usually only occurs on plugin start. If there is a cleaner way to perform this reinitialization, please let me know! 2. Turns out [`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587) can't be run in test env's with `ssl:true`, which is the default security config. You can read more about that issue in [elastic#193477](elastic#193477), but the current workaround is to turn off `ssl` for this specific test configuration, so that's why [`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22) looks a little different. If there's a better way to manage this config, also please let me know! ##### Additional notes We don't currently have a `securityAssistant` API client/service to use in integration tests, so I've just been creating one-off functions using `supertest` for now. I don't have the bandwidth to work this now, but perhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I did need to test multi-user and multi-space scenarios, so I ported over the same [auth helpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth) I saw used in other suites. Would be nice if these were bundled into the client as well ala how the o11y folks have done it [here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34). Perhaps this is also on the list of things for @maximpn to generate from OAS's.... 🙃 ### RBAC In plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all the places I came across that will require an RBAC check/change. This includes some of the API integration tests, which I currently have skipped as they would fail without RBAC. ### Other notable changes * There are now dedicated `legacy` and `v2` helper functions when managing persistence/retrieval of knowledge base entries. This should help with tearing out the old KB later, and better readability now. * I've tried to remove dependency on the `ElasticsearchStore` as much as possible. The store's only use should now be within tools as a retriever [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405), and in post_evaluate [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179). If we adopt the new [`naturalLanguageToESQL`](elastic#192042) tool in `8.16` (or update our existing ESQL tool to use the `kbDataClient` for retrieval), we should be able to get rid of this entirely. * Added a [`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1) file for adding spaces, roles, users, and a sample `slackbot` index for use with [sample `IndexEntries` here](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56). ### // TODO In effort to make incremental progress and facilitate early knowledge share with @patrykkopycinski, I'm capping this PR where it's at, and so here are the remaining items to complete full integration of the new Knowledge Base Management Settings interface: - [ ] Support `Update` action - [ ] Move from `EuiInMemoryTable` - [ ] Finalize `Setup` UI - [ ] Cleanup `Save` loaders - [ ] Plumb through `{{knowledge_history}}` prompt template and include use's `required` entries All this work is behind the aforementioned feature flag and required code change, and this changeset has also been manually upgrade tested to ensure there are no issues that would impact the regularly scheduled serverless releases. This is more of a note to reviewers when testing that full functionality is not present. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> (cherry picked from commit 63730ea)
spong
added a commit
to elastic/kibana
that referenced
this issue
Sep 26, 2024
…s UI (#192665) (#194074) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Assistant] Adds new Knowledge Base Management Settings UI (#192665)](#192665) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Garrett Spong","email":"spong@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-09-25T20:38:18Z","message":"[Security Assistant] Adds new Knowledge Base Management Settings UI (#192665)\n\n## Summary\r\n\r\nThis PR updates the Knowledge Base Management Settings page to use the\r\nnew `entries` API introduced in\r\nhttps://github.com//pull/186566. Many thanks to @angorayc\r\nfor her work on the Assistant Management Settings overhaul, and initial\r\nimplementation of this new KB Management UI over in\r\nhttps://github.com//pull/186847.\r\n\r\n<p align=\"center\">\r\n<img width=\"600\"\r\nsrc=\"https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa\"\r\n/>\r\n</p> \r\n\r\n\r\n\r\n### Feature Flag & Setup\r\nThe changes in this PR, as with the other [recent V2 KB\r\nenhancements](#186566), are behind\r\nthe following feature flag:\r\n```\r\nxpack.securitySolution.enableExperimental:\r\n - 'assistantKnowledgeBaseByDefault'\r\n```\r\n\r\n~They also require a code change in the `AIAssistantService` to enable\r\nthe new mapping (since setup happens on plugin start before FF\r\nregistration), so be sure to update `fieldMap` to\r\n`knowledgeBaseFieldMapV2` below before testing:~\r\n\r\nThis is no longer the case as of\r\n[cdec104](https://github.com/elastic/kibana/pull/192665/commits/cdec10402f2e9b889598693f9f415c98ccd9855c).\r\nJust changing the above feature flag is now sufficient, just note that\r\nif upgrading and the KB was previously setup, you'll need to manually\r\ndelete the data stream (`DELETE\r\n/_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or\r\nthe management table will be littered with the old ESQL docs instead of\r\nbeing a single aggregate entry.\r\n\r\nOnce configured, the new Knowledge Base Management Settings will become\r\navailable in Stack Management. The old settings UI is currently still\r\navailable via the Settings Modal, but will soon be removed and replaced\r\nwith links to the new interface via the Assistant Settings Context Menu\r\n(replacing the existing `cog`). Please see the designs ([Security\r\nGenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0)\r\n/ [Unified AI\r\nAssistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0))\r\nfor all changes.\r\n\r\n> [!IMPORTANT]\r\n> There are no migrations in place between the legacy and v2 KB\r\nmappings, so be sure to start with a clean ES data directory.\r\n\r\n### Testing\r\n\r\nTo aid with developing the UI, I took the opportunity to start fleshing\r\nout the KB Entries API integration tests. These live in\r\n[x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries),\r\nand are currently configured to only run on `@ess`, as running\r\n`tiny_elser` in serverless and MKI environments can be tricky (more on\r\nthat later).\r\n\r\nTo start the server and run the tests, from the\r\n`x-pack/test/security_solution_api_integration/` directory run `yarn\r\ngenai_kb_entries:server:ess`, and once started, `yarn\r\ngenai_kb_entries:runner:ess`.\r\n\r\n##### Changes in support of testing\r\n\r\nIn order to setup the API integration tests for use with the Knowledge\r\nBase, some functional changes needed to be made to the assistant/config:\r\n\r\n1. Since ELSER is a heavy model to run in CI, the ML folks have created\r\n`pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()`\r\nhelper off the `ml` client that we use to get the `modelld` for\r\ninstalling ELSER, ingest pipelines, etc, cannot be overridden\r\n([#193633](#193633)), so we must\r\nhave some other means of doing that. So to get things working in the\r\ntest env, I've plumbed through an optional `modelId` override to the\r\nPOST knowledge base route (`/ internal/ elastic_assistant/\r\nknowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides\r\nthe aiAssistantService `getELSER()` function [when\r\nfetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354)\r\na `kbDataClient` using the request, which appears to be the only way to\r\nalso trigger a reinitialization of the ingest pipeline (which required\r\nthe `modelId`), since that usually only occurs on plugin start. If there\r\nis a cleaner way to perform this reinitialization, please let me know!\r\n\r\n2. Turns out\r\n[`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587)\r\ncan't be run in test env's with `ssl:true`, which is the default\r\nsecurity config. You can read more about that issue in\r\n[#193477](#193477), but the\r\ncurrent workaround is to turn off `ssl` for this specific test\r\nconfiguration, so that's why\r\n[`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22)\r\nlooks a little different. If there's a better way to manage this config,\r\nalso please let me know!\r\n\r\n##### Additional notes\r\n\r\nWe don't currently have a `securityAssistant` API client/service to use\r\nin integration tests, so I've just been creating one-off functions using\r\n`supertest` for now. I don't have the bandwidth to work this now, but\r\nperhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I\r\ndid need to test multi-user and multi-space scenarios, so I ported over\r\nthe same [auth\r\nhelpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth)\r\nI saw used in other suites. Would be nice if these were bundled into the\r\nclient as well ala how the o11y folks have done it\r\n[here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34).\r\nPerhaps this is also on the list of things for @maximpn to generate from\r\nOAS's.... 🙃\r\n\r\n### RBAC\r\nIn plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all\r\nthe places I came across that will require an RBAC check/change. This\r\nincludes some of the API integration tests, which I currently have\r\nskipped as they would fail without RBAC.\r\n\r\n### Other notable changes\r\n\r\n* There are now dedicated `legacy` and `v2` helper functions when\r\nmanaging persistence/retrieval of knowledge base entries. This should\r\nhelp with tearing out the old KB later, and better readability now.\r\n* I've tried to remove dependency on the `ElasticsearchStore` as much as\r\npossible. The store's only use should now be within tools as a retriever\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405),\r\nand in post_evaluate\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179).\r\nIf we adopt the new\r\n[`naturalLanguageToESQL`](https://github.com/elastic/kibana/pull/192042)\r\ntool in `8.16` (or update our existing ESQL tool to use the\r\n`kbDataClient` for retrieval), we should be able to get rid of this\r\nentirely.\r\n* Added a\r\n[`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1)\r\nfile for adding spaces, roles, users, and a sample `slackbot` index for\r\nuse with [sample `IndexEntries`\r\nhere](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56).\r\n\r\n### // TODO\r\nIn effort to make incremental progress and facilitate early knowledge\r\nshare with @patrykkopycinski, I'm capping this PR where it's at, and so\r\nhere are the remaining items to complete full integration of the new\r\nKnowledge Base Management Settings interface:\r\n\r\n- [ ] Support `Update` action\r\n- [ ] Move from `EuiInMemoryTable` \r\n- [ ] Finalize `Setup` UI\r\n- [ ] Cleanup `Save` loaders\r\n- [ ] Plumb through `{{knowledge_history}}` prompt template and include\r\nuse's `required` entries\r\n\r\nAll this work is behind the aforementioned feature flag and required\r\ncode change, and this changeset has also been manually upgrade tested to\r\nensure there are no issues that would impact the regularly scheduled\r\nserverless releases. This is more of a note to reviewers when testing\r\nthat full functionality is not present.\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"63730ea0c9d9b036a05cb919b25b6d19c2ea8f03","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Feature:Security Assistant","Team:Security Generative AI","v8.16.0","backport:version"],"number":192665,"url":"https://github.com/elastic/kibana/pull/192665","mergeCommit":{"message":"[Security Assistant] Adds new Knowledge Base Management Settings UI (#192665)\n\n## Summary\r\n\r\nThis PR updates the Knowledge Base Management Settings page to use the\r\nnew `entries` API introduced in\r\nhttps://github.com//pull/186566. Many thanks to @angorayc\r\nfor her work on the Assistant Management Settings overhaul, and initial\r\nimplementation of this new KB Management UI over in\r\nhttps://github.com//pull/186847.\r\n\r\n<p align=\"center\">\r\n<img width=\"600\"\r\nsrc=\"https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa\"\r\n/>\r\n</p> \r\n\r\n\r\n\r\n### Feature Flag & Setup\r\nThe changes in this PR, as with the other [recent V2 KB\r\nenhancements](#186566), are behind\r\nthe following feature flag:\r\n```\r\nxpack.securitySolution.enableExperimental:\r\n - 'assistantKnowledgeBaseByDefault'\r\n```\r\n\r\n~They also require a code change in the `AIAssistantService` to enable\r\nthe new mapping (since setup happens on plugin start before FF\r\nregistration), so be sure to update `fieldMap` to\r\n`knowledgeBaseFieldMapV2` below before testing:~\r\n\r\nThis is no longer the case as of\r\n[cdec104](https://github.com/elastic/kibana/pull/192665/commits/cdec10402f2e9b889598693f9f415c98ccd9855c).\r\nJust changing the above feature flag is now sufficient, just note that\r\nif upgrading and the KB was previously setup, you'll need to manually\r\ndelete the data stream (`DELETE\r\n/_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or\r\nthe management table will be littered with the old ESQL docs instead of\r\nbeing a single aggregate entry.\r\n\r\nOnce configured, the new Knowledge Base Management Settings will become\r\navailable in Stack Management. The old settings UI is currently still\r\navailable via the Settings Modal, but will soon be removed and replaced\r\nwith links to the new interface via the Assistant Settings Context Menu\r\n(replacing the existing `cog`). Please see the designs ([Security\r\nGenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0)\r\n/ [Unified AI\r\nAssistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0))\r\nfor all changes.\r\n\r\n> [!IMPORTANT]\r\n> There are no migrations in place between the legacy and v2 KB\r\nmappings, so be sure to start with a clean ES data directory.\r\n\r\n### Testing\r\n\r\nTo aid with developing the UI, I took the opportunity to start fleshing\r\nout the KB Entries API integration tests. These live in\r\n[x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries),\r\nand are currently configured to only run on `@ess`, as running\r\n`tiny_elser` in serverless and MKI environments can be tricky (more on\r\nthat later).\r\n\r\nTo start the server and run the tests, from the\r\n`x-pack/test/security_solution_api_integration/` directory run `yarn\r\ngenai_kb_entries:server:ess`, and once started, `yarn\r\ngenai_kb_entries:runner:ess`.\r\n\r\n##### Changes in support of testing\r\n\r\nIn order to setup the API integration tests for use with the Knowledge\r\nBase, some functional changes needed to be made to the assistant/config:\r\n\r\n1. Since ELSER is a heavy model to run in CI, the ML folks have created\r\n`pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()`\r\nhelper off the `ml` client that we use to get the `modelld` for\r\ninstalling ELSER, ingest pipelines, etc, cannot be overridden\r\n([#193633](#193633)), so we must\r\nhave some other means of doing that. So to get things working in the\r\ntest env, I've plumbed through an optional `modelId` override to the\r\nPOST knowledge base route (`/ internal/ elastic_assistant/\r\nknowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides\r\nthe aiAssistantService `getELSER()` function [when\r\nfetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354)\r\na `kbDataClient` using the request, which appears to be the only way to\r\nalso trigger a reinitialization of the ingest pipeline (which required\r\nthe `modelId`), since that usually only occurs on plugin start. If there\r\nis a cleaner way to perform this reinitialization, please let me know!\r\n\r\n2. Turns out\r\n[`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587)\r\ncan't be run in test env's with `ssl:true`, which is the default\r\nsecurity config. You can read more about that issue in\r\n[#193477](#193477), but the\r\ncurrent workaround is to turn off `ssl` for this specific test\r\nconfiguration, so that's why\r\n[`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22)\r\nlooks a little different. If there's a better way to manage this config,\r\nalso please let me know!\r\n\r\n##### Additional notes\r\n\r\nWe don't currently have a `securityAssistant` API client/service to use\r\nin integration tests, so I've just been creating one-off functions using\r\n`supertest` for now. I don't have the bandwidth to work this now, but\r\nperhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I\r\ndid need to test multi-user and multi-space scenarios, so I ported over\r\nthe same [auth\r\nhelpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth)\r\nI saw used in other suites. Would be nice if these were bundled into the\r\nclient as well ala how the o11y folks have done it\r\n[here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34).\r\nPerhaps this is also on the list of things for @maximpn to generate from\r\nOAS's.... 🙃\r\n\r\n### RBAC\r\nIn plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all\r\nthe places I came across that will require an RBAC check/change. This\r\nincludes some of the API integration tests, which I currently have\r\nskipped as they would fail without RBAC.\r\n\r\n### Other notable changes\r\n\r\n* There are now dedicated `legacy` and `v2` helper functions when\r\nmanaging persistence/retrieval of knowledge base entries. This should\r\nhelp with tearing out the old KB later, and better readability now.\r\n* I've tried to remove dependency on the `ElasticsearchStore` as much as\r\npossible. The store's only use should now be within tools as a retriever\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405),\r\nand in post_evaluate\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179).\r\nIf we adopt the new\r\n[`naturalLanguageToESQL`](https://github.com/elastic/kibana/pull/192042)\r\ntool in `8.16` (or update our existing ESQL tool to use the\r\n`kbDataClient` for retrieval), we should be able to get rid of this\r\nentirely.\r\n* Added a\r\n[`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1)\r\nfile for adding spaces, roles, users, and a sample `slackbot` index for\r\nuse with [sample `IndexEntries`\r\nhere](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56).\r\n\r\n### // TODO\r\nIn effort to make incremental progress and facilitate early knowledge\r\nshare with @patrykkopycinski, I'm capping this PR where it's at, and so\r\nhere are the remaining items to complete full integration of the new\r\nKnowledge Base Management Settings interface:\r\n\r\n- [ ] Support `Update` action\r\n- [ ] Move from `EuiInMemoryTable` \r\n- [ ] Finalize `Setup` UI\r\n- [ ] Cleanup `Save` loaders\r\n- [ ] Plumb through `{{knowledge_history}}` prompt template and include\r\nuse's `required` entries\r\n\r\nAll this work is behind the aforementioned feature flag and required\r\ncode change, and this changeset has also been manually upgrade tested to\r\nensure there are no issues that would impact the regularly scheduled\r\nserverless releases. This is more of a note to reviewers when testing\r\nthat full functionality is not present.\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"63730ea0c9d9b036a05cb919b25b6d19c2ea8f03"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192665","number":192665,"mergeCommit":{"message":"[Security Assistant] Adds new Knowledge Base Management Settings UI (#192665)\n\n## Summary\r\n\r\nThis PR updates the Knowledge Base Management Settings page to use the\r\nnew `entries` API introduced in\r\nhttps://github.com//pull/186566. Many thanks to @angorayc\r\nfor her work on the Assistant Management Settings overhaul, and initial\r\nimplementation of this new KB Management UI over in\r\nhttps://github.com//pull/186847.\r\n\r\n<p align=\"center\">\r\n<img width=\"600\"\r\nsrc=\"https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa\"\r\n/>\r\n</p> \r\n\r\n\r\n\r\n### Feature Flag & Setup\r\nThe changes in this PR, as with the other [recent V2 KB\r\nenhancements](#186566), are behind\r\nthe following feature flag:\r\n```\r\nxpack.securitySolution.enableExperimental:\r\n - 'assistantKnowledgeBaseByDefault'\r\n```\r\n\r\n~They also require a code change in the `AIAssistantService` to enable\r\nthe new mapping (since setup happens on plugin start before FF\r\nregistration), so be sure to update `fieldMap` to\r\n`knowledgeBaseFieldMapV2` below before testing:~\r\n\r\nThis is no longer the case as of\r\n[cdec104](https://github.com/elastic/kibana/pull/192665/commits/cdec10402f2e9b889598693f9f415c98ccd9855c).\r\nJust changing the above feature flag is now sufficient, just note that\r\nif upgrading and the KB was previously setup, you'll need to manually\r\ndelete the data stream (`DELETE\r\n/_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or\r\nthe management table will be littered with the old ESQL docs instead of\r\nbeing a single aggregate entry.\r\n\r\nOnce configured, the new Knowledge Base Management Settings will become\r\navailable in Stack Management. The old settings UI is currently still\r\navailable via the Settings Modal, but will soon be removed and replaced\r\nwith links to the new interface via the Assistant Settings Context Menu\r\n(replacing the existing `cog`). Please see the designs ([Security\r\nGenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0)\r\n/ [Unified AI\r\nAssistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0))\r\nfor all changes.\r\n\r\n> [!IMPORTANT]\r\n> There are no migrations in place between the legacy and v2 KB\r\nmappings, so be sure to start with a clean ES data directory.\r\n\r\n### Testing\r\n\r\nTo aid with developing the UI, I took the opportunity to start fleshing\r\nout the KB Entries API integration tests. These live in\r\n[x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries),\r\nand are currently configured to only run on `@ess`, as running\r\n`tiny_elser` in serverless and MKI environments can be tricky (more on\r\nthat later).\r\n\r\nTo start the server and run the tests, from the\r\n`x-pack/test/security_solution_api_integration/` directory run `yarn\r\ngenai_kb_entries:server:ess`, and once started, `yarn\r\ngenai_kb_entries:runner:ess`.\r\n\r\n##### Changes in support of testing\r\n\r\nIn order to setup the API integration tests for use with the Knowledge\r\nBase, some functional changes needed to be made to the assistant/config:\r\n\r\n1. Since ELSER is a heavy model to run in CI, the ML folks have created\r\n`pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()`\r\nhelper off the `ml` client that we use to get the `modelld` for\r\ninstalling ELSER, ingest pipelines, etc, cannot be overridden\r\n([#193633](#193633)), so we must\r\nhave some other means of doing that. So to get things working in the\r\ntest env, I've plumbed through an optional `modelId` override to the\r\nPOST knowledge base route (`/ internal/ elastic_assistant/\r\nknowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides\r\nthe aiAssistantService `getELSER()` function [when\r\nfetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354)\r\na `kbDataClient` using the request, which appears to be the only way to\r\nalso trigger a reinitialization of the ingest pipeline (which required\r\nthe `modelId`), since that usually only occurs on plugin start. If there\r\nis a cleaner way to perform this reinitialization, please let me know!\r\n\r\n2. Turns out\r\n[`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587)\r\ncan't be run in test env's with `ssl:true`, which is the default\r\nsecurity config. You can read more about that issue in\r\n[#193477](#193477), but the\r\ncurrent workaround is to turn off `ssl` for this specific test\r\nconfiguration, so that's why\r\n[`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22)\r\nlooks a little different. If there's a better way to manage this config,\r\nalso please let me know!\r\n\r\n##### Additional notes\r\n\r\nWe don't currently have a `securityAssistant` API client/service to use\r\nin integration tests, so I've just been creating one-off functions using\r\n`supertest` for now. I don't have the bandwidth to work this now, but\r\nperhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I\r\ndid need to test multi-user and multi-space scenarios, so I ported over\r\nthe same [auth\r\nhelpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth)\r\nI saw used in other suites. Would be nice if these were bundled into the\r\nclient as well ala how the o11y folks have done it\r\n[here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34).\r\nPerhaps this is also on the list of things for @maximpn to generate from\r\nOAS's.... 🙃\r\n\r\n### RBAC\r\nIn plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all\r\nthe places I came across that will require an RBAC check/change. This\r\nincludes some of the API integration tests, which I currently have\r\nskipped as they would fail without RBAC.\r\n\r\n### Other notable changes\r\n\r\n* There are now dedicated `legacy` and `v2` helper functions when\r\nmanaging persistence/retrieval of knowledge base entries. This should\r\nhelp with tearing out the old KB later, and better readability now.\r\n* I've tried to remove dependency on the `ElasticsearchStore` as much as\r\npossible. The store's only use should now be within tools as a retriever\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405),\r\nand in post_evaluate\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179).\r\nIf we adopt the new\r\n[`naturalLanguageToESQL`](https://github.com/elastic/kibana/pull/192042)\r\ntool in `8.16` (or update our existing ESQL tool to use the\r\n`kbDataClient` for retrieval), we should be able to get rid of this\r\nentirely.\r\n* Added a\r\n[`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1)\r\nfile for adding spaces, roles, users, and a sample `slackbot` index for\r\nuse with [sample `IndexEntries`\r\nhere](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56).\r\n\r\n### // TODO\r\nIn effort to make incremental progress and facilitate early knowledge\r\nshare with @patrykkopycinski, I'm capping this PR where it's at, and so\r\nhere are the remaining items to complete full integration of the new\r\nKnowledge Base Management Settings interface:\r\n\r\n- [ ] Support `Update` action\r\n- [ ] Move from `EuiInMemoryTable` \r\n- [ ] Finalize `Setup` UI\r\n- [ ] Cleanup `Save` loaders\r\n- [ ] Plumb through `{{knowledge_history}}` prompt template and include\r\nuse's `required` entries\r\n\r\nAll this work is behind the aforementioned feature flag and required\r\ncode change, and this changeset has also been manually upgrade tested to\r\nensure there are no issues that would impact the regularly scheduled\r\nserverless releases. This is more of a note to reviewers when testing\r\nthat full functionality is not present.\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"63730ea0c9d9b036a05cb919b25b6d19c2ea8f03"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
benironside
changed the title
13 tasks
angorayc
pushed a commit
to angorayc/kibana
that referenced
this issue
Oct 1, 2024
…lastic#192665) ## Summary This PR updates the Knowledge Base Management Settings page to use the new `entries` API introduced in elastic#186566. Many thanks to @angorayc for her work on the Assistant Management Settings overhaul, and initial implementation of this new KB Management UI over in elastic#186847. <p align="center"> <img width="600" src="https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa" /> </p> ### Feature Flag & Setup The changes in this PR, as with the other [recent V2 KB enhancements](elastic#186566), are behind the following feature flag: ``` xpack.securitySolution.enableExperimental: - 'assistantKnowledgeBaseByDefault' ``` ~They also require a code change in the `AIAssistantService` to enable the new mapping (since setup happens on plugin start before FF registration), so be sure to update `fieldMap` to `knowledgeBaseFieldMapV2` below before testing:~ This is no longer the case as of [cdec104](elastic@cdec104). Just changing the above feature flag is now sufficient, just note that if upgrading and the KB was previously setup, you'll need to manually delete the data stream (`DELETE /_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or the management table will be littered with the old ESQL docs instead of being a single aggregate entry. Once configured, the new Knowledge Base Management Settings will become available in Stack Management. The old settings UI is currently still available via the Settings Modal, but will soon be removed and replaced with links to the new interface via the Assistant Settings Context Menu (replacing the existing `cog`). Please see the designs ([Security GenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0) / [Unified AI Assistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0)) for all changes. > [!IMPORTANT] > There are no migrations in place between the legacy and v2 KB mappings, so be sure to start with a clean ES data directory. ### Testing To aid with developing the UI, I took the opportunity to start fleshing out the KB Entries API integration tests. These live in [x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries), and are currently configured to only run on `@ess`, as running `tiny_elser` in serverless and MKI environments can be tricky (more on that later). To start the server and run the tests, from the `x-pack/test/security_solution_api_integration/` directory run `yarn genai_kb_entries:server:ess`, and once started, `yarn genai_kb_entries:runner:ess`. ##### Changes in support of testing In order to setup the API integration tests for use with the Knowledge Base, some functional changes needed to be made to the assistant/config: 1. Since ELSER is a heavy model to run in CI, the ML folks have created `pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()` helper off the `ml` client that we use to get the `modelld` for installing ELSER, ingest pipelines, etc, cannot be overridden ([elastic#193633](elastic#193633)), so we must have some other means of doing that. So to get things working in the test env, I've plumbed through an optional `modelId` override to the POST knowledge base route (`/ internal/ elastic_assistant/ knowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides the aiAssistantService `getELSER()` function [when fetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354) a `kbDataClient` using the request, which appears to be the only way to also trigger a reinitialization of the ingest pipeline (which required the `modelId`), since that usually only occurs on plugin start. If there is a cleaner way to perform this reinitialization, please let me know! 2. Turns out [`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587) can't be run in test env's with `ssl:true`, which is the default security config. You can read more about that issue in [elastic#193477](elastic#193477), but the current workaround is to turn off `ssl` for this specific test configuration, so that's why [`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22) looks a little different. If there's a better way to manage this config, also please let me know! ##### Additional notes We don't currently have a `securityAssistant` API client/service to use in integration tests, so I've just been creating one-off functions using `supertest` for now. I don't have the bandwidth to work this now, but perhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I did need to test multi-user and multi-space scenarios, so I ported over the same [auth helpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth) I saw used in other suites. Would be nice if these were bundled into the client as well ala how the o11y folks have done it [here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34). Perhaps this is also on the list of things for @maximpn to generate from OAS's.... 🙃 ### RBAC In plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all the places I came across that will require an RBAC check/change. This includes some of the API integration tests, which I currently have skipped as they would fail without RBAC. ### Other notable changes * There are now dedicated `legacy` and `v2` helper functions when managing persistence/retrieval of knowledge base entries. This should help with tearing out the old KB later, and better readability now. * I've tried to remove dependency on the `ElasticsearchStore` as much as possible. The store's only use should now be within tools as a retriever [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405), and in post_evaluate [here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179). If we adopt the new [`naturalLanguageToESQL`](elastic#192042) tool in `8.16` (or update our existing ESQL tool to use the `kbDataClient` for retrieval), we should be able to get rid of this entirely. * Added a [`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1) file for adding spaces, roles, users, and a sample `slackbot` index for use with [sample `IndexEntries` here](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56). ### // TODO In effort to make incremental progress and facilitate early knowledge share with @patrykkopycinski, I'm capping this PR where it's at, and so here are the remaining items to complete full integration of the new Knowledge Base Management Settings interface: - [ ] Support `Update` action - [ ] Move from `EuiInMemoryTable` - [ ] Finalize `Setup` UI - [ ] Cleanup `Save` loaders - [ ] Plumb through `{{knowledge_history}}` prompt template and include use's `required` entries All this work is behind the aforementioned feature flag and required code change, and this changeset has also been manually upgrade tested to ensure there are no issues that would impact the regularly scheduled serverless releases. This is more of a note to reviewers when testing that full functionality is not present. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
neptunian
pushed a commit
to neptunian/kibana
that referenced
this issue
Oct 1, 2024
…s UI (elastic#192665) (elastic#194074) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Assistant] Adds new Knowledge Base Management Settings UI (elastic#192665)](elastic#192665) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Garrett Spong","email":"spong@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-09-25T20:38:18Z","message":"[Security Assistant] Adds new Knowledge Base Management Settings UI (elastic#192665)\n\n## Summary\r\n\r\nThis PR updates the Knowledge Base Management Settings page to use the\r\nnew `entries` API introduced in\r\nhttps://github.com/elastic/pull/186566. Many thanks to @angorayc\r\nfor her work on the Assistant Management Settings overhaul, and initial\r\nimplementation of this new KB Management UI over in\r\nhttps://github.com/elastic/pull/186847.\r\n\r\n<p align=\"center\">\r\n<img width=\"600\"\r\nsrc=\"https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa\"\r\n/>\r\n</p> \r\n\r\n\r\n\r\n### Feature Flag & Setup\r\nThe changes in this PR, as with the other [recent V2 KB\r\nenhancements](elastic#186566), are behind\r\nthe following feature flag:\r\n```\r\nxpack.securitySolution.enableExperimental:\r\n - 'assistantKnowledgeBaseByDefault'\r\n```\r\n\r\n~They also require a code change in the `AIAssistantService` to enable\r\nthe new mapping (since setup happens on plugin start before FF\r\nregistration), so be sure to update `fieldMap` to\r\n`knowledgeBaseFieldMapV2` below before testing:~\r\n\r\nThis is no longer the case as of\r\n[cdec104](https://github.com/elastic/kibana/pull/192665/commits/cdec10402f2e9b889598693f9f415c98ccd9855c).\r\nJust changing the above feature flag is now sufficient, just note that\r\nif upgrading and the KB was previously setup, you'll need to manually\r\ndelete the data stream (`DELETE\r\n/_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or\r\nthe management table will be littered with the old ESQL docs instead of\r\nbeing a single aggregate entry.\r\n\r\nOnce configured, the new Knowledge Base Management Settings will become\r\navailable in Stack Management. The old settings UI is currently still\r\navailable via the Settings Modal, but will soon be removed and replaced\r\nwith links to the new interface via the Assistant Settings Context Menu\r\n(replacing the existing `cog`). Please see the designs ([Security\r\nGenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0)\r\n/ [Unified AI\r\nAssistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0))\r\nfor all changes.\r\n\r\n> [!IMPORTANT]\r\n> There are no migrations in place between the legacy and v2 KB\r\nmappings, so be sure to start with a clean ES data directory.\r\n\r\n### Testing\r\n\r\nTo aid with developing the UI, I took the opportunity to start fleshing\r\nout the KB Entries API integration tests. These live in\r\n[x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries),\r\nand are currently configured to only run on `@ess`, as running\r\n`tiny_elser` in serverless and MKI environments can be tricky (more on\r\nthat later).\r\n\r\nTo start the server and run the tests, from the\r\n`x-pack/test/security_solution_api_integration/` directory run `yarn\r\ngenai_kb_entries:server:ess`, and once started, `yarn\r\ngenai_kb_entries:runner:ess`.\r\n\r\n##### Changes in support of testing\r\n\r\nIn order to setup the API integration tests for use with the Knowledge\r\nBase, some functional changes needed to be made to the assistant/config:\r\n\r\n1. Since ELSER is a heavy model to run in CI, the ML folks have created\r\n`pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()`\r\nhelper off the `ml` client that we use to get the `modelld` for\r\ninstalling ELSER, ingest pipelines, etc, cannot be overridden\r\n([elastic#193633](elastic#193633)), so we must\r\nhave some other means of doing that. So to get things working in the\r\ntest env, I've plumbed through an optional `modelId` override to the\r\nPOST knowledge base route (`/ internal/ elastic_assistant/\r\nknowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides\r\nthe aiAssistantService `getELSER()` function [when\r\nfetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354)\r\na `kbDataClient` using the request, which appears to be the only way to\r\nalso trigger a reinitialization of the ingest pipeline (which required\r\nthe `modelId`), since that usually only occurs on plugin start. If there\r\nis a cleaner way to perform this reinitialization, please let me know!\r\n\r\n2. Turns out\r\n[`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587)\r\ncan't be run in test env's with `ssl:true`, which is the default\r\nsecurity config. You can read more about that issue in\r\n[elastic#193477](elastic#193477), but the\r\ncurrent workaround is to turn off `ssl` for this specific test\r\nconfiguration, so that's why\r\n[`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22)\r\nlooks a little different. If there's a better way to manage this config,\r\nalso please let me know!\r\n\r\n##### Additional notes\r\n\r\nWe don't currently have a `securityAssistant` API client/service to use\r\nin integration tests, so I've just been creating one-off functions using\r\n`supertest` for now. I don't have the bandwidth to work this now, but\r\nperhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I\r\ndid need to test multi-user and multi-space scenarios, so I ported over\r\nthe same [auth\r\nhelpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth)\r\nI saw used in other suites. Would be nice if these were bundled into the\r\nclient as well ala how the o11y folks have done it\r\n[here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34).\r\nPerhaps this is also on the list of things for @maximpn to generate from\r\nOAS's.... 🙃\r\n\r\n### RBAC\r\nIn plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all\r\nthe places I came across that will require an RBAC check/change. This\r\nincludes some of the API integration tests, which I currently have\r\nskipped as they would fail without RBAC.\r\n\r\n### Other notable changes\r\n\r\n* There are now dedicated `legacy` and `v2` helper functions when\r\nmanaging persistence/retrieval of knowledge base entries. This should\r\nhelp with tearing out the old KB later, and better readability now.\r\n* I've tried to remove dependency on the `ElasticsearchStore` as much as\r\npossible. The store's only use should now be within tools as a retriever\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405),\r\nand in post_evaluate\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179).\r\nIf we adopt the new\r\n[`naturalLanguageToESQL`](https://github.com/elastic/kibana/pull/192042)\r\ntool in `8.16` (or update our existing ESQL tool to use the\r\n`kbDataClient` for retrieval), we should be able to get rid of this\r\nentirely.\r\n* Added a\r\n[`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1)\r\nfile for adding spaces, roles, users, and a sample `slackbot` index for\r\nuse with [sample `IndexEntries`\r\nhere](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56).\r\n\r\n### // TODO\r\nIn effort to make incremental progress and facilitate early knowledge\r\nshare with @patrykkopycinski, I'm capping this PR where it's at, and so\r\nhere are the remaining items to complete full integration of the new\r\nKnowledge Base Management Settings interface:\r\n\r\n- [ ] Support `Update` action\r\n- [ ] Move from `EuiInMemoryTable` \r\n- [ ] Finalize `Setup` UI\r\n- [ ] Cleanup `Save` loaders\r\n- [ ] Plumb through `{{knowledge_history}}` prompt template and include\r\nuse's `required` entries\r\n\r\nAll this work is behind the aforementioned feature flag and required\r\ncode change, and this changeset has also been manually upgrade tested to\r\nensure there are no issues that would impact the regularly scheduled\r\nserverless releases. This is more of a note to reviewers when testing\r\nthat full functionality is not present.\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"63730ea0c9d9b036a05cb919b25b6d19c2ea8f03","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Feature:Security Assistant","Team:Security Generative AI","v8.16.0","backport:version"],"number":192665,"url":"https://github.com/elastic/kibana/pull/192665","mergeCommit":{"message":"[Security Assistant] Adds new Knowledge Base Management Settings UI (elastic#192665)\n\n## Summary\r\n\r\nThis PR updates the Knowledge Base Management Settings page to use the\r\nnew `entries` API introduced in\r\nhttps://github.com/elastic/pull/186566. Many thanks to @angorayc\r\nfor her work on the Assistant Management Settings overhaul, and initial\r\nimplementation of this new KB Management UI over in\r\nhttps://github.com/elastic/pull/186847.\r\n\r\n<p align=\"center\">\r\n<img width=\"600\"\r\nsrc=\"https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa\"\r\n/>\r\n</p> \r\n\r\n\r\n\r\n### Feature Flag & Setup\r\nThe changes in this PR, as with the other [recent V2 KB\r\nenhancements](elastic#186566), are behind\r\nthe following feature flag:\r\n```\r\nxpack.securitySolution.enableExperimental:\r\n - 'assistantKnowledgeBaseByDefault'\r\n```\r\n\r\n~They also require a code change in the `AIAssistantService` to enable\r\nthe new mapping (since setup happens on plugin start before FF\r\nregistration), so be sure to update `fieldMap` to\r\n`knowledgeBaseFieldMapV2` below before testing:~\r\n\r\nThis is no longer the case as of\r\n[cdec104](https://github.com/elastic/kibana/pull/192665/commits/cdec10402f2e9b889598693f9f415c98ccd9855c).\r\nJust changing the above feature flag is now sufficient, just note that\r\nif upgrading and the KB was previously setup, you'll need to manually\r\ndelete the data stream (`DELETE\r\n/_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or\r\nthe management table will be littered with the old ESQL docs instead of\r\nbeing a single aggregate entry.\r\n\r\nOnce configured, the new Knowledge Base Management Settings will become\r\navailable in Stack Management. The old settings UI is currently still\r\navailable via the Settings Modal, but will soon be removed and replaced\r\nwith links to the new interface via the Assistant Settings Context Menu\r\n(replacing the existing `cog`). Please see the designs ([Security\r\nGenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0)\r\n/ [Unified AI\r\nAssistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0))\r\nfor all changes.\r\n\r\n> [!IMPORTANT]\r\n> There are no migrations in place between the legacy and v2 KB\r\nmappings, so be sure to start with a clean ES data directory.\r\n\r\n### Testing\r\n\r\nTo aid with developing the UI, I took the opportunity to start fleshing\r\nout the KB Entries API integration tests. These live in\r\n[x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries),\r\nand are currently configured to only run on `@ess`, as running\r\n`tiny_elser` in serverless and MKI environments can be tricky (more on\r\nthat later).\r\n\r\nTo start the server and run the tests, from the\r\n`x-pack/test/security_solution_api_integration/` directory run `yarn\r\ngenai_kb_entries:server:ess`, and once started, `yarn\r\ngenai_kb_entries:runner:ess`.\r\n\r\n##### Changes in support of testing\r\n\r\nIn order to setup the API integration tests for use with the Knowledge\r\nBase, some functional changes needed to be made to the assistant/config:\r\n\r\n1. Since ELSER is a heavy model to run in CI, the ML folks have created\r\n`pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()`\r\nhelper off the `ml` client that we use to get the `modelld` for\r\ninstalling ELSER, ingest pipelines, etc, cannot be overridden\r\n([elastic#193633](elastic#193633)), so we must\r\nhave some other means of doing that. So to get things working in the\r\ntest env, I've plumbed through an optional `modelId` override to the\r\nPOST knowledge base route (`/ internal/ elastic_assistant/\r\nknowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides\r\nthe aiAssistantService `getELSER()` function [when\r\nfetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354)\r\na `kbDataClient` using the request, which appears to be the only way to\r\nalso trigger a reinitialization of the ingest pipeline (which required\r\nthe `modelId`), since that usually only occurs on plugin start. If there\r\nis a cleaner way to perform this reinitialization, please let me know!\r\n\r\n2. Turns out\r\n[`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587)\r\ncan't be run in test env's with `ssl:true`, which is the default\r\nsecurity config. You can read more about that issue in\r\n[elastic#193477](elastic#193477), but the\r\ncurrent workaround is to turn off `ssl` for this specific test\r\nconfiguration, so that's why\r\n[`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22)\r\nlooks a little different. If there's a better way to manage this config,\r\nalso please let me know!\r\n\r\n##### Additional notes\r\n\r\nWe don't currently have a `securityAssistant` API client/service to use\r\nin integration tests, so I've just been creating one-off functions using\r\n`supertest` for now. I don't have the bandwidth to work this now, but\r\nperhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I\r\ndid need to test multi-user and multi-space scenarios, so I ported over\r\nthe same [auth\r\nhelpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth)\r\nI saw used in other suites. Would be nice if these were bundled into the\r\nclient as well ala how the o11y folks have done it\r\n[here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34).\r\nPerhaps this is also on the list of things for @maximpn to generate from\r\nOAS's.... 🙃\r\n\r\n### RBAC\r\nIn plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all\r\nthe places I came across that will require an RBAC check/change. This\r\nincludes some of the API integration tests, which I currently have\r\nskipped as they would fail without RBAC.\r\n\r\n### Other notable changes\r\n\r\n* There are now dedicated `legacy` and `v2` helper functions when\r\nmanaging persistence/retrieval of knowledge base entries. This should\r\nhelp with tearing out the old KB later, and better readability now.\r\n* I've tried to remove dependency on the `ElasticsearchStore` as much as\r\npossible. The store's only use should now be within tools as a retriever\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405),\r\nand in post_evaluate\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179).\r\nIf we adopt the new\r\n[`naturalLanguageToESQL`](https://github.com/elastic/kibana/pull/192042)\r\ntool in `8.16` (or update our existing ESQL tool to use the\r\n`kbDataClient` for retrieval), we should be able to get rid of this\r\nentirely.\r\n* Added a\r\n[`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1)\r\nfile for adding spaces, roles, users, and a sample `slackbot` index for\r\nuse with [sample `IndexEntries`\r\nhere](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56).\r\n\r\n### // TODO\r\nIn effort to make incremental progress and facilitate early knowledge\r\nshare with @patrykkopycinski, I'm capping this PR where it's at, and so\r\nhere are the remaining items to complete full integration of the new\r\nKnowledge Base Management Settings interface:\r\n\r\n- [ ] Support `Update` action\r\n- [ ] Move from `EuiInMemoryTable` \r\n- [ ] Finalize `Setup` UI\r\n- [ ] Cleanup `Save` loaders\r\n- [ ] Plumb through `{{knowledge_history}}` prompt template and include\r\nuse's `required` entries\r\n\r\nAll this work is behind the aforementioned feature flag and required\r\ncode change, and this changeset has also been manually upgrade tested to\r\nensure there are no issues that would impact the regularly scheduled\r\nserverless releases. This is more of a note to reviewers when testing\r\nthat full functionality is not present.\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"63730ea0c9d9b036a05cb919b25b6d19c2ea8f03"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192665","number":192665,"mergeCommit":{"message":"[Security Assistant] Adds new Knowledge Base Management Settings UI (elastic#192665)\n\n## Summary\r\n\r\nThis PR updates the Knowledge Base Management Settings page to use the\r\nnew `entries` API introduced in\r\nhttps://github.com/elastic/pull/186566. Many thanks to @angorayc\r\nfor her work on the Assistant Management Settings overhaul, and initial\r\nimplementation of this new KB Management UI over in\r\nhttps://github.com/elastic/pull/186847.\r\n\r\n<p align=\"center\">\r\n<img width=\"600\"\r\nsrc=\"https://github.com/user-attachments/assets/0a82587e-f33c-45f1-9165-1a676d6db5fa\"\r\n/>\r\n</p> \r\n\r\n\r\n\r\n### Feature Flag & Setup\r\nThe changes in this PR, as with the other [recent V2 KB\r\nenhancements](elastic#186566), are behind\r\nthe following feature flag:\r\n```\r\nxpack.securitySolution.enableExperimental:\r\n - 'assistantKnowledgeBaseByDefault'\r\n```\r\n\r\n~They also require a code change in the `AIAssistantService` to enable\r\nthe new mapping (since setup happens on plugin start before FF\r\nregistration), so be sure to update `fieldMap` to\r\n`knowledgeBaseFieldMapV2` below before testing:~\r\n\r\nThis is no longer the case as of\r\n[cdec104](https://github.com/elastic/kibana/pull/192665/commits/cdec10402f2e9b889598693f9f415c98ccd9855c).\r\nJust changing the above feature flag is now sufficient, just note that\r\nif upgrading and the KB was previously setup, you'll need to manually\r\ndelete the data stream (`DELETE\r\n/_data_stream/.kibana-elastic-ai-assistant-knowledge-base-default`) or\r\nthe management table will be littered with the old ESQL docs instead of\r\nbeing a single aggregate entry.\r\n\r\nOnce configured, the new Knowledge Base Management Settings will become\r\navailable in Stack Management. The old settings UI is currently still\r\navailable via the Settings Modal, but will soon be removed and replaced\r\nwith links to the new interface via the Assistant Settings Context Menu\r\n(replacing the existing `cog`). Please see the designs ([Security\r\nGenAI](https://www.figma.com/design/BMvpY9EhcPIaoOS7LSrkL0/%5B8.15%2C-%5D-GenAI-Security-Settings?node-id=51-25207&node-type=canvas&t=t3vZSPhMxQhScJVt-0)\r\n/ [Unified AI\r\nAssistant](https://www.figma.com/design/xN20zMRNtMlirWB6n9n1xJ/Unified-AI-Assistant-Settings?node-id=0-1&node-type=canvas&t=3RDYE7h2DjLlFlcN-0))\r\nfor all changes.\r\n\r\n> [!IMPORTANT]\r\n> There are no migrations in place between the legacy and v2 KB\r\nmappings, so be sure to start with a clean ES data directory.\r\n\r\n### Testing\r\n\r\nTo aid with developing the UI, I took the opportunity to start fleshing\r\nout the KB Entries API integration tests. These live in\r\n[x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries](https://github.com/spong/kibana/tree/7ae6be136ad992b2163df13b55118556b01b6cb9/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries),\r\nand are currently configured to only run on `@ess`, as running\r\n`tiny_elser` in serverless and MKI environments can be tricky (more on\r\nthat later).\r\n\r\nTo start the server and run the tests, from the\r\n`x-pack/test/security_solution_api_integration/` directory run `yarn\r\ngenai_kb_entries:server:ess`, and once started, `yarn\r\ngenai_kb_entries:runner:ess`.\r\n\r\n##### Changes in support of testing\r\n\r\nIn order to setup the API integration tests for use with the Knowledge\r\nBase, some functional changes needed to be made to the assistant/config:\r\n\r\n1. Since ELSER is a heavy model to run in CI, the ML folks have created\r\n`pt_tiny_elser` for use in testing. Unfortunately, the `getELSER()`\r\nhelper off the `ml` client that we use to get the `modelld` for\r\ninstalling ELSER, ingest pipelines, etc, cannot be overridden\r\n([elastic#193633](elastic#193633)), so we must\r\nhave some other means of doing that. So to get things working in the\r\ntest env, I've plumbed through an optional `modelId` override to the\r\nPOST knowledge base route (`/ internal/ elastic_assistant/\r\nknowledge_base/{resource?}?modelId=pt_tiny_elser`). This then overrides\r\nthe aiAssistantService `getELSER()` function [when\r\nfetching](https://github.com/elastic/kibana/blob/645b3b863be16d70b8a7130a84b248c19729c340/x-pack/plugins/elastic_assistant/server/ai_assistant_service/index.ts#L334-L354)\r\na `kbDataClient` using the request, which appears to be the only way to\r\nalso trigger a reinitialization of the ingest pipeline (which required\r\nthe `modelId`), since that usually only occurs on plugin start. If there\r\nis a cleaner way to perform this reinitialization, please let me know!\r\n\r\n2. Turns out\r\n[`getService('ml').importTrainedModel()`](https://github.com/elastic/kibana/blob/f18224c6869ae52228da3764ca9a427106b872fb/x-pack/test/functional/services/ml/api.ts#L1575-L1587)\r\ncan't be run in test env's with `ssl:true`, which is the default\r\nsecurity config. You can read more about that issue in\r\n[elastic#193477](elastic#193477), but the\r\ncurrent workaround is to turn off `ssl` for this specific test\r\nconfiguration, so that's why\r\n[`ess.config.ts`](https://github.com/spong/kibana/blob/cf73d4c7fcd69207a9625046456a94212da833c7/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/trial_license_complete_tier/configs/ess.config.ts#L22)\r\nlooks a little different. If there's a better way to manage this config,\r\nalso please let me know!\r\n\r\n##### Additional notes\r\n\r\nWe don't currently have a `securityAssistant` API client/service to use\r\nin integration tests, so I've just been creating one-off functions using\r\n`supertest` for now. I don't have the bandwidth to work this now, but\r\nperhaps @MadameSheema / @muskangulati-qasource could lend a hand here? I\r\ndid need to test multi-user and multi-space scenarios, so I ported over\r\nthe same [auth\r\nhelpers](https://github.com/elastic/kibana/tree/dc26f1012f35c2445028a87dcc8cb3f063e058b0/x-pack/test/security_solution_api_integration/test_suites/genai/knowledge_base/entries/utils/auth)\r\nI saw used in other suites. Would be nice if these were bundled into the\r\nclient as well ala how the o11y folks have done it\r\n[here](https://github.com/elastic/kibana/blob/e9f23aa98e3abadd491be61b17e7daa3cc110cdb/x-pack/test/observability_ai_assistant_api_integration/tests/knowledge_base/knowledge_base.spec.ts#L27-L34).\r\nPerhaps this is also on the list of things for @maximpn to generate from\r\nOAS's.... 🙃\r\n\r\n### RBAC\r\nIn plumbing the UI, I've tried to place `// TODO: KB-RBAC` tags in all\r\nthe places I came across that will require an RBAC check/change. This\r\nincludes some of the API integration tests, which I currently have\r\nskipped as they would fail without RBAC.\r\n\r\n### Other notable changes\r\n\r\n* There are now dedicated `legacy` and `v2` helper functions when\r\nmanaging persistence/retrieval of knowledge base entries. This should\r\nhelp with tearing out the old KB later, and better readability now.\r\n* I've tried to remove dependency on the `ElasticsearchStore` as much as\r\npossible. The store's only use should now be within tools as a retriever\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/helpers.ts#L397-L405),\r\nand in post_evaluate\r\n[here](https://github.com/elastic/kibana/blob/de89153368848397df823c062e907a607d347dff/x-pack/plugins/elastic_assistant/server/routes/evaluate/post_evaluate.ts#L170-L179).\r\nIf we adopt the new\r\n[`naturalLanguageToESQL`](https://github.com/elastic/kibana/pull/192042)\r\ntool in `8.16` (or update our existing ESQL tool to use the\r\n`kbDataClient` for retrieval), we should be able to get rid of this\r\nentirely.\r\n* Added a\r\n[`spaces_roles_users_data.http`](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/utils/spaces_roles_users_data.http#L1)\r\nfile for adding spaces, roles, users, and a sample `slackbot` index for\r\nuse with [sample `IndexEntries`\r\nhere](https://github.com/elastic/kibana/blob/7447394fe39d5e2e098c266c14875d3aa17d3067/x-pack/packages/kbn-elastic-assistant-common/impl/schemas/knowledge_base/entries/crud_knowledge_base_entries_route.http#L18-L56).\r\n\r\n### // TODO\r\nIn effort to make incremental progress and facilitate early knowledge\r\nshare with @patrykkopycinski, I'm capping this PR where it's at, and so\r\nhere are the remaining items to complete full integration of the new\r\nKnowledge Base Management Settings interface:\r\n\r\n- [ ] Support `Update` action\r\n- [ ] Move from `EuiInMemoryTable` \r\n- [ ] Finalize `Setup` UI\r\n- [ ] Cleanup `Save` loaders\r\n- [ ] Plumb through `{{knowledge_history}}` prompt template and include\r\nuse's `required` entries\r\n\r\nAll this work is behind the aforementioned feature flag and required\r\ncode change, and this changeset has also been manually upgrade tested to\r\nensure there are no issues that would impact the regularly scheduled\r\nserverless releases. This is more of a note to reviewers when testing\r\nthat full functionality is not present.\r\n\r\n\r\n\r\n\r\n### Checklist\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>","sha":"63730ea0c9d9b036a05cb919b25b6d19c2ea8f03"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
patrykkopycinski
added a commit
to elastic/kibana
that referenced
this issue
Oct 3, 2024
## Summary This PR adds the [Elastic Security Labs](https://www.elastic.co/security-labs) content to the Security Assistant Knowledge Base. Content is currently indexed and embedded from its source `.mdx`, and we're using the same default chunking strategy as with the ESQL documentation at the moment. Since some of this content is quite large, we may want to explore other chunking strategies. Now you can ask to fetch your recent alerts and ask if there's any Elastic Security Labs content about them 🙂. LangSmith trace [here](https://smith.langchain.com/public/0e906acd-ab56-479d-b6d2-9c7538f54d45/r). Malware alerts courtesy of @jamesspi's https://github.com/jamesspi/ohmymalware project <img width="16" src="https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png" /> <p align="center"> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/750b6dc5-90b9-444e-be1d-c3df892dae61" /> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/e052cea0-ada4-4099-8b97-be05c1379b00" /> </p> <img width="2056" alt="image" src="https://github.com/user-attachments/assets/9a58a064-deb3-4ad4-a3a9-bb72d056e8b4"> ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Oct 3, 2024
…ic#184885) ## Summary This PR adds the [Elastic Security Labs](https://www.elastic.co/security-labs) content to the Security Assistant Knowledge Base. Content is currently indexed and embedded from its source `.mdx`, and we're using the same default chunking strategy as with the ESQL documentation at the moment. Since some of this content is quite large, we may want to explore other chunking strategies. Now you can ask to fetch your recent alerts and ask if there's any Elastic Security Labs content about them 🙂. LangSmith trace [here](https://smith.langchain.com/public/0e906acd-ab56-479d-b6d2-9c7538f54d45/r). Malware alerts courtesy of @jamesspi's https://github.com/jamesspi/ohmymalware project <img width="16" src="https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png" /> <p align="center"> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/750b6dc5-90b9-444e-be1d-c3df892dae61" /> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/e052cea0-ada4-4099-8b97-be05c1379b00" /> </p> <img width="2056" alt="image" src="https://github.com/user-attachments/assets/9a58a064-deb3-4ad4-a3a9-bb72d056e8b4"> ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 3499fbb)
kibanamachine
added a commit
to elastic/kibana
that referenced
this issue
Oct 4, 2024
…184885) (#194886) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Assistant] Adds Security Labs Knowledge Base content (#184885)](#184885) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Garrett Spong","email":"spong@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-03T23:22:03Z","message":"[Security Assistant] Adds Security Labs Knowledge Base content (#184885)\n\n## Summary\r\n\r\nThis PR adds the [Elastic Security\r\nLabs](https://www.elastic.co/security-labs) content to the Security\r\nAssistant Knowledge Base. Content is currently indexed and embedded from\r\nits source `.mdx`, and we're using the same default chunking strategy as\r\nwith the ESQL documentation at the moment. Since some of this content is\r\nquite large, we may want to explore other chunking strategies.\r\n\r\nNow you can ask to fetch your recent alerts and ask if there's any\r\nElastic Security Labs content about them 🙂.\r\n\r\nLangSmith trace\r\n[here](https://smith.langchain.com/public/0e906acd-ab56-479d-b6d2-9c7538f54d45/r).\r\n\r\nMalware alerts courtesy of @jamesspi's\r\nhttps://github.com/jamesspi/ohmymalware project <img width=\"16\"\r\nsrc=\"https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png\"\r\n/>\r\n\r\n\r\n<p align=\"center\">\r\n<img width=\"375\"\r\nsrc=\"https://github.com/elastic/kibana/assets/2946766/750b6dc5-90b9-444e-be1d-c3df892dae61\"\r\n/> <img width=\"375\"\r\nsrc=\"https://github.com/elastic/kibana/assets/2946766/e052cea0-ada4-4099-8b97-be05c1379b00\"\r\n/>\r\n</p> \r\n\r\n<img width=\"2056\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9a58a064-deb3-4ad4-a3a9-bb72d056e8b4\">\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"3499fbbc831ce6b9e65f73a9eb11b9677701c963","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Feature:Security Assistant","Team:Security Generative AI","v8.16.0","backport:version"],"title":"[Security Assistant] Adds Security Labs Knowledge Base content","number":184885,"url":"https://github.com/elastic/kibana/pull/184885","mergeCommit":{"message":"[Security Assistant] Adds Security Labs Knowledge Base content (#184885)\n\n## Summary\r\n\r\nThis PR adds the [Elastic Security\r\nLabs](https://www.elastic.co/security-labs) content to the Security\r\nAssistant Knowledge Base. Content is currently indexed and embedded from\r\nits source `.mdx`, and we're using the same default chunking strategy as\r\nwith the ESQL documentation at the moment. Since some of this content is\r\nquite large, we may want to explore other chunking strategies.\r\n\r\nNow you can ask to fetch your recent alerts and ask if there's any\r\nElastic Security Labs content about them 🙂.\r\n\r\nLangSmith trace\r\n[here](https://smith.langchain.com/public/0e906acd-ab56-479d-b6d2-9c7538f54d45/r).\r\n\r\nMalware alerts courtesy of @jamesspi's\r\nhttps://github.com/jamesspi/ohmymalware project <img width=\"16\"\r\nsrc=\"https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png\"\r\n/>\r\n\r\n\r\n<p align=\"center\">\r\n<img width=\"375\"\r\nsrc=\"https://github.com/elastic/kibana/assets/2946766/750b6dc5-90b9-444e-be1d-c3df892dae61\"\r\n/> <img width=\"375\"\r\nsrc=\"https://github.com/elastic/kibana/assets/2946766/e052cea0-ada4-4099-8b97-be05c1379b00\"\r\n/>\r\n</p> \r\n\r\n<img width=\"2056\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9a58a064-deb3-4ad4-a3a9-bb72d056e8b4\">\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"3499fbbc831ce6b9e65f73a9eb11b9677701c963"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/184885","number":184885,"mergeCommit":{"message":"[Security Assistant] Adds Security Labs Knowledge Base content (#184885)\n\n## Summary\r\n\r\nThis PR adds the [Elastic Security\r\nLabs](https://www.elastic.co/security-labs) content to the Security\r\nAssistant Knowledge Base. Content is currently indexed and embedded from\r\nits source `.mdx`, and we're using the same default chunking strategy as\r\nwith the ESQL documentation at the moment. Since some of this content is\r\nquite large, we may want to explore other chunking strategies.\r\n\r\nNow you can ask to fetch your recent alerts and ask if there's any\r\nElastic Security Labs content about them 🙂.\r\n\r\nLangSmith trace\r\n[here](https://smith.langchain.com/public/0e906acd-ab56-479d-b6d2-9c7538f54d45/r).\r\n\r\nMalware alerts courtesy of @jamesspi's\r\nhttps://github.com/jamesspi/ohmymalware project <img width=\"16\"\r\nsrc=\"https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png\"\r\n/>\r\n\r\n\r\n<p align=\"center\">\r\n<img width=\"375\"\r\nsrc=\"https://github.com/elastic/kibana/assets/2946766/750b6dc5-90b9-444e-be1d-c3df892dae61\"\r\n/> <img width=\"375\"\r\nsrc=\"https://github.com/elastic/kibana/assets/2946766/e052cea0-ada4-4099-8b97-be05c1379b00\"\r\n/>\r\n</p> \r\n\r\n<img width=\"2056\" alt=\"image\"\r\nsrc=\"https://github.com/user-attachments/assets/9a58a064-deb3-4ad4-a3a9-bb72d056e8b4\">\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Feature currently behind feature flag. Documentation to be added\r\nbefore flag is removed. Tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337\r\n- [X] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"3499fbbc831ce6b9e65f73a9eb11b9677701c963"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
tiansivive
pushed a commit
to tiansivive/kibana
that referenced
this issue
Oct 7, 2024
…ic#184885) ## Summary This PR adds the [Elastic Security Labs](https://www.elastic.co/security-labs) content to the Security Assistant Knowledge Base. Content is currently indexed and embedded from its source `.mdx`, and we're using the same default chunking strategy as with the ESQL documentation at the moment. Since some of this content is quite large, we may want to explore other chunking strategies. Now you can ask to fetch your recent alerts and ask if there's any Elastic Security Labs content about them 🙂. LangSmith trace [here](https://smith.langchain.com/public/0e906acd-ab56-479d-b6d2-9c7538f54d45/r). Malware alerts courtesy of @jamesspi's https://github.com/jamesspi/ohmymalware project <img width="16" src="https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png" /> <p align="center"> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/750b6dc5-90b9-444e-be1d-c3df892dae61" /> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/e052cea0-ada4-4099-8b97-be05c1379b00" /> </p> <img width="2056" alt="image" src="https://github.com/user-attachments/assets/9a58a064-deb3-4ad4-a3a9-bb72d056e8b4"> ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
tiansivive
pushed a commit
to tiansivive/kibana
that referenced
this issue
Oct 7, 2024
…ic#184885) ## Summary This PR adds the [Elastic Security Labs](https://www.elastic.co/security-labs) content to the Security Assistant Knowledge Base. Content is currently indexed and embedded from its source `.mdx`, and we're using the same default chunking strategy as with the ESQL documentation at the moment. Since some of this content is quite large, we may want to explore other chunking strategies. Now you can ask to fetch your recent alerts and ask if there's any Elastic Security Labs content about them 🙂. LangSmith trace [here](https://smith.langchain.com/public/0e906acd-ab56-479d-b6d2-9c7538f54d45/r). Malware alerts courtesy of @jamesspi's https://github.com/jamesspi/ohmymalware project <img width="16" src="https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png" /> <p align="center"> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/750b6dc5-90b9-444e-be1d-c3df892dae61" /> <img width="375" src="https://github.com/elastic/kibana/assets/2946766/e052cea0-ada4-4099-8b97-be05c1379b00" /> </p> <img width="2056" alt="image" src="https://github.com/user-attachments/assets/9a58a064-deb3-4ad4-a3a9-bb72d056e8b4"> ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Feature currently behind feature flag. Documentation to be added before flag is removed. Tracked in elastic/security-docs#5337 - [X] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com> Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
patrykkopycinski
added a commit
to elastic/kibana
that referenced
this issue
Oct 9, 2024
…194354) ## Summary This PR is a follow up to #192665 and addresses a bunch of feedback and fixes including: - [X] Adds support for updating/editing entries - [X] Fixes initial loading experience of the KB Settings Setup/Table - [X] Fixes two bugs where `semantic_text` and `text` must be declared for `IndexEntries` to work - [X] Add new Settings Context Menu items for KB and Alerts - [X] Add support for `required` entries in initial prompt * See [this trace](https://smith.langchain.com/public/84a17a31-8ce8-4bd9-911e-38a854484dd8/r) for included knowledge. Note that the KnowledgeBaseRetrievalTool was not selected. * Note: All prompts were updated to include the `{knowledge_history}` placeholder, and _not behind the feature flag_, as this will just be the empty case until the feature flag is enabled. TODO (in this or follow-up PR): - [ ] Add suggestions to `index` and `fields` inputs - [ ] Adds URL deeplinking to securityAssistantManagement - [ ] Fix bug where updating entry does not re-create embeddings (see [comment](#194354 (comment))) - [ ] Fix loading indicators when adding/editing entries - [ ] API integration tests for update API (@e40pud) ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Docs being tracked in elastic/security-docs#5337 for when feature flag is enabled - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this issue
Oct 9, 2024
…lastic#194354) ## Summary This PR is a follow up to elastic#192665 and addresses a bunch of feedback and fixes including: - [X] Adds support for updating/editing entries - [X] Fixes initial loading experience of the KB Settings Setup/Table - [X] Fixes two bugs where `semantic_text` and `text` must be declared for `IndexEntries` to work - [X] Add new Settings Context Menu items for KB and Alerts - [X] Add support for `required` entries in initial prompt * See [this trace](https://smith.langchain.com/public/84a17a31-8ce8-4bd9-911e-38a854484dd8/r) for included knowledge. Note that the KnowledgeBaseRetrievalTool was not selected. * Note: All prompts were updated to include the `{knowledge_history}` placeholder, and _not behind the feature flag_, as this will just be the empty case until the feature flag is enabled. TODO (in this or follow-up PR): - [ ] Add suggestions to `index` and `fields` inputs - [ ] Adds URL deeplinking to securityAssistantManagement - [ ] Fix bug where updating entry does not re-create embeddings (see [comment](elastic#194354 (comment))) - [ ] Fix loading indicators when adding/editing entries - [ ] API integration tests for update API (@e40pud) ### Checklist Delete any items that are not applicable to this PR. - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials * Docs being tracked in elastic/security-docs#5337 for when feature flag is enabled - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com> (cherry picked from commit 7df3672)
kibanamachine
added a commit
to elastic/kibana
that referenced
this issue
Oct 9, 2024
…xes (#194354) (#195644) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Assistant] V2 Knowledge Base Settings feedback and fixes (#194354)](#194354) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Garrett Spong","email":"spong@users.noreply.github.com"},"sourceCommit":{"committedDate":"2024-10-09T16:17:47Z","message":"[Security Assistant] V2 Knowledge Base Settings feedback and fixes (#194354)\n\n## Summary\r\n\r\nThis PR is a follow up to #192665 and addresses a bunch of feedback and\r\nfixes including:\r\n\r\n- [X] Adds support for updating/editing entries\r\n- [X] Fixes initial loading experience of the KB Settings Setup/Table\r\n- [X] Fixes two bugs where `semantic_text` and `text` must be declared\r\nfor `IndexEntries` to work\r\n- [X] Add new Settings Context Menu items for KB and Alerts\r\n - [X] Add support for `required` entries in initial prompt\r\n* See [this\r\ntrace](https://smith.langchain.com/public/84a17a31-8ce8-4bd9-911e-38a854484dd8/r)\r\nfor included knowledge. Note that the KnowledgeBaseRetrievalTool was not\r\nselected.\r\n* Note: All prompts were updated to include the `{knowledge_history}`\r\nplaceholder, and _not behind the feature flag_, as this will just be the\r\nempty case until the feature flag is enabled.\r\n\r\nTODO (in this or follow-up PR):\r\n - [ ] Add suggestions to `index` and `fields` inputs\r\n - [ ] Adds URL deeplinking to securityAssistantManagement\r\n- [ ] Fix bug where updating entry does not re-create embeddings (see\r\n[comment](https://github.com/elastic/kibana/pull/194354#discussion_r1786475496))\r\n - [ ] Fix loading indicators when adding/editing entries\r\n - [ ] API integration tests for update API (@e40pud)\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Docs being tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337 for when feature\r\nflag is enabled\r\n- [ ] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>","sha":"7df36721923159f45bc4fdbd26f76b20ad84249a","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Feature:Security Assistant","Team:Security Generative AI","v8.16.0","backport:version"],"title":"[Security Assistant] V2 Knowledge Base Settings feedback and fixes","number":194354,"url":"https://github.com/elastic/kibana/pull/194354","mergeCommit":{"message":"[Security Assistant] V2 Knowledge Base Settings feedback and fixes (#194354)\n\n## Summary\r\n\r\nThis PR is a follow up to #192665 and addresses a bunch of feedback and\r\nfixes including:\r\n\r\n- [X] Adds support for updating/editing entries\r\n- [X] Fixes initial loading experience of the KB Settings Setup/Table\r\n- [X] Fixes two bugs where `semantic_text` and `text` must be declared\r\nfor `IndexEntries` to work\r\n- [X] Add new Settings Context Menu items for KB and Alerts\r\n - [X] Add support for `required` entries in initial prompt\r\n* See [this\r\ntrace](https://smith.langchain.com/public/84a17a31-8ce8-4bd9-911e-38a854484dd8/r)\r\nfor included knowledge. Note that the KnowledgeBaseRetrievalTool was not\r\nselected.\r\n* Note: All prompts were updated to include the `{knowledge_history}`\r\nplaceholder, and _not behind the feature flag_, as this will just be the\r\nempty case until the feature flag is enabled.\r\n\r\nTODO (in this or follow-up PR):\r\n - [ ] Add suggestions to `index` and `fields` inputs\r\n - [ ] Adds URL deeplinking to securityAssistantManagement\r\n- [ ] Fix bug where updating entry does not re-create embeddings (see\r\n[comment](https://github.com/elastic/kibana/pull/194354#discussion_r1786475496))\r\n - [ ] Fix loading indicators when adding/editing entries\r\n - [ ] API integration tests for update API (@e40pud)\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Docs being tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337 for when feature\r\nflag is enabled\r\n- [ ] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>","sha":"7df36721923159f45bc4fdbd26f76b20ad84249a"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/194354","number":194354,"mergeCommit":{"message":"[Security Assistant] V2 Knowledge Base Settings feedback and fixes (#194354)\n\n## Summary\r\n\r\nThis PR is a follow up to #192665 and addresses a bunch of feedback and\r\nfixes including:\r\n\r\n- [X] Adds support for updating/editing entries\r\n- [X] Fixes initial loading experience of the KB Settings Setup/Table\r\n- [X] Fixes two bugs where `semantic_text` and `text` must be declared\r\nfor `IndexEntries` to work\r\n- [X] Add new Settings Context Menu items for KB and Alerts\r\n - [X] Add support for `required` entries in initial prompt\r\n* See [this\r\ntrace](https://smith.langchain.com/public/84a17a31-8ce8-4bd9-911e-38a854484dd8/r)\r\nfor included knowledge. Note that the KnowledgeBaseRetrievalTool was not\r\nselected.\r\n* Note: All prompts were updated to include the `{knowledge_history}`\r\nplaceholder, and _not behind the feature flag_, as this will just be the\r\nempty case until the feature flag is enabled.\r\n\r\nTODO (in this or follow-up PR):\r\n - [ ] Add suggestions to `index` and `fields` inputs\r\n - [ ] Adds URL deeplinking to securityAssistantManagement\r\n- [ ] Fix bug where updating entry does not re-create embeddings (see\r\n[comment](https://github.com/elastic/kibana/pull/194354#discussion_r1786475496))\r\n - [ ] Fix loading indicators when adding/editing entries\r\n - [ ] API integration tests for update API (@e40pud)\r\n\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [X] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n* Docs being tracked in\r\nhttps://github.com/elastic/security-docs/issues/5337 for when feature\r\nflag is enabled\r\n- [ ] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>\r\nCo-authored-by: Patryk Kopycinski <contact@patrykkopycinski.com>","sha":"7df36721923159f45bc4fdbd26f76b20ad84249a"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Garrett Spong <spong@users.noreply.github.com>
![https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png\"\r\n/>\r\n\r\n\r\n<p](https://user-images.githubusercontent.com/2946766/141219243-86619f7f-c300-4355-96f5-316d70aa6a0e.png\"\r\n/>\r\n\r\n\r\n<p){kind=link}
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
Wanted to create this epic to cover all the new work around the Security Assistant Knowledge Base for
8.15. The main effort can be summarized as adding support for 'custom Knowledge Base content' (both adding raw content and linking indices/data streams), but also includes improvements to how the KB is setup, managed in settings, and some new content we're planning on shipping, like exports of the Elastic Security Labs content.Background & resources
Open internal issues for tracking:
[UX] Knowledge Base design: Stack management and AI Assistant UIs #9392
[Epic] AI Assistant - Incorporate Latest Elastic Security Labs' Research into KB #8043
[Epic] AI Assistant and Insights - Allow users to import custom KB articles #8737
PR's thus far:
[Security Assistant] Automatically Install Knowledge Base #182763
[Security Assistant] Migrates to LangGraph and adds KB Tools #184554
[Security Assistant] Adds Security Labs Knowledge Base content #184885
[Security Assistant] Enables automatic setup of Knowledge Base and LangGraph code paths for 8.15 #188168
Point of contact: @spong @jamesspi @YulNaumenko
Test environments: TBD, but available by enabling the
assistantKnowledgeBaseByDefaultfeature flagWhich documentation set does this change impact?
ESS and serverless
ESS release
8.15
Serverless release
Soon after
8.15FF, so near mid-JulyFeature differences
Should have fully parity, but if anything ELSER/KB setup might end up being automatic in Serverless (so no 'Install Knowledge Base' button within the assistant).
API docs impact
We'll be introducing a whole new API for managing KB documents. This API is intended to be public at some point, but may ship as experimental or internal initially so we have flexibility to make modifications.
Initial OpenAPI Specs were added in [Security Assistant] Automatically Install Knowledge Base #182763, but they are still subject to change.
Prerequisites, privileges, feature flags
assistantKnowledgeBaseByDefaultxpack.securitySolution.enableExperimentalfeature flagThe text was updated successfully, but these errors were encountered: