Backport PRs: #494, #503, #398 (#512)

* Update Users section of the docs (#494) * Update example of setting model provider and API key * resize screenshots * update chat settings screenshots * Promote “Custom model providers” and “Customizing prompt templates” to subsections, move them to the bottom of the users doc page * Update docs/source/users/index.md Co-authored-by: Michał Krassowski <5832902+krassowski@users.noreply.github.com> * Update docs/source/users/index.md Co-authored-by: Michał Krassowski <5832902+krassowski@users.noreply.github.com> * change double-backtick to single-backticks as they serve the same function * move Prompt templates sections to Developers page * Update snapshots to use the same zoom level * Update docs/source/developers/index.md Co-authored-by: Jason Weill <93281816+JasonWeill@users.noreply.github.com> * move "Custom model providers" section to Dev page * Update lang model choice screen to show OpenAI models mentioned in text around the screenshot --------- Co-authored-by: Michał Krassowski <5832902+krassowski@users.noreply.github.com> Co-authored-by: Jason Weill <93281816+JasonWeill@users.noreply.github.com> * remove config.json-related information (#503) * Base chat handler refactor for custom slash commands (#398) * Adds attributes, starts adding to subclasses * Consistent syntax * Help for all handlers * Fix slash ID error * Iterate through entry points * Fix typo in call to select() * Moves config to magics, modifies extensions to attempt to load classes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Moves config to proper location, improves error logging * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * WIP: Updates per feedback, adds custom handler * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Removes redundant code, style fixes * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Removes unnecessary custom message * Instantiates class * Validates slash ID * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Consistent arguments to chat handlers * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Refactors to avoid intentionally unused params * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Updates docs, removes custom handler from source and config * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci * Renames process_message to match base class * Adds needed parameter that had been deleted * Joins lines in contributor doc * Removes natural language routing type, which is not yet used * Update docs/source/developers/index.md Co-authored-by: Piyush Jain <piyushjain@duck.com> * Update docs/source/developers/index.md Co-authored-by: Piyush Jain <piyushjain@duck.com> * Update docs/source/developers/index.md Co-authored-by: Piyush Jain <piyushjain@duck.com> * Revises per @3coins, avoids Latinism * Removes Configurable, since we do not yet have configurable traits * Uses Literal for validation * [pre-commit.ci] auto fixes from pre-commit.com hooks for more information, see https://pre-commit.ci --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piyush Jain <piyushjain@duck.com> --------- Co-authored-by: Andrii Ieroshenko <aieroshe@amazon.com> Co-authored-by: Michał Krassowski <5832902+krassowski@users.noreply.github.com> Co-authored-by: Jason Weill <93281816+JasonWeill@users.noreply.github.com> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Piyush Jain <piyushjain@duck.com>
jupyterlab · Dec 8, 2023 · 512ee81 · 512ee81
1 parent dc4ed76
commit 512ee81
Show file tree

Hide file tree

Showing 16 changed files with 311 additions and 154 deletions.
diff --git a/docs/source/_static/chat-getting-started.png b/docs/source/_static/chat-getting-started.png
diff --git a/docs/source/_static/chat-select-model-complete.png b/docs/source/_static/chat-select-model-complete.png
diff --git a/docs/source/_static/chat-select-model.png b/docs/source/_static/chat-select-model.png
diff --git a/docs/source/_static/chat-settings-choose-language-model.png b/docs/source/_static/chat-settings-choose-language-model.png
diff --git a/docs/source/developers/index.md b/docs/source/developers/index.md
@@ -16,3 +16,149 @@ Jupyter AI classes.
 For more details about using `langchain.pydantic_v1` in an environment with
 Pydantic v2 installed, see the
 [LangChain documentation on Pydantic compatibility](https://python.langchain.com/docs/guides/pydantic_compatibility).
+
+## Custom model providers
+
+You can define new providers using the LangChain framework API. Custom providers
+inherit from both `jupyter-ai`'s `BaseProvider` and `langchain`'s [`LLM`][LLM].
+You can either import a pre-defined model from [LangChain LLM list][langchain_llms],
+or define a [custom LLM][custom_llm].
+In the example below, we define a provider with two models using
+a dummy `FakeListLLM` model, which returns responses from the `responses`
+keyword argument.
+
+```python
+# my_package/my_provider.py
+from jupyter_ai_magics import BaseProvider
+from langchain.llms import FakeListLLM
+
+
+class MyProvider(BaseProvider, FakeListLLM):
+    id = "my_provider"
+    name = "My Provider"
+    model_id_key = "model"
+    models = [
+        "model_a",
+        "model_b"
+    ]
+    def __init__(self, **kwargs):
+        model = kwargs.get("model_id")
+        kwargs["responses"] = (
+            ["This is a response from model 'a'"]
+            if model == "model_a" else
+            ["This is a response from model 'b'"]
+        )
+        super().__init__(**kwargs)
+```
+
+
+If the new provider inherits from [`BaseChatModel`][BaseChatModel], it will be available
+both in the chat UI and with magic commands. Otherwise, users can only use the new provider
+with magic commands.
+
+To make the new provider available, you need to declare it as an [entry point](https://setuptools.pypa.io/en/latest/userguide/entry_point.html):
+
+```toml
+# my_package/pyproject.toml
+[project]
+name = "my_package"
+version = "0.0.1"
+
+[project.entry-points."jupyter_ai.model_providers"]
+my-provider = "my_provider:MyProvider"
+```
+
+To test that the above minimal provider package works, install it with:
+
+```sh
+# from `my_package` directory
+pip install -e .
+```
+
+Then, restart JupyterLab. You should now see an info message in the log that mentions
+your new provider's `id`:
+
+```
+[I 2023-10-29 13:56:16.915 AiExtension] Registered model provider `my_provider`.
+```
+
+[langchain_llms]: https://api.python.langchain.com/en/v0.0.339/api_reference.html#module-langchain.llms
+[custom_llm]: https://python.langchain.com/docs/modules/model_io/models/llms/custom_llm
+[LLM]: https://api.python.langchain.com/en/v0.0.339/llms/langchain.llms.base.LLM.html#langchain.llms.base.LLM
+[BaseChatModel]: https://api.python.langchain.com/en/v0.0.339/chat_models/langchain.chat_models.base.BaseChatModel.html
+
+## Prompt templates
+
+Each provider can define **prompt templates** for each supported format. A prompt
+template guides the language model to produce output in a particular
+format. The default prompt templates are a
+[Python dictionary mapping formats to templates](https://github.com/jupyterlab/jupyter-ai/blob/57a758fa5cdd5a87da5519987895aa688b3766a8/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py#L138-L166).
+Developers who write subclasses of `BaseProvider` can override templates per
+output format, per model, and based on the prompt being submitted, by
+implementing their own
+[`get_prompt_template` function](https://github.com/jupyterlab/jupyter-ai/blob/57a758fa5cdd5a87da5519987895aa688b3766a8/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py#L186-L195).
+Each prompt template includes the string `{prompt}`, which is replaced with
+the user-provided prompt when the user runs a magic command.
+
+### Customizing prompt templates
+
+To modify the prompt template for a given format, override the `get_prompt_template` method:
+
+```python
+from langchain.prompts import PromptTemplate
+
+
+class MyProvider(BaseProvider, FakeListLLM):
+    # (... properties as above ...)
+    def get_prompt_template(self, format) -> PromptTemplate:
+        if format === "code":
+            return PromptTemplate.from_template(
+                "{prompt}\n\nProduce output as source code only, "
+                "with no text or explanation before or after it."
+            )
+        return super().get_prompt_template(format)
+```
+
+Please note that this will only work with Jupyter AI magics (the `%ai` and `%%ai` magic commands). Custom prompt templates are not used in the chat interface yet.
+
+## Custom slash commands in the chat UI
+
+You can add a custom slash command to the chat interface by
+creating a new class that inherits from `BaseChatHandler`. Set
+its `id`, `name`, `help` message for display in the user interface,
+and `routing_type`. Each custom slash command must have a unique
+slash command. Slash commands can only contain ASCII letters, numerals,
+and underscores. Each slash command must be unique; custom slash
+commands cannot replace built-in slash commands.
+
+Add your custom handler in Python code:
+
+```python
+from jupyter_ai.chat_handlers.base import BaseChatHandler, SlashCommandRoutingType
+from jupyter_ai.models import HumanChatMessage
+
+class CustomChatHandler(BaseChatHandler):
+    id = "custom"
+    name = "Custom"
+    help = "A chat handler that does something custom"
+    routing_type = SlashCommandRoutingType(slash_id="custom")
+
+    def __init__(self, *args, **kwargs):
+        super().__init__(*args, **kwargs)
+
+    async def process_message(self, message: HumanChatMessage):
+        # Put your custom logic here
+        self.reply("<your-response>", message)
+```
+
+Jupyter AI uses entry points to support custom slash commands.
+In the `pyproject.toml` file, add your custom handler to the
+`[project.entry-points."jupyter_ai.chat_handlers"]` section:
+
+```
+[project.entry-points."jupyter_ai.chat_handlers"]
+custom = "custom_package:CustomChatHandler"
+```
+
+Then, install your package so that Jupyter AI adds custom chat handlers
+to the existing chat handlers.
diff --git a/docs/source/users/index.md b/docs/source/users/index.md
@@ -155,13 +155,13 @@ in the SageMaker documentation.
 To use SageMaker's models, you will need to authenticate via
 [boto3](https://github.com/boto/boto3).
 
-For example, to use OpenAI models, install the necessary package, and set an environment
-variable when you start JupyterLab from a terminal:
+For example, to use OpenAI models, use the chat interface settings panel to choose the OpenAI language model:
 
-```bash
-pip install openai
-OPENAI_API_KEY=your-api-key-here jupyter lab
-```
+<img src="../_static/chat-settings-choose-language-model.png"
+    alt="Screen shot of the chat settings interface with language model dropdown open"
+    class="screenshot" />
+
+Then, enter your API key in the 'API Keys' section.
 
 :::{attention}
 :name: open-ai-cost
@@ -170,96 +170,6 @@ responsible for all charges they incur when they make API requests. Review your
 provider's pricing information before submitting requests via Jupyter AI.
 :::
 
-### Custom model providers
-
-You can define new providers using the LangChain framework API. Custom providers
-inherit from both `jupyter-ai`'s ``BaseProvider`` and `langchain`'s [``LLM``][LLM].
-You can either import a pre-defined model from [LangChain LLM list][langchain_llms],
-or define a [custom LLM][custom_llm].
-In the example below, we define a provider with two models using
-a dummy ``FakeListLLM`` model, which returns responses from the ``responses``
-keyword argument.
-
-```python
-# my_package/my_provider.py
-from jupyter_ai_magics import BaseProvider
-from langchain.llms import FakeListLLM
-
-
-class MyProvider(BaseProvider, FakeListLLM):
-    id = "my_provider"
-    name = "My Provider"
-    model_id_key = "model"
-    models = [
-        "model_a",
-        "model_b"
-    ]
-    def __init__(self, **kwargs):
-        model = kwargs.get("model_id")
-        kwargs["responses"] = (
-            ["This is a response from model 'a'"]
-            if model == "model_a" else
-            ["This is a response from model 'b'"]
-        )
-        super().__init__(**kwargs)
-```
-
-
-If the new provider inherits from [``BaseChatModel``][BaseChatModel], it will be available
-both in the chat UI and with magic commands. Otherwise, users can only use the new provider
-with magic commands.
-
-To make the new provider available, you need to declare it as an [entry point](https://setuptools.pypa.io/en/latest/userguide/entry_point.html):
-
-```toml
-# my_package/pyproject.toml
-[project]
-name = "my_package"
-version = "0.0.1"
-
-[project.entry-points."jupyter_ai.model_providers"]
-my-provider = "my_provider:MyProvider"
-```
-
-To test that the above minimal provider package works, install it with:
-
-```sh
-# from `my_package` directory
-pip install -e .
-```
-
-Then, restart JupyterLab. You should now see an info message in the log that mentions
-your new provider's `id`:
-
-```
-[I 2023-10-29 13:56:16.915 AiExtension] Registered model provider `my_provider`.
-```
-
-[langchain_llms]: https://api.python.langchain.com/en/latest/api_reference.html#module-langchain.llms
-[custom_llm]: https://python.langchain.com/docs/modules/model_io/models/llms/custom_llm
-[LLM]: https://api.python.langchain.com/en/latest/llms/langchain.llms.base.LLM.html#langchain.llms.base.LLM
-[BaseChatModel]: https://api.python.langchain.com/en/latest/chat_models/langchain.chat_models.base.BaseChatModel.html
-
-
-### Customizing prompt templates
-
-To modify the prompt template for a given format, override the ``get_prompt_template`` method:
-
-```python
-from langchain.prompts import PromptTemplate
-
-
-class MyProvider(BaseProvider, FakeListLLM):
-    # (... properties as above ...)
-    def get_prompt_template(self, format) -> PromptTemplate:
-        if format === "code":
-            return PromptTemplate.from_template(
-                "{prompt}\n\nProduce output as source code only, "
-                "with no text or explanation before or after it."
-            )
-        return super().get_prompt_template(format)
-```
-
 ## The chat interface
 
 The easiest way to get started with Jupyter AI is to use the chat interface.
@@ -692,20 +602,6 @@ A function that computes the lowest common multiples of two integers, and
 a function that runs 5 test cases of the lowest common multiple function
 ```
 
-### Prompt templates
-
-Each provider can define **prompt templates** for each supported format. A prompt
-template guides the language model to produce output in a particular
-format. The default prompt templates are a
-[Python dictionary mapping formats to templates](https://github.com/jupyterlab/jupyter-ai/blob/57a758fa5cdd5a87da5519987895aa688b3766a8/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py#L138-L166).
-Developers who write subclasses of `BaseProvider` can override templates per
-output format, per model, and based on the prompt being submitted, by
-implementing their own
-[`get_prompt_template` function](https://github.com/jupyterlab/jupyter-ai/blob/57a758fa5cdd5a87da5519987895aa688b3766a8/packages/jupyter-ai-magics/jupyter_ai_magics/providers.py#L186-L195).
-Each prompt template includes the string `{prompt}`, which is replaced with
-the user-provided prompt when the user runs a magic command.
-
-
 ### Clearing the OpenAI chat history
 
 With the `openai-chat` provider *only*, you can run a cell magic command using the `-r` or

diff --git a/packages/jupyter-ai/jupyter_ai/chat_handlers/__init__.py b/packages/jupyter-ai/jupyter_ai/chat_handlers/__init__.py
@@ -1,5 +1,5 @@
 from .ask import AskChatHandler
-from .base import BaseChatHandler
+from .base import BaseChatHandler, SlashCommandRoutingType
 from .clear import ClearChatHandler
 from .default import DefaultChatHandler
 from .generate import GenerateChatHandler

diff --git a/packages/jupyter-ai/jupyter_ai/chat_handlers/ask.py b/packages/jupyter-ai/jupyter_ai/chat_handlers/ask.py
@@ -7,7 +7,7 @@
 from langchain.memory import ConversationBufferWindowMemory
 from langchain.prompts import PromptTemplate
 
-from .base import BaseChatHandler
+from .base import BaseChatHandler, SlashCommandRoutingType
 
 PROMPT_TEMPLATE = """Given the following conversation and a follow up question, rephrase the follow up question to be a standalone question.
 
@@ -26,6 +26,11 @@ class AskChatHandler(BaseChatHandler):
     to the LLM to generate the final reply.
     """
 
+    id = "ask"
+    name = "Ask with Local Data"
+    help = "Asks a question with retrieval augmented generation (RAG)"
+    routing_type = SlashCommandRoutingType(slash_id="ask")
+
     def __init__(self, retriever, *args, **kwargs):
         super().__init__(*args, **kwargs)