Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setting default model providers #421

Merged
merged 14 commits into from
Jan 26, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Corrected config names, simplified docs.
  • Loading branch information
3coins committed Jan 26, 2024
commit 50e038b339c94474f7d9d08a5fad2cfc1ffe3581
24 changes: 18 additions & 6 deletions docs/source/users/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -765,14 +765,26 @@ The `--response-path` option is a [JSONPath](https://goessner.net/articles/JsonP
You can specify an allowlist, to only allow only a certain list of providers, or
a blocklist, to block some providers.

### Initializing default model providers
### Configuring default models and API keys

This configuration allows for setting a default model, and embedding provider with their corresponding API keys.
This configuration allows for setting a default language and embedding models, and their corresponding API keys.
These values are offered as a starting point for users, so they don't have to select the models and API keys, however,
the selections they make in the settings panel will take precedence over these values.

Following command line arguments can be used to initialize Jupyter AI extension with default providers:
1. ```--AiExtension.default_language_model```: Specify the default LLM model. Sample value: ```--AiExtension.default_language_model="bedrock-chat:anthropic.claude-v2"```
2. ```--AiExtension.default_embedding_model```: Specify default embedding model. Sample value: ```--AiExtension.default_embedding_model="bedrock:amazon.titan-embed-text-v1"```
3. ```--AiExtension.api_keys```: Specify model keys in a JSON format. Sample CLI argument: ```--AiExtension.api_keys='{"OPENAI_API_KEY": "sk-abcd}'```
Specify default language model
```
jupyter lab --AiExtension.default_language_model=bedrock-chat:anthropic.claude-v2
```

Specify default embedding model
```
jupyter lab --AiExtension.default_embedding_model=bedrock:amazon.titan-embed-text-v1
```

Specify default API keys
```
jupyter lab --AiExtension.default_api_keys={'OPENAI_API_KEY': 'sk-abcd'}
```


### Blocklisting providers
Expand Down
40 changes: 23 additions & 17 deletions packages/jupyter-ai/jupyter_ai/extension.py
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,6 @@ class AiExtension(ExtensionApp):
config=True,
)

api_keys = Dict(
key_trait=Unicode(),
value_trait=Unicode(),
default_value=None,
allow_none=True,
help="""API keys for model providers, as a dictionary, in the format
`<key-name>:<key-value>`. Defaults to None.""",
config=True,
)

model_parameters = Dict(
key_trait=Unicode(),
value_trait=Dict(),
Expand All @@ -119,16 +109,32 @@ class AiExtension(ExtensionApp):
default_language_model = Unicode(
default_value=None,
allow_none=True,
help="""Default language model to use, as string in the format
<provider-id>:<model-id>, defaults to None.""",
help="""
Default language model to use, as string in the format
<provider-id>:<model-id>, defaults to None.
""",
config=True,
)

default_embeddings_model = Unicode(
default_value=None,
allow_none=True,
help="""
Default embeddings model to use, as string in the format
<provider-id>:<model-id>, defaults to None.
""",
config=True,
)

default_embedding_model = Unicode(
default_api_keys = Dict(
key_trait=Unicode(),
value_trait=Unicode(),
default_value=None,
allow_none=True,
help="""Default embedding model to use, as string in the format
<provider-id>:<model-id>, defaults to None.""",
help="""
Default API keys for model providers, as a dictionary,
in the format `<key-name>:<key-value>`. Defaults to None.
""",
config=True,
)

Expand All @@ -152,8 +158,8 @@ def initialize_settings(self):

defaults = {
"model_provider_id": self.default_language_model,
"embeddings_provider_id": self.default_embedding_model,
"api_keys": self.api_keys,
"embeddings_provider_id": self.default_embeddings_model,
"api_keys": self.default_api_keys,
"fields": self.model_parameters,
}

Expand Down
Loading