Skip to content

Commit

Permalink
chore: Added AI support docs for automation (#2249)
Browse files Browse the repository at this point in the history
  • Loading branch information
jsumners-nr authored Jun 11, 2024
1 parent 7cd6aef commit e2efc1f
Show file tree
Hide file tree
Showing 3 changed files with 230 additions and 0 deletions.
1 change: 1 addition & 0 deletions .github/workflows/compatibility-report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ on:
- main
paths:
- 'test/versioned/**/package.json'
- 'ai-support.json'

jobs:
local:
Expand Down
153 changes: 153 additions & 0 deletions ai-support.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
[
{
"kind": "gateway",
"title": "Amazon Bedrock",
"preamble": "Through the `@aws-sdk/client-bedrock-runtime` module, we support:",
"footnote": "Note: if a model supports streaming, we also instrument the streaming variant.",
"models": [
{
"name": "Anthropic Claude",
"features": [
{
"title": "Text",
"supported": true
},
{
"title": "Image",
"supported": false
}
]
},

{
"name": "Cohere",
"features": [
{
"title": "Text",
"supported": true
},
{
"title": "Image",
"supported": false
}
]
},

{
"name": "AI21 Labs Jurassic-2",
"features": [
{
"title": "Text",
"supported": true
},
{
"title": "Image",
"supported": false
}
]
},

{
"name": "Meta Llama2",
"features": [
{
"title": "Text",
"supported": true
},
{
"title": "Image",
"supported": false
}
]
},

{
"name": "Amazon Titan",
"features": [
{
"title": "Text",
"supported": true
},
{
"title": "Image",
"supported": false
}
]
}
]
},

{
"kind": "abstraction",
"title": "Langchain",
"featuresPreamble": "The following general features of Langchain are supported:",
"providersPreamble": "Models/providers are generally supported transitively by our instrumentation of the provider's module.",
"features": [
{
"title": "Agents",
"supported": true
},
{
"title": "Chains",
"supported": true
},
{
"title": "Vectorstores",
"supported": true
},
{
"title": "Tools",
"supported": true
}
],
"providers": [
{
"name": "Azure OpenAI",
"supported": false,
"transitively": false
},
{
"name": "Amazon Bedrock",
"supported": false,
"transitively": false
},
{
"name": "OpenAI",
"supported": true,
"transitively": true
}
]
},

{
"kind": "sdk",
"title": "OpenAI",
"featuresPreamble": "Through the `openai` module, we support:",
"features": [
{
"title": "Completions",
"supported": true
},
{
"title": "Chat",
"supported": true
},
{
"title": "Embeddings",
"supported": true
},
{
"title": "Files",
"supported": false
},
{
"title": "Images",
"supported": false
},
{
"title": "Audio",
"supported": false
}
]
}
]
76 changes: 76 additions & 0 deletions ai-support.spec.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# AI Support Spec

This document describes the structure of the [ai-support.json](./ai-support.json)
file. The JSON file is utilized by an automation to update our documentation
with the AI libraries/abstractions/gateways that we support.

## Structure

The general structure of the JSON file is that of an object who's keys
map to a descriptor for a supported AI thing. Each descriptor has a `kind`
property that indicates what sort of AI thing is being described.

### kind = "abstraction"

An abstraction is a module that provides a single common interface to many
LLMs or gateways. An abstraction descriptor has the following fields:

+ `title` (string): A human readable name for the abstraction.
+ `featuresPreamble` (string): An optional block of text that should be added
to the document prior to the features table.
+ `providersPreamble` (string): An optional block of text that should be added
to the document prior to the providers table.
+ `features` (object[]): An array of feature entities.
+ `providers` (object[]): An array of provider entities.

### kind = "gateway"

A gateway is a service that provides access to multiple large language models
(LLMs) through a single API. A gateway descriptor has the following fields:

+ `title` (string): A human readable name for the gateway.
+ `preamble` (string): An optional block of text that should be added to the
document prior to the models table.
+ `footnote` (string): An optional block of text that should be added to the
document subsequent to the models table.
+ `models` (object[]): An array of model entities.

### kind = "sdk"

A SDK is a module that provides an API that is specific to a single LLM. An SDK
descriptor has the following fields:

+ `title` (string): A human readable name for the SDK.
+ `featuresPreamble` (string): An optional block of text that should be added
to the document prior to the features table.
+ `features` (object[]): An array of feature entities.

## Entities

### Feature

Describes an LLM feature. It is an object with the following fields:

+ `title` (string): A human readable name for the feature.
+ `supported` (boolean): Indicates if our instrumentation supports the feature
or not.

### Model

Describes an LLM, the features it supports, and the features we instrument. It
is an object with the following fields:

+ `name` (string): A human readable name for the LLM.
+ `features` (object[]): An array of feature entities.

### Provider

Describes an LLM or gateway that is supported by an abstraction. It is an object
with the following fields:

+ `name` (string): A human readable name for the LLM or gateway.
+ `supported` (boolean): Indicates if we instrument this provider within the
context of the abstraction.
+ `transitively` (boolean): Indicates if we instrument this provider directly
in the instrumentation for the abstraction (`false`), or if we rely on a
transitive instrumentation (`true`).

0 comments on commit e2efc1f

Please sign in to comment.