Inference plugin: Add Bedrock model adapter #191434
Conversation
pgayvallet
added
release_note:skip
|
/ci |
pgayvallet
changed the title
pgayvallet
force-pushed
the
kbn-xxx-bedrock-adapter
branch
from
c62a666
to
49bb04b
Compare
|
/ci |
|
/ci |
|
/ci |
|
Pinging @elastic/appex-ai-infra (Team:AI Infra) |
| }; | ||
| } | ||
| // ToolChoiceType.none is not supported by claude. | ||
| return undefined; |
There was a problem hiding this comment.
in this case, should we append something to the system message?
There was a problem hiding this comment.
Yeah, I had it on my TODO and somehow forgot about it, I will add it
| case MessageRole.User: | ||
| return { | ||
| role: 'user' as const, | ||
| // content: message.content, |
There was a problem hiding this comment.
I initially used the content field for user message, then I decided to use rawContent (which gets mapped directly to content in the connector without transformations) for all type of messages.
I just forgot to delete the commented bits, thanks
| // before all operations have completed. | ||
| let nextPromise = Promise.resolve(); | ||
|
|
||
| async function handleNext(chunkBody: CompletionChunk) { |
There was a problem hiding this comment.
I don't think this actually needs to be async (my bad). so we don't need the promise either?
There was a problem hiding this comment.
TBH I kept it from your implementation because I wasn't sure what it was doing, so if you confirm it's not needed I absolutely can get rid of it.
| toolCallId: chunkBody.content_block.id, | ||
| function: { | ||
| name: chunkBody.content_block.name, | ||
| // the API returns '{}' here, which can't be merged with the deltas... |
| } | ||
| error = createInferenceInternalError(message); | ||
| } catch (decodeError) { | ||
| error = createInferenceInternalError(); |
There was a problem hiding this comment.
should we use createInferenceInternalError(decodeError.message)?
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]
History
To update your PR or re-run it, just comment with: |
## Summary Fix regression on `InvokeAIRaw` bedrock subaction introduced by #191434
Summary
Add the
bedrock(well, bedrock-claude) model adapter for the inference plugin. Also had to perform minor changes on the associated connector to add support for new capabilities.