Refactor prompt stack #861

collindutter · 2024-06-13T00:28:29Z

Added

Message for storing messages in a MessageStack. Messages consist of a role, content, and usage.
DeltaMessage for storing partial messages in a MessageStack. Multiple DeltaMessage can be combined to form a Message.
TextMessageContent for storing textual content in a Message.
ImageMessageContent for storing image content in a Message.
Support for adding TextArtifacts, ImageArtifacts, and ListArtifacts to MessageStack.
Support for image inputs to OpenAiChatPromptDriver, AzureOpenAiChatPromptDriver, AmazonBedrockPromptDriver, AnthropicPromptDriver, and GooglePromptDriver.
Input/output token usage metrics to all Prompt Drivers.
FinishPromptEvent.input_token_count and FinishPromptEvent.output_token_count.
Support for storing Artifacts as inputs/outputs in Conversation Memory Runs.
Agent.input for passing Artifacts as input.
Support for PromptTasks to take TextArtifacts, ImageArtifacts, and ListArtifacts as input.

Changed

BREAKING: Moved/renamed griptape.utils.PromptStack to griptape.common.MessageStack.
BREAKING: Renamed PromptStack.inputs to PromptStack.messages.
BREAKING: Moved PromptStack.USER_ROLE, PromptStack.ASSISTANT_ROLE, and PromptStack.SYSTEM_ROLE to Message.
BREAKING: Updated return type of PromptDriver.try_run from TextArtifact to Message.
BREAKING: Updated return type of PromptDriver.try_stream from Iterator[TextArtifact] to Iterator[DeltaMessage].
BREAKING: Removed BasePromptEvent.token_count in favor of FinishPromptEvent.input_token_count and FinishPromptEvent.output_token_count.
BREAKING: Removed StartPromptEvent.prompt. Use StartPromptEvent.message_stack instead.
BREAKING: Removed Agent.input_template in favor of Agent.input.
Default Prompt Driver model in GoogleStructureConfig to gemini-1.5-pro.

Closes #712

📚 Documentation preview 📚: https://griptape--861.org.readthedocs.build//861/

codecov · 2024-06-13T00:54:42Z

Codecov Report

Attention: Patch coverage is 90.66901% with 53 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
griptape/drivers/prompt/anthropic_prompt_driver.py	82.60%	3 Missing and 5 partials ⚠️
...iptape/drivers/prompt/openai_chat_prompt_driver.py	84.78%	3 Missing and 4 partials ⚠️
...mmon/prompt_stack/contents/base_message_content.py	75.00%	4 Missing and 1 partial ⚠️
griptape/drivers/prompt/cohere_prompt_driver.py	84.37%	2 Missing and 3 partials ⚠️
griptape/common/prompt_stack/prompt_stack.py	89.74%	3 Missing and 1 partial ⚠️
griptape/common/prompt_stack/messages/message.py	87.50%	2 Missing and 1 partial ⚠️
...ape/drivers/prompt/amazon_bedrock_prompt_driver.py	87.50%	1 Missing and 2 partials ⚠️
griptape/drivers/prompt/base_prompt_driver.py	91.89%	1 Missing and 2 partials ⚠️
griptape/tasks/prompt_task.py	92.50%	2 Missing and 1 partial ⚠️
...prompt/amazon_sagemaker_jumpstart_prompt_driver.py	91.66%	1 Missing and 1 partial ⚠️
... and 6 more

📢 Thoughts on this report? Let us know!

vachillo · 2024-06-19T23:00:05Z

griptape/common/prompt_stack/contents/text_prompt_stack_content.py

+
+    @classmethod
+    def from_deltas(cls, deltas: Sequence[BaseDeltaPromptStackContent]) -> TextPromptStackContent:
+        text_deltas = [delta for delta in deltas if isinstance(delta, TextDeltaPromptStackContent)]


should this silently fail if its not the right type?

Not sure I follow -- this will not silently fail as it currently is.

vachillo · 2024-06-19T23:07:05Z

griptape/common/prompt_stack/messages/prompt_stack_message.py

+        if isinstance(content, str):
+            content = [TextPromptStackContent(TextArtifact(value=content))]
+        self.__attrs_init__(content, **kwargs)  # pyright: ignore[reportAttributeAccessIssue]


could use an attrs converter here

I don't think there's a good way to use a converter and get proper type hints. With a converter, you'll get yelled at for trying to pass in a string. And updating the field's type hints leads down a path of Unions everywhere.

dylanholmes

Love it

griptape/common/prompt_stack/messages/delta_prompt_stack_message.py

dylanholmes · 2024-06-20T14:40:07Z

griptape/common/prompt_stack/messages/prompt_stack_message.py

+            content = [TextPromptStackContent(TextArtifact(value=content))]
+        self.__attrs_init__(content, **kwargs)  # pyright: ignore[reportAttributeAccessIssue]
+
+    content: list[BasePromptStackContent] = field(metadata={"serializable": True})


What's stopping us from replacing this with artifacts: list[BaseArtifacts]?

Is BasePromptStackContent more than a wrapper for BaseArtifact?

Went back and forth on this a million times. In this PR, yeah it's not much more. But the upcoming function calling PR, for instance, has an ActionResultPromptStackContent that contains an artifact and some metadata about the action.

Additionally, to quote something @vasinov said:

I think the reason we wanted to keep it separate is because artifacts are a pretty clean abstraction for data passing. In the PromptStack inputs really are elements that currently somewhat match artifacts but we don't know if that's going to hold true in the future. It would also be much easier to add API-specific elements if an API requires it.

Is there any way to avoid creating a PromptStackContent for each type of Artifact? Could there be like a Non-ActionResultPromptStackContent that just contains any artifact?

docs/griptape-framework/misc/events.md

griptape/common/prompt_stack/messages/base_prompt_stack_message.py

griptape/drivers/prompt/huggingface_pipeline_prompt_driver.py

dylanholmes

Minor comments and a couple of topics for discussion

dylanholmes · 2024-06-21T12:06:59Z

griptape/common/prompt_stack/messages/prompt_stack_message.py

+            content = [TextPromptStackContent(TextArtifact(value=content))]
+        self.__attrs_init__(content, **kwargs)  # pyright: ignore[reportAttributeAccessIssue]
+
+    content: list[BasePromptStackContent] = field(metadata={"serializable": True})


Is there any way to avoid creating a PromptStackContent for each type of Artifact? Could there be like a Non-ActionResultPromptStackContent that just contains any artifact?

griptape/drivers/prompt/amazon_bedrock_prompt_driver.py

griptape/drivers/prompt/anthropic_prompt_driver.py

griptape/drivers/prompt/base_prompt_driver.py

griptape/drivers/prompt/cohere_prompt_driver.py

griptape/common/prompt_stack/prompt_stack.py

docs/griptape-framework/misc/events.md

…nap!

dylanholmes

💯

dylanholmes

🥮

collindutter force-pushed the refactor/prompt-stack-elements2 branch 11 times, most recently from 1767a83 to a53762d Compare June 19, 2024 21:29

collindutter requested review from vasinov, andrewfrench and dylanholmes June 19, 2024 22:36

collindutter marked this pull request as ready for review June 19, 2024 22:36

vachillo reviewed Jun 19, 2024

View reviewed changes

dylanholmes reviewed Jun 20, 2024

View reviewed changes

collindutter force-pushed the refactor/prompt-stack-elements2 branch from 6dcf841 to c725b94 Compare June 20, 2024 16:05

collindutter requested review from dylanholmes and vachillo June 20, 2024 19:57

collindutter force-pushed the refactor/prompt-stack-elements2 branch from a0a5391 to 032bc04 Compare June 20, 2024 22:14

dylanholmes requested changes Jun 21, 2024

View reviewed changes

collindutter requested a review from dylanholmes June 21, 2024 17:13

collindutter force-pushed the refactor/prompt-stack-elements2 branch from 9611162 to f3033bd Compare June 21, 2024 17:56

collindutter added 5 commits June 21, 2024 11:01

Refactor prompt stack

aa00bdd

Add support for more modalities to conversation memory

d3995b8

Update default artifact

524a30d

Fix bad merge

7f38288

Rename Prompt Stack Element to Prompt Stack Message

2380b37

collindutter added 5 commits June 21, 2024 11:01

Improve test coverage

1297528

Add missing module

2b7fe92

Update docs

85b8cd0

Simplify inputs

635249c

Regenerate lock file

52ea784

collindutter force-pushed the refactor/prompt-stack-elements2 branch from f3033bd to 52ea784 Compare June 21, 2024 18:01

collindutter added 5 commits June 21, 2024 11:21

Update changelog

75a5c1a

Fix test

06cb2a3

Simplify cohere

5c797ad

Simplify system_prompt access

182ff92

Change prompt driver run return value

8dc4944

dylanholmes mentioned this pull request Jun 25, 2024

Add observable #851

Merged

collindutter added 3 commits July 1, 2024 16:41

Merge branch 'dev' into refactor/prompt-stack-elements2

5c663fe

Regenerate lock file

49b93f9

Fix bad merge, tests

ebedd83

collindutter force-pushed the refactor/prompt-stack-elements2 branch from 17b59cc to 29dfb0f Compare July 1, 2024 21:57

dylanholmes requested changes Jul 1, 2024

View reviewed changes

docs/griptape-framework/misc/events.md Outdated Show resolved Hide resolved

docs/griptape-framework/misc/events.md Outdated Show resolved Hide resolved

Rename MessageStack back to PromptStack. Snip-snap! Snip-snap! Snip-s…

7802c11

…nap!

collindutter force-pushed the refactor/prompt-stack-elements2 branch from 29dfb0f to 7802c11 Compare July 1, 2024 22:38

Merge branch 'dev' into refactor/prompt-stack-elements2

36b6e37

dylanholmes previously approved these changes Jul 1, 2024

View reviewed changes

Merge branch 'dev' into refactor/prompt-stack-elements2

44d54dc

collindutter force-pushed the refactor/prompt-stack-elements2 branch from 81afadb to 44d54dc Compare July 1, 2024 23:54

Fix docs

2662ff7

collindutter dismissed dylanholmes’s stale review via 2662ff7 July 1, 2024 23:55

dylanholmes approved these changes Jul 2, 2024

View reviewed changes

Merge branch 'dev' into refactor/prompt-stack-elements2

919b480

collindutter merged commit 2dcfe2d into dev Jul 2, 2024
11 checks passed

collindutter deleted the refactor/prompt-stack-elements2 branch July 2, 2024 14:48

collindutter mentioned this pull request Aug 8, 2024

Add Gemini Image Query Driver #650

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor prompt stack #861

Refactor prompt stack #861

collindutter commented Jun 13, 2024 •

edited

Loading

codecov bot commented Jun 13, 2024 •

edited

Loading

vachillo Jun 19, 2024

collindutter Jun 20, 2024

vachillo Jun 19, 2024

collindutter Jun 20, 2024

dylanholmes left a comment

dylanholmes Jun 20, 2024

collindutter Jun 20, 2024

dylanholmes Jun 21, 2024

dylanholmes left a comment

dylanholmes Jun 21, 2024

dylanholmes left a comment

dylanholmes left a comment

Refactor prompt stack #861

Refactor prompt stack #861

Conversation

collindutter commented Jun 13, 2024 • edited Loading

Added

Changed

codecov bot commented Jun 13, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dylanholmes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dylanholmes left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dylanholmes left a comment

Choose a reason for hiding this comment

dylanholmes left a comment

Choose a reason for hiding this comment

collindutter commented Jun 13, 2024 •

edited

Loading

codecov bot commented Jun 13, 2024 •

edited

Loading