Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add conversation API, introduce Exchange history #450

Merged
merged 4 commits into from
May 3, 2023

Conversation

calyptobai
Copy link
Contributor

@calyptobai calyptobai commented May 3, 2023

In this PR, we add an sqlite database for storing conversation history. We also add a few endpoints for managing this history, namely listing, deleting, and retrieving conversations.

The ResponseState struct has been simplified and refactored into the idea of an Exchange. Exchanges are stored directly in the corresponding Conversation for retrieval at a later date.

Using sqlx with the database

To create new queries or modify existing ones, we can use sqlx. You'll need to cargo install sqlx-cli first, then these commands can be run from within server/bleep:

  • sqlx database reset --database-url=sqlite://bleep.db: This will set up the database from scratch, deleting an old database if it exists
  • cargo sqlx prepare --database-url=sqlite://bleep.db: This will update the sqlx-data.json file, which is nominally used to build the app without requiring a database instance.

Copy link
Collaborator

@oppiliappan oppiliappan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

works great, we should merge this in.

couple of questions (just want to discuss these, no changes required here):

  • we introduce a new data_dir, is there a reason we need a separation between the index_dir and the data_dir?
  • a conversation id is unique by means of (user_id, thread_id), can we make that the primary key instead?

server/bleep/src/webserver.rs Outdated Show resolved Hide resolved
@calyptobai calyptobai force-pushed the answer-tooling-conversations branch from dfd06a3 to 9cd545a Compare May 3, 2023 13:49
@calyptobai
Copy link
Contributor Author

@nerdypepper

we introduce a new data_dir, is there a reason we need a separation between the index_dir and the data_dir?

The index_dir is a cache dir stored in a location like ~/.cache; it's not persistent so it's not suitable for storing the DB.

a conversation id is unique by means of (user_id, thread_id), can we make that the primary key instead?

Storing an integer ID allows us to reference these rows in the future in a much easier way using a single INTEGER. This way if we add new tables, we don't have to store composite foreign keys, which tend to be very bulky to work with. We could add an index over the composite key, although the table size will likely always be small making this totally unnecessary.

@calyptobai calyptobai merged commit a8bf2ac into answer-tooling May 3, 2023
@calyptobai calyptobai deleted the answer-tooling-conversations branch May 3, 2023 14:11
calyptobai added a commit that referenced this pull request May 9, 2023
In this PR, we add an sqlite database for storing conversation history. We
also add a few endpoints for managing this history, namely listing, deleting,
and retrieving conversations.

The `ResponseState` struct has been simplified and refactored into the
idea of an `Exchange`. Exchanges are stored directly in the corresponding
`Conversation` for retrieval at a later date.

To create new queries or modify existing ones, we can use `sqlx`. You'll
need to `cargo install sqlx-cli` first, then these commands can be run from
within `server/bleep`:

- `sqlx database reset --database-url=sqlite://bleep.db`: This will set up the
  database from scratch, deleting an old database if it exists
- `cargo sqlx prepare --database-url=sqlite://bleep.db`: This will update the
  `sqlx-data.json` file, which is nominally used to build the app without
  requiring a database instance.

* Store conversation history in persistent db

* Add conversation delete endpoint

* Rework response to use Exchange list, add more routes

* Address comments
calyptobai added a commit that referenced this pull request May 19, 2023
In this PR, we add an sqlite database for storing conversation history. We
also add a few endpoints for managing this history, namely listing, deleting,
and retrieving conversations.

The `ResponseState` struct has been simplified and refactored into the
idea of an `Exchange`. Exchanges are stored directly in the corresponding
`Conversation` for retrieval at a later date.

To create new queries or modify existing ones, we can use `sqlx`. You'll
need to `cargo install sqlx-cli` first, then these commands can be run from
within `server/bleep`:

- `sqlx database reset --database-url=sqlite://bleep.db`: This will set up the
  database from scratch, deleting an old database if it exists
- `cargo sqlx prepare --database-url=sqlite://bleep.db`: This will update the
  `sqlx-data.json` file, which is nominally used to build the app without
  requiring a database instance.

* Store conversation history in persistent db

* Add conversation delete endpoint

* Rework response to use Exchange list, add more routes

* Address comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants