Publish the anatomy of a coding assistant blog post #7002
Conversation
✅ Deploy Preview for sourcegraph ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
Co-authored-by: Beyang Liu <beyang@sourcegraph.com>
…o the Rust fine tuning article
|
|
||
| 1. **Conversation history:** In a given chat session, we record previous messages as they may contain relevant context for the user's next request. | ||
| 2. **Code search:** We fetch the most relevant code snippets related to the user's query from the codebase using search, similar to how a human dev might search for these code snippets. | ||
| 3. **User control:** Users should have the ability to mention specific files and provide those directly to the model, and also include the option to reference external sources like Slack threads or Notion documents to enrich the context further. |
There was a problem hiding this comment.
| 3. **User control:** Users should have the ability to mention specific files and provide those directly to the model, and also include the option to reference external sources like Slack threads or Notion documents to enrich the context further. | |
| 3. **User choice:** Users should have the ability to mention specific files and provide those directly to the model, and also include the option to reference external sources like Slack threads or Notion documents to enrich the context further. |
maybe "choice" instead of "control"?
There was a problem hiding this comment.
Hmm here, I think "control" fits better. "Choice" makes me think of choosing an LLM model - so maybe more like choosing one thing out of several options.
| - Diagnostic information like warnings and errors | ||
| - User-specified context from @-mentions | ||
|
|
||
| By including diagnostic information, we're able to provide more appropriate code edits to the selected range of code. In the future, we plan to incorporate code graph context here as well. |
There was a problem hiding this comment.
Do we also want to say something along the lines of "With the code editing feature, we put greater emphasis on the users prompt and rely on the above context sources to generate high quality code output."
This section feels a little incomplete and could use a few more details imo.
There was a problem hiding this comment.
Added:
With the code editing feature, we currently put greater emphasis on the users prompt and rely on the above context sources to generate high quality code output.
This section feels a little incomplete and could use a few more details imo.
Definitely. Writing this part helped me realized that there's more we can do here from the product development perspective, too.
There was a problem hiding this comment.
Awesome! Thank you. Yeah I think the editing/inserting code feature is a sleeping giant that if we invest more into, could be huge.
Co-authored-by: Ado Kukic <kukicadnan@gmail.com>
Co-authored-by: Ado Kukic <kukicadnan@gmail.com>
Co-authored-by: Ado Kukic <kukicadnan@gmail.com>
Co-authored-by: Ado Kukic <kukicadnan@gmail.com>
Co-authored-by: Ado Kukic <kukicadnan@gmail.com>
Co-authored-by: Ado Kukic <kukicadnan@gmail.com>
Co-authored-by: Ado Kukic <kukicadnan@gmail.com>
|
|
||
|  | ||
|
|
||
| For this, we look at a few sources of information: the cursor position, the surrounding code, and the code graph (a code graph is a representation of the relationships and structures within a codebase, mapping entities such as classes and methods to show how they are interconnected). We use the current cursor position within the code graph to determine if the user wants a single-line suggestion or a multi-line suggestion. Once we determine that, we add more context by looking through recent files and open tabs. Within those, we find code snippets related to the code you're currently writing. |
There was a problem hiding this comment.
It'd be good to explain how we use the code graph for autocomplete context. I know this is unclear to many people (both inside and out of Sourcegraph).
There was a problem hiding this comment.
Added a link to this to answer it - hope it clarifies it: https://sourcegraph.notion.site/How-do-we-use-the-code-graph-for-autocomplete-context-11145ad3e07049e8ab1add7ac3012f81
There was a problem hiding this comment.
That indeed helps. However It's still unclear to me when and if we actually use this. For example, is LSP context actually enabled by default for our customers?
There was a problem hiding this comment.
I looked into it, and it looks like it's determined by a feature flag: https://github.com/sourcegraph/cody/blob/main/vscode/src/completions/completion-provider-config.ts#L77
As well as a config variable: https://github.com/sourcegraph/cody/blob/54b25dc7d17bac425c2b59b04ca08d600ef184ed/vscode/package.json#L1137
There was a problem hiding this comment.
I like the "product of products" framing, and how it ties into the different context retrieval strategies. I had a couple high-level thoughts:
- I left some suggestions for expanding the "Keyword Search" description, to help emphasize our expertise in information retrieval/ search and how we have a nuanced strategy based on code. This applies to both the "query understanding" and ranking steps.
- I wonder if we should mention our internal evals briefly? Maybe just a short note about how we are "data driven" in making changes, using both offline and online metrics. I almost never hear other companies mention evals in blog posts, and this gives me a bad impression ... like "why do you have such a complicated pipeline? Did you even measure it??"
There was a problem hiding this comment.
Thanks for resolving those comments! Did you see my comment about evals?
I wonder if we should mention our internal evals briefly? Maybe just a short note about how we are "data driven" in making changes, using both offline and online metrics. I almost never hear other companies mention evals in blog posts, and this gives me a bad impression ... like "why do you have such a complicated pipeline? Did you even measure it??"
|
@jtibshirani Sorry I missed it, but mentioning our internal evals sounds good. What can we say about it specifically - or where can I find more about it? |
Co-authored-by: Julie Tibshirani <julietibs@apache.org>
No description provided.