Enable FTS5 full-text searches. #2794
Conversation
Session uses SQLite FTS5 tables that allow full-text searching, but also
includes code that renders this type of advanced search inoperable.
This patch restores the functionality, allowing the use of prefix
queries (e.g. 'lin*'), Boolean operators (AND, OR and NOT), initial
token queries (e.g. '^Linux') and NEAR groups.
Operators must appear in upper case, so 'AND' is a Boolean operator, but
'and' is a search for the literal word 'and'.
Note that whitespace implies a Boolean AND, so 'linux AND applications'
is syntactically equivalent to 'linux applications', either of which
searches for any message containing both words in any order, and not
necessarily appearing contiguously.
To search for the contiguous expression 'linux applications', surround
it with double quotes: '"linux applications"'.
Parentheses may be used to change operator precedence and build complex
expressions.
The BNF for the FTS query syntax is this:
```
<phrase> := string [*]
<phrase> := <phrase> + <phrase>
<neargroup> := NEAR ( <phrase> <phrase> ... [, N] )
<query> := [ [-] <colspec> :] [^] <phrase>
<query> := [ [-] <colspec> :] <neargroup>
<query> := [ [-] <colspec> :] ( <query> )
<query> := <query> AND <query>
<query> := <query> OR <query>
<query> := <query> NOT <query>
<colspec> := colname
<colspec> := { colname1 colname2 ... }
```
For more information, please see:
https://www.sqlite.org/fts5.html#full_text_query_syntax
|
My general feeling towards this PR is positive. I do want to enable these operators, as they facilitate richer searches. However, this PR lacks a UI implementation, which means users have no way to discover that these operators exist. This could lead to issues when users search for terms containing the operator words. I believe the way we should introduce them should mirror Discord's approach, which utilizes more user-friendly, prompt-based operators and special highlighting to distinguish them in the search box. Please refer to the screenshots below for examples. Operators listed when search is clicked on
When an operator is clicked on special formatting is used and options are given per the users in the server/chat
tracked internally via https://optf.atlassian.net/browse/SES-726 |
Session uses SQLite FTS5 tables that allow full-text searching, but also includes code that renders this type of advanced search inoperable.
This patch restores the functionality, allowing the use of prefix queries (e.g. 'lin*'), Boolean operators (AND, OR and NOT), initial token queries (e.g. '^Linux') and NEAR groups.
Operators must appear in upper case, so 'AND' is a Boolean operator, but 'and' is a search for the literal word 'and'.
Note that whitespace implies a Boolean AND, so 'linux AND applications' is syntactically equivalent to 'linux applications', either of which searches for any message containing both words in any order, and not necessarily appearing contiguously.
To search for the contiguous expression 'linux applications', surround it with double quotes: '"linux applications"'.
Parentheses may be used to change operator precedence and build complex expressions.
The BNF for the FTS query syntax is this:
For more information, please see:
https://www.sqlite.org/fts5.html#full_text_query_syntax
Contributor checklist:
clearnetbranchyarn readyrun passes successfully (more about tests here)