From ac0e70c99a636ef5ab0acaa6b300de497143a7a6 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 6 Jan 2023 13:22:18 -0500 Subject: [PATCH 01/55] Add proposal for intentional mentions. --- proposals/3952-intentional-mentions.md | 268 +++++++++++++++++++++++++ 1 file changed, 268 insertions(+) create mode 100644 proposals/3952-intentional-mentions.md diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md new file mode 100644 index 00000000000..a785e829e27 --- /dev/null +++ b/proposals/3952-intentional-mentions.md @@ -0,0 +1,268 @@ +# MSC3952: Intentional Mentions + +Mentioning other users on Matrix is difficult -- it is not possible to know if +[mentioning a user by display name or Matrix ID](https://github.com/matrix-org/matrix-spec/issues/353) +will count as a mention, but is also too easy to mistakenly mention a user. + +(Note that throughout this proposal "mention" is considered equivalent to a "ping" +or highlight notification.) + +Some situations that result in unintentional mentions include: + +* Replying to a message will re-issue pings from the initial message due to + [fallback replies](https://spec.matrix.org/v1.5/client-server-api/#fallbacks-for-rich-replies). [^1] +* Each time a message is edited the new version will be re-evaluated for mentions. +* Mentions occurring [in spoiler contents](https://github.com/matrix-org/matrix-spec/issues/16) + or [code blocks](https://github.com/matrix-org/matrix-spec/issues/15) are + evaluated. +* If the [localpart of your Matrix ID is a common word](https://github.com/matrix-org/matrix-spec-proposals/issues/3011) + then the push rule matching usernames (`.m.rule.contains_user_name`) matches + too often (e.g. Travis CI matching if your Matrix ID is `@travis:example.com`). +* If the [localpart or display name of your Matrix ID matches the hostname](https://github.com/matrix-org/matrix-spec-proposals/issues/2735) + (e.g. `@example:example.com` receives notifications whenever `@foo:example.com` + is replied to). + +As a sender you do not know if including the user's display name or Matrix ID would +even be interpreting as a mention (see [issue 353](https://github.com/matrix-org/matrix-spec/issues/353)). + +This also results in some unexpected behavior and bugs: + +* Matrix users use "leetspeak" when sending messages to avoid mentions (e.g. + referring to M4tthew instead of Matthew). +* Matrix users will append emoji or other unique text in their display names to + avoid unintentional pings. +* It is impossible to ping one out of multiple people with the same localpart + (or display name). +* Since the relation between `body` and `formatted_body` is ill-defined and + ["pills" are converted to display names](https://github.com/matrix-org/matrix-spec/issues/714), + this can result in missed messages. +* Bridges [must use display names](https://github.com/matrix-org/matrix-spec/issues/353#issuecomment-1055809364) + as a trick to get pings to work. +* If a user changes their display name in a room than whether or not they are + mentioned changes unless you use historical display names to process push rules. + (TODO I think there's an issue about this?) + +## Background + +Mentions are powered by two of the default push rules that search an event's +`content.body` field for the current user's display name +([`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules)) +or the localpart of their Matrix ID ([`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules)). + +There's also a [section about "user and room mentions"](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) +which defines that messages which mention the current user in the `formatted_body` +of the message should be colored differently: + +> If the current user is mentioned in a message (either by a mention as defined +> in this module or by a push rule), the client should show that mention differently +> from other mentions, such as by using a red background color to signify to the +> user that they were mentioned. + +## Proposal + +Instead of searching a message's `body` for the user's display name or the localpart +of their Matrix ID, it is proposed to use a field specific to mentions,[^2] and +augment the push rules to search for the current user's Matrix ID. + +### New event field + +A new `mentions` field is added to the event content, it is an array of strings +consistent of Matrix IDs. + +To limit the potential for abuse, only the first 10 items in the array should be +considered. It is recommended that homeservers reject locally created events with +more than 10 mentions with an error with a status code of `400` and an errcode of +`M_INVALID_PARAM`. + +Clients should add a Matrix ID to this array whenever composing a message which +includes an intentional [user mention](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) +(often called a "pill"). Clients may also add them at other times when it is +obvious the user intends to explicitly mention a user. + +The `mentions` field is part of the plaintext event body and should be encrypted +into the ciphertext for encrypted events. + +### New push rules + +A new push rule condition and a new default push rule will be added: + +```json +{ + "rule_id": ".m.rule.user_is_mentioned", + "default": true, + "enabled": true, + "conditions": [ + { + "kind": "is_mention" + } + ], + "actions": [ + "notify", + { + "set_tweak": "sound", + "value": "default" + }, + { + "set_tweak": "highlight" + } + ] +} +``` + +The `is_mention` push condition is defined as[^3]: + +> This matches messages where the `content.mentions` is an array containing the +> owner’s Matrix ID in the first 10 entries. This condition has no parameters. +> If the `mentions` key is absent or not an array then the rule does not match; +> any array entries which are not strings are ignored, but count against the limit. + +An example matching event is provided below: + +```json +{ + "content": { + "body": "This is an example mention @alice:example.org", + "format": "org.matrix.custom.html", + "formatted_body": "This is an example mention Alice", + "msgtype": "m.text", + "mentions": ["@alice:example.org"] + }, + "event_id": "$143273582443PhrSn:example.org", + "origin_server_ts": 1432735824653, + "room_id": "!somewhere:over.the.rainbow", + "sender": "@example:example.org", + "type": "m.room.message", + "unsigned": { + "age": 1234 + } +} +``` + +### Backwards compatibility + +The the [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) +and [`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules) +push rules are both deprecated. To avoid the unintentional mentions they are both +modified to only apply when the `mentions` field is missing. As this is for +backwards-compatibility it is not implemented using a generic mechanism, but +behavior specific to those push rules with those IDs. + +While this is being deployed there will be a mismatch for legacy clients which +implement the deprecated `.m.rule.contains_display_name` and `.m.rule.contains_user_name` +push rules locally while the `.m.rule.user_is_mentioned` push rule is used on +the homeserver; as messages which +[mention the user should also include the user's localpart](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) +in the message `body` it is likely that the `.m.rule.contains_user_name` +will also match. It is postulated that this will not cause issues in most cases. + +## Potential issues + +### Abuse potential + +This proposal makes it trivial to "hide" mentions since it does not require the +mentioned Matrix IDs to be part of the displayed text. This is not seen as +worse than what is possible today. + +From discussions and research while writing this MSC there are some benefits to +using a separate field for mentions: + +* The number of mentions is trivially limited. +* Various forms of "mention bombing" are no longer possible. +* It is simpler to collect data on how many users are being mentioned (without + having to process the textual `body` for every user's display name and local + part). + +Nonetheless, the conversations did result in some ideas to combat the potential +for abuse, many of which apply regardless of whether this proposal is implemented. + +Clients could expose *why* an event has caused a notification and give users inline +tools to combat potential abuse. For example, a client might show a tooltip: + +> The sender of the message (`@alice:example.org`) mentioned you in this event. +> +> Block `@alice:example.org` from sending you messages? `[Yes]` `[No]` + +It could also be worth exposing user interface for moderators to see messages +which mention many users. + +A future MSC might wish to explore features for trusted contacts or soft-ignores +to give users more control over who can generate notifications. + +Overall the proposal does not seem to increase the potential for malicious behavior. + +## Alternatives + +### Prior proposals + +There's a few prior proposals which tackle subsets of the above problem: + +* [MSC2463](https://github.com/matrix-org/matrix-spec-proposals/pull/2463): + excludes searching inside a Matrix ID for localparts / display names of other + users. +* [MSC3517](https://github.com/matrix-org/matrix-spec-proposals/pull/3517): + searches for Matrix IDs (instead of display name / localpart) and only searches + specific event types & fields. +* [Custom push rules](https://o.librepush.net/aux/matrix_reitools/pill_mention_rules.html) + to search for matrix.to links instead of display name / localpart. + + + + This generates a new push rule to replace `.m.rule.contains_display_name` and + `.m.rule.contains_user_name`: + + ```json + { + "conditions": [ + { + "kind": "event_match", + "key": "content.formatted_body", + "pattern": "*https://matrix.to/#/@alice:example.org*" + } + ], + "actions" : [ + "notify", + { + "set_tweak": "sound", + "value": "default" + }, + { + "set_tweak": "highlight" + } + ] + } + ``` + + + +### Room version for backwards compatibility + +Alternative backwards compatibility suggestions included using a new room version, +similar to [MSC3932](https://github.com/matrix-org/matrix-spec-proposals/pull/3932) +for extensible events. This does not seem like a good fit since room versions are +not usually interested in non-state events. It would additionally require a stable +room version before use, which would unnecessarily delay usage. + +## Security considerations + +None foreseen. + +## Unstable prefix + +During development the `.org.matrix.msc3952.is_user_mentioned` push rule will be +used. If a client sees this rule available it should apply the custom logic discussed +in the [backwards compatibility](#backwards-compatibility) section. + +## Dependencies + +N/A + +[^1]: This MSC does not attempt to solve this problem, but [MSC2781](https://github.com/matrix-org/matrix-spec-proposals/pull/2781) +proposes removing reply fallbacks which would solve it. Although as noted in +[MSC3676](https://github.com/matrix-org/matrix-spec-proposals/pull/3676) this +may require landing [MSC3664](https://github.com/matrix-org/matrix-doc/pull/3664) +in order to receive notifications of replies. + +[^2]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). + +[^3]: A new push condition is necessary since none of the current push conditions +(e.g. `event_match`) can process arrays. From 5d44598db67277fda5ef6397ecf06de0f87c8465 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Mon, 9 Jan 2023 14:50:44 -0500 Subject: [PATCH 02/55] Fix typo. Co-authored-by: NebulaOnion <84784521+NebulaOnion@users.noreply.github.com> --- proposals/3952-intentional-mentions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index a785e829e27..c732de47821 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -140,7 +140,7 @@ An example matching event is provided below: ### Backwards compatibility -The the [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) +The [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) and [`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules) push rules are both deprecated. To avoid the unintentional mentions they are both modified to only apply when the `mentions` field is missing. As this is for From a2896b2318812257971c854b306e3c870edc03e0 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 10 Jan 2023 11:00:53 -0500 Subject: [PATCH 03/55] Add a note about message edits not being fixed in this MSC. --- proposals/3952-intentional-mentions.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index c732de47821..23a41481720 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -11,7 +11,7 @@ Some situations that result in unintentional mentions include: * Replying to a message will re-issue pings from the initial message due to [fallback replies](https://spec.matrix.org/v1.5/client-server-api/#fallbacks-for-rich-replies). [^1] -* Each time a message is edited the new version will be re-evaluated for mentions. +* Each time a message is edited the new version will be re-evaluated for mentions. [^2] * Mentions occurring [in spoiler contents](https://github.com/matrix-org/matrix-spec/issues/16) or [code blocks](https://github.com/matrix-org/matrix-spec/issues/15) are evaluated. @@ -61,7 +61,7 @@ of the message should be colored differently: ## Proposal Instead of searching a message's `body` for the user's display name or the localpart -of their Matrix ID, it is proposed to use a field specific to mentions,[^2] and +of their Matrix ID, it is proposed to use a field specific to mentions,[^3] and augment the push rules to search for the current user's Matrix ID. ### New event field @@ -109,7 +109,7 @@ A new push rule condition and a new default push rule will be added: } ``` -The `is_mention` push condition is defined as[^3]: +The `is_mention` push condition is defined as[^4]: > This matches messages where the `content.mentions` is an array containing the > owner’s Matrix ID in the first 10 entries. This condition has no parameters. @@ -262,7 +262,10 @@ proposes removing reply fallbacks which would solve it. Although as noted in may require landing [MSC3664](https://github.com/matrix-org/matrix-doc/pull/3664) in order to receive notifications of replies. -[^2]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). +[^2]: Note that this MSC does not attempt to solve the problem of issues generating +spurious notifications. -[^3]: A new push condition is necessary since none of the current push conditions +[^3]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). + +[^4]: A new push condition is necessary since none of the current push conditions (e.g. `event_match`) can process arrays. From 99ab781f03bdea04cea34a418162d6968309cc13 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 10 Jan 2023 11:11:14 -0500 Subject: [PATCH 04/55] Note CC potential. --- proposals/3952-intentional-mentions.md | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 23a41481720..14593e86134 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -160,8 +160,9 @@ will also match. It is postulated that this will not cause issues in most cases. ### Abuse potential This proposal makes it trivial to "hide" mentions since it does not require the -mentioned Matrix IDs to be part of the displayed text. This is not seen as -worse than what is possible today. +mentioned Matrix IDs to be part of the displayed text. Depending on the use this +can be a feature (similar to CCing a user on an e-mail) or an abuse vector. Overall, +this MSC does not enable additional malicious behavior than what is possible today. From discussions and research while writing this MSC there are some benefits to using a separate field for mentions: @@ -182,8 +183,10 @@ tools to combat potential abuse. For example, a client might show a tooltip: > > Block `@alice:example.org` from sending you messages? `[Yes]` `[No]` -It could also be worth exposing user interface for moderators to see messages -which mention many users. +It could also be worth exposing user interface showing all of the mentions in a +message, especially if those users are not included in the message body. One way +to do this could be a deemphasized "CC" list. Additionally, it would be useful for +moderators to quickly find messages which mention many users. A future MSC might wish to explore features for trusted contacts or soft-ignores to give users more control over who can generate notifications. From 0ce2f60b881ff64b5e6e09bca03046a07593ac0a Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 10 Jan 2023 11:18:31 -0500 Subject: [PATCH 05/55] Note about display names via content rules. --- proposals/3952-intentional-mentions.md | 24 +++++++++++++++++++++++- 1 file changed, 23 insertions(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 14593e86134..112fa05811b 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -142,11 +142,33 @@ An example matching event is provided below: The [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) and [`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules) -push rules are both deprecated. To avoid the unintentional mentions they are both +push rules are both marked for removal. To avoid unintentional mentions both rules are modified to only apply when the `mentions` field is missing. As this is for backwards-compatibility it is not implemented using a generic mechanism, but behavior specific to those push rules with those IDs. +If users wish to continue to be notified of messages containing their display name +it is recommended that clients create a specific keyword rule for this, e.g. a +`content` rule of the form: + +```json +{ + "actions": [ + "notify", + { + "set_tweak": "sound", + "value": "default" + }, + { + "set_tweak": "highlight" + } + ], + "pattern": "alice", + "rule_id": "alice", + "enabled": true +} +``` + While this is being deployed there will be a mismatch for legacy clients which implement the deprecated `.m.rule.contains_display_name` and `.m.rule.contains_user_name` push rules locally while the `.m.rule.user_is_mentioned` push rule is used on From 521a5fd6ab4a0c310c61bc335a4b82226c78a3f5 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 11 Jan 2023 13:55:16 -0500 Subject: [PATCH 06/55] Updates from @daniellekirkwood's proof reading. --- proposals/3952-intentional-mentions.md | 68 +++++++++++++++++++------- 1 file changed, 49 insertions(+), 19 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 112fa05811b..aa30ba8d318 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -23,23 +23,25 @@ Some situations that result in unintentional mentions include: is replied to). As a sender you do not know if including the user's display name or Matrix ID would -even be interpreting as a mention (see [issue 353](https://github.com/matrix-org/matrix-spec/issues/353)). - -This also results in some unexpected behavior and bugs: +even be interpreted as a mention (see [issue 353](https://github.com/matrix-org/matrix-spec/issues/353)). +This results in some unexpected behavior and bugs: * Matrix users use "leetspeak" when sending messages to avoid mentions (e.g. referring to M4tthew instead of Matthew). -* Matrix users will append emoji or other unique text in their display names to - avoid unintentional pings. * It is impossible to ping one out of multiple people with the same localpart (or display name). * Since the relation between `body` and `formatted_body` is ill-defined and ["pills" are converted to display names](https://github.com/matrix-org/matrix-spec/issues/714), - this can result in missed messages. + this can result in missed messages. [^B] + +There is also some other related bugs: + +* Matrix users will append emoji or other unique text in their display names to + avoid unintentional pings. * Bridges [must use display names](https://github.com/matrix-org/matrix-spec/issues/353#issuecomment-1055809364) as a trick to get pings to work. -* If a user changes their display name in a room than whether or not they are - mentioned changes unless you use historical display names to process push rules. +* If a user changes their display name in a room, they might not be mentioned + unless the historical display name is used while processing push rules. (TODO I think there's an issue about this?) ## Background @@ -79,6 +81,10 @@ includes an intentional [user mention](https://spec.matrix.org/v1.5/client-serve (often called a "pill"). Clients may also add them at other times when it is obvious the user intends to explicitly mention a user. +If a user generates a message with more than 10 mentions, the client may wish to +show a warning message to the user; it may silently limit the number sent in the +message to 10 or force the user to remove some mentions. + The `mentions` field is part of the plaintext event body and should be encrypted into the ciphertext for encrypted events. @@ -169,13 +175,16 @@ it is recommended that clients create a specific keyword rule for this, e.g. a } ``` -While this is being deployed there will be a mismatch for legacy clients which -implement the deprecated `.m.rule.contains_display_name` and `.m.rule.contains_user_name` -push rules locally while the `.m.rule.user_is_mentioned` push rule is used on -the homeserver; as messages which -[mention the user should also include the user's localpart](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) -in the message `body` it is likely that the `.m.rule.contains_user_name` -will also match. It is postulated that this will not cause issues in most cases. +After acceptance, it is likely for different clients and homeservers to disagree +about which push rules are implemented: legacy clients and homeservers may not +yet have deprecated the `.m.rule.contains_display_name` and `.m.rule.contains_user_name` +push rules, while up-to-date clients and homeservers will support the +`.m.rule.user_is_mentioned` push rule. + +It is expected that this will usually take the form of some of the current behavior +(documented in the preamble) since most messages which contain a mention will also +[contain the user's display name](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) +in the message `body` and match the `.m.rule.contains_user_name` push rule. ## Potential issues @@ -215,12 +224,23 @@ to give users more control over who can generate notifications. Overall the proposal does not seem to increase the potential for malicious behavior. +## Future extensions + +It maybe desirable for room administrators to configure the number of allowed +mentions in a message, e.g. a conference may want to mention 50 people at once +or it may be appropriate for a kudos room to mention the 15 people on your team. +Since it is easy enough to work around the limit of 10 mentions in socially +appropriate situations (by sending multiple messages) it does not seem worth +the technical complexity of allowing this number to be configurable. + ## Alternatives ### Prior proposals There's a few prior proposals which tackle subsets of the above problem: +* [MSC1796](https://github.com/matrix-org/matrix-spec-proposals/pull/1796): + extremely similar to the proposal in this MSC, but limited to encrypted events. * [MSC2463](https://github.com/matrix-org/matrix-spec-proposals/pull/2463): excludes searching inside a Matrix ID for localparts / display names of other users. @@ -230,10 +250,10 @@ There's a few prior proposals which tackle subsets of the above problem: * [Custom push rules](https://o.librepush.net/aux/matrix_reitools/pill_mention_rules.html) to search for matrix.to links instead of display name / localpart. - +
- This generates a new push rule to replace `.m.rule.contains_display_name` and - `.m.rule.contains_user_name`: + The above generates a new push rule to replace `.m.rule.contains_display_name` + and `.m.rule.contains_user_name`: ```json { @@ -257,7 +277,7 @@ There's a few prior proposals which tackle subsets of the above problem: } ``` -
+ ### Room version for backwards compatibility @@ -290,6 +310,16 @@ in order to receive notifications of replies. [^2]: Note that this MSC does not attempt to solve the problem of issues generating spurious notifications. +[^B]: The `body` is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes): + +> When [the `format` field is set to `org.matrix.custom.html`], a formatted_body +> with the HTML must be provided. The plain text version of the HTML should be +> provided in the `body`. + +But there is no particular algorithm to convert from HTML to plain text *except* +when converting mentions, where the +[plain text version uses the text, not the link](https://spec.matrix.org/v1.5/client-server-api/#client-behaviour-26). + [^3]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). [^4]: A new push condition is necessary since none of the current push conditions From 4de784893e53995991b18f44329ca074b54547ae Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 11 Jan 2023 14:26:48 -0500 Subject: [PATCH 07/55] Add information on cleartext mentions. --- proposals/3952-intentional-mentions.md | 48 ++++++++++++++++++++++++-- 1 file changed, 45 insertions(+), 3 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index aa30ba8d318..7f23e625631 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -85,8 +85,11 @@ If a user generates a message with more than 10 mentions, the client may wish to show a warning message to the user; it may silently limit the number sent in the message to 10 or force the user to remove some mentions. -The `mentions` field is part of the plaintext event body and should be encrypted -into the ciphertext for encrypted events. +The `mentions` field is part of the cleartext event body and should **not** be +encrypted into the ciphertext for encrypted events. This expoes slightly more +metadata to homeservers, but allows mentions to work properly via push notifications +(which is a requirement for a usable chat application) and should result in bandwidth +and battery savings. ### New push rules @@ -226,6 +229,8 @@ Overall the proposal does not seem to increase the potential for malicious behav ## Future extensions +### Configurable mentions limits + It maybe desirable for room administrators to configure the number of allowed mentions in a message, e.g. a conference may want to mention 50 people at once or it may be appropriate for a kudos room to mention the 15 people on your team. @@ -233,6 +238,14 @@ Since it is easy enough to work around the limit of 10 mentions in socially appropriate situations (by sending multiple messages) it does not seem worth the technical complexity of allowing this number to be configurable. +### Muted except for mentions push rules + +By leaving the mentions array unencrypted it allows for a "muted-except-for-mentions" +push rule. This is particularly useful for large (encrypted) rooms where you would +like to receive push notifications for mentions *only*. It is imperative for +the homeserver to be able to send a push in this case since some mobile platforms, +e.g. iOS, do not sync regularly in the background. + ## Alternatives ### Prior proposals @@ -287,9 +300,38 @@ for extensible events. This does not seem like a good fit since room versions ar not usually interested in non-state events. It would additionally require a stable room version before use, which would unnecessarily delay usage. +### Encrypting the `mentions` array + +Encrypting the `mentions` array would solve some unintentional mentions, but +will not help with???. + +Allowing an encrypted `mentions` array on a per-message basis could allow users +to choose, but would result in inconsistencies and complications: + +* What happens if the cleartext `mentions` field does not match the plaintext + `mentions` field? +* Do push rules get processed on both the cleartext and plaintext message? + +It may be argued that clients need to decrypt all messages anyway to handle +user-specific keywords, the process for doing this is costly (either receiving +every message via push notifications or backpaginating every room fully, in case +of a gappy sync) in terms of bandwidth, CPU, and battery. + +It is asserted that the use-cases for custom keywords and mentions are sufficiently +different that having different solutions, and different urgencies to receiving +those notifications, is reasonable. In particular, custom keywords is more of a +power-user feature while user mentions working properly (and promptly) is a table-stakes +feature for a messaging application. + ## Security considerations -None foreseen. +Including the mentioned users in cleartext is a small metadata leak, but the tradeoff +of mentions working properly and the savings in bandwidth and battery makes it worthwhile. + +The additional metadata leaked is minor since the homeserver already knows the +room members, how often users are sending messages, when users are receiving messages +(via read receipts, read markers, and sync), etc. Exposing explicit mentions does not +significantly increase metadata leakage. ## Unstable prefix From 5dfc0bce5793f761073ec1da12eb66fa0f4508ba Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 11 Jan 2023 14:50:22 -0500 Subject: [PATCH 08/55] Info on @room. --- proposals/3952-intentional-mentions.md | 99 +++++++++++++++++++------- 1 file changed, 73 insertions(+), 26 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 7f23e625631..fb599e8ab40 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -69,16 +69,18 @@ augment the push rules to search for the current user's Matrix ID. ### New event field A new `mentions` field is added to the event content, it is an array of strings -consistent of Matrix IDs. +consistent of Matrix IDs or the special value: `"@room"`. To limit the potential for abuse, only the first 10 items in the array should be considered. It is recommended that homeservers reject locally created events with more than 10 mentions with an error with a status code of `400` and an errcode of -`M_INVALID_PARAM`. +`M_INVALID_PARAM`. It is valid to include specific users in addition to the +`"@room"` value. Clients should add a Matrix ID to this array whenever composing a message which includes an intentional [user mention](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) -(often called a "pill"). Clients may also add them at other times when it is +(often called a "pill"). Clients should add the `"@room"` value when making a +room-wide announcement. Clients may also add them at other times when it is obvious the user intends to explicitly mention a user. If a user generates a message with more than 10 mentions, the client may wish to @@ -93,16 +95,33 @@ and battery savings. ### New push rules -A new push rule condition and a new default push rule will be added: +Two new push rule conditions and corresponding default push rule are proposed: + +The `is_user_mention` push condition is defined as[^4]: + +> This matches messages where the `content.mentions` is an array containing the +> owner’s Matrix ID in the first 10 entries. This condition has no parameters. +> If the `mentions` key is absent or not an array then the rule does not match; +> any array entries which are not strings are ignored, but count against the limit. + +The `is_room_mention` push condition is defined as[^4]: + +> This matches messages where the `content.mentions` is an array containing the +> string `"@room"` in the first 10 entries. This condition has no parameters. +> If the `mentions` key is absent or not an array then the rule does not match; +> any array entries which are not strings are ignored, but count against the limit. + +The proposed `.m.rule.is_user_mention` override push rule would appear directly +before the `.m.rule.contains_display_name` push rule: ```json { - "rule_id": ".m.rule.user_is_mentioned", + "rule_id": ".m.rule.is_user_mention", "default": true, "enabled": true, "conditions": [ { - "kind": "is_mention" + "kind": "is_user_mention" } ], "actions": [ @@ -118,14 +137,38 @@ A new push rule condition and a new default push rule will be added: } ``` -The `is_mention` push condition is defined as[^4]: +The proposed `.m.rule.is_room_mention` override push rule would appear directly +before the `.m.rule.roomnotif` push rule: -> This matches messages where the `content.mentions` is an array containing the -> owner’s Matrix ID in the first 10 entries. This condition has no parameters. -> If the `mentions` key is absent or not an array then the rule does not match; -> any array entries which are not strings are ignored, but count against the limit. +```json +{ + "rule_id": ".m.rule.is_room_mention", + "default": true, + "enabled": true, + "conditions": [ + { + "kind": "is_room_mention" + }, + { + "kind": "sender_notification_permission", + "key": "room" + } + ], + "actions": [ + "notify", + { + "set_tweak": "sound", + "value": "default" + }, + { + "set_tweak": "highlight" + } + ] +} +``` -An example matching event is provided below: +An example event matching both `.m.rule.is_user_mention` (for `@alice:example.org`) +and `.m.rule.is_room_mention` is provided below: ```json { @@ -134,7 +177,7 @@ An example matching event is provided below: "format": "org.matrix.custom.html", "formatted_body": "This is an example mention Alice", "msgtype": "m.text", - "mentions": ["@alice:example.org"] + "mentions": ["@alice:example.org", "@room"] }, "event_id": "$143273582443PhrSn:example.org", "origin_server_ts": 1432735824653, @@ -149,12 +192,13 @@ An example matching event is provided below: ### Backwards compatibility -The [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) -and [`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules) -push rules are both marked for removal. To avoid unintentional mentions both rules are +The [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules), +[`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules), +and [`.m.rule.roomnotif`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) +push rules are to be removed. To avoid unintentional mentions these rules are modified to only apply when the `mentions` field is missing. As this is for backwards-compatibility it is not implemented using a generic mechanism, but -behavior specific to those push rules with those IDs. +behavior specific to these push rules. If users wish to continue to be notified of messages containing their display name it is recommended that clients create a specific keyword rule for this, e.g. a @@ -178,16 +222,13 @@ it is recommended that clients create a specific keyword rule for this, e.g. a } ``` -After acceptance, it is likely for different clients and homeservers to disagree -about which push rules are implemented: legacy clients and homeservers may not -yet have deprecated the `.m.rule.contains_display_name` and `.m.rule.contains_user_name` +After acceptance, it is likely for some disagreement about which push rules are +implemented: legacy clients and homeservers may not yet have deprecated the +`.m.rule.contains_display_name`, `.m.rule.contains_user_name`, and `.m.rule.roomnotif` push rules, while up-to-date clients and homeservers will support the -`.m.rule.user_is_mentioned` push rule. - -It is expected that this will usually take the form of some of the current behavior -(documented in the preamble) since most messages which contain a mention will also -[contain the user's display name](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) -in the message `body` and match the `.m.rule.contains_user_name` push rule. +`.m.rule.is_user_mention` and `.m.rule.is_room_mention` push rules. It is expected +that both sets of push rules will need to be supported for a period of time, but +at worst case should simply result in the current behavior (documented in the preamble). ## Potential issues @@ -246,6 +287,12 @@ like to receive push notifications for mentions *only*. It is imperative for the homeserver to be able to send a push in this case since some mobile platforms, e.g. iOS, do not sync regularly in the background. +### Pillifying `@room` + +Some clients attempt to create a "pill" out of `@room` mentions, but this is not +a requirement of the Matrix specification. The current [user and rooms mentions](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) +section could be expanded for this situation. + ## Alternatives ### Prior proposals From e809231133043998c9f8aa4ac58857b5b59254b8 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 11 Jan 2023 15:04:15 -0500 Subject: [PATCH 09/55] Some more unstable / backwards compatibility. --- proposals/3952-intentional-mentions.md | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index fb599e8ab40..aee415312d9 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -195,10 +195,11 @@ and `.m.rule.is_room_mention` is provided below: The [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules), [`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules), and [`.m.rule.roomnotif`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) -push rules are to be removed. To avoid unintentional mentions these rules are -modified to only apply when the `mentions` field is missing. As this is for -backwards-compatibility it is not implemented using a generic mechanism, but -behavior specific to these push rules. +push rules are to be removed. + +To avoid unintentional mentions these rules are modified to only apply when the +`mentions` field is missing and clients should provide the `mentions` field on +every message to avoid the unintentional mentions discussed above. If users wish to continue to be notified of messages containing their display name it is recommended that clients create a specific keyword rule for this, e.g. a @@ -382,10 +383,21 @@ significantly increase metadata leakage. ## Unstable prefix -During development the `.org.matrix.msc3952.is_user_mentioned` push rule will be -used. If a client sees this rule available it should apply the custom logic discussed +During development the following mapping will be used: + +| What | Stable | Unstable | +|---------------------|---------------------------|---------------------------------------| +| Push rule ID | `.m.rule.*` | `.org.matrix.msc3952.*` | +| Push condition kind | `is_user_mention` | `org.matrix.msc3952.is_user_mention` | +| Push condition kind | `is_room_mention` | `org.matrix.msc3952.is_room_mention` | + + +If a client sees this rule available it can choose to apply the custom logic discussed in the [backwards compatibility](#backwards-compatibility) section. +If a client sees the *stable* identifiers available, they should apply the new +logic and start creating events with the `mentions` array. + ## Dependencies N/A From c4516ce5113307002799220592ffb407aba38297 Mon Sep 17 00:00:00 2001 From: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> Date: Sat, 21 Jan 2023 14:31:42 +0000 Subject: [PATCH 10/55] fix a couple typos --- proposals/3952-intentional-mentions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index aee415312d9..d8d1b7a1621 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -69,7 +69,7 @@ augment the push rules to search for the current user's Matrix ID. ### New event field A new `mentions` field is added to the event content, it is an array of strings -consistent of Matrix IDs or the special value: `"@room"`. +consisting of Matrix IDs or the special value: `"@room"`. To limit the potential for abuse, only the first 10 items in the array should be considered. It is recommended that homeservers reject locally created events with @@ -88,7 +88,7 @@ show a warning message to the user; it may silently limit the number sent in the message to 10 or force the user to remove some mentions. The `mentions` field is part of the cleartext event body and should **not** be -encrypted into the ciphertext for encrypted events. This expoes slightly more +encrypted into the ciphertext for encrypted events. This exposes slightly more metadata to homeservers, but allows mentions to work properly via push notifications (which is a requirement for a usable chat application) and should result in bandwidth and battery savings. From 3deebe776e1f098d721fe62a0112e163240ec362 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 11 Jan 2023 15:09:39 -0500 Subject: [PATCH 11/55] Update footnotes. --- proposals/3952-intentional-mentions.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index d8d1b7a1621..8cc14254285 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -32,7 +32,7 @@ This results in some unexpected behavior and bugs: (or display name). * Since the relation between `body` and `formatted_body` is ill-defined and ["pills" are converted to display names](https://github.com/matrix-org/matrix-spec/issues/714), - this can result in missed messages. [^B] + this can result in missed messages. [^3] There is also some other related bugs: @@ -63,7 +63,7 @@ of the message should be colored differently: ## Proposal Instead of searching a message's `body` for the user's display name or the localpart -of their Matrix ID, it is proposed to use a field specific to mentions,[^3] and +of their Matrix ID, it is proposed to use a field specific to mentions,[^4] and augment the push rules to search for the current user's Matrix ID. ### New event field @@ -97,14 +97,14 @@ and battery savings. Two new push rule conditions and corresponding default push rule are proposed: -The `is_user_mention` push condition is defined as[^4]: +The `is_user_mention` push condition is defined as[^5]: > This matches messages where the `content.mentions` is an array containing the > owner’s Matrix ID in the first 10 entries. This condition has no parameters. > If the `mentions` key is absent or not an array then the rule does not match; > any array entries which are not strings are ignored, but count against the limit. -The `is_room_mention` push condition is defined as[^4]: +The `is_room_mention` push condition is defined as[^5]: > This matches messages where the `content.mentions` is an array containing the > string `"@room"` in the first 10 entries. This condition has no parameters. @@ -411,7 +411,7 @@ in order to receive notifications of replies. [^2]: Note that this MSC does not attempt to solve the problem of issues generating spurious notifications. -[^B]: The `body` is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes): +[^3]: The `body` is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes): > When [the `format` field is set to `org.matrix.custom.html`], a formatted_body > with the HTML must be provided. The plain text version of the HTML should be @@ -421,7 +421,7 @@ But there is no particular algorithm to convert from HTML to plain text *except* when converting mentions, where the [plain text version uses the text, not the link](https://spec.matrix.org/v1.5/client-server-api/#client-behaviour-26). -[^3]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). +[^4]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). -[^4]: A new push condition is necessary since none of the current push conditions +[^5]: A new push condition is necessary since none of the current push conditions (e.g. `event_match`) can process arrays. From 8a3c3c8b15165bde5956e49702df84e2fdb525ff Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 24 Jan 2023 11:35:31 -0500 Subject: [PATCH 12/55] Updates from reviews. --- proposals/3952-intentional-mentions.md | 148 ++++++++++++++++--------- 1 file changed, 94 insertions(+), 54 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 8cc14254285..2ac864a5c92 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -68,30 +68,39 @@ augment the push rules to search for the current user's Matrix ID. ### New event field -A new `mentions` field is added to the event content, it is an array of strings -consisting of Matrix IDs or the special value: `"@room"`. +A new `m.mentions` field is added to the event content, it is an object with two +optional fields: + +* `user_ids`: an array of strings consisting of Matrix IDs to mention. +* `room`: a boolean, true indicates an "@room" mention. Any other value or the + field missing is interpreted as not an "@room" mention. + +It is valid to include both the `user_ids` and `room` fields. To limit the potential for abuse, only the first 10 items in the array should be considered. It is recommended that homeservers reject locally created events with more than 10 mentions with an error with a status code of `400` and an errcode of -`M_INVALID_PARAM`. It is valid to include specific users in addition to the -`"@room"` value. +`M_INVALID_PARAM`. -Clients should add a Matrix ID to this array whenever composing a message which +Clients should add a Matrix ID to the `user_ids` array whenever composing a message which includes an intentional [user mention](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) -(often called a "pill"). Clients should add the `"@room"` value when making a -room-wide announcement. Clients may also add them at other times when it is +(often called a "pill"). Clients should set the `room` value to `true` when making a +room-wide announcement. Clients may also set these values at other times when it is obvious the user intends to explicitly mention a user. If a user generates a message with more than 10 mentions, the client may wish to show a warning message to the user; it may silently limit the number sent in the message to 10 or force the user to remove some mentions. -The `mentions` field is part of the cleartext event body and should **not** be +The `m.mentions` field is part of the cleartext event body and should **not** be encrypted into the ciphertext for encrypted events. This exposes slightly more -metadata to homeservers, but allows mentions to work properly via push notifications -(which is a requirement for a usable chat application) and should result in bandwidth -and battery savings. +metadata to homeservers, but allows mentions to work properly. It allows the +server to properly handle push notifications (which is a requirement for a usable +chat application) and could result in bandwidth and battery savings (see the +[future extensions](#muted-except-for-mentions-push-rules) section). Additionally, +it may be useful for the homeserver to filter or prioritize rooms based on whether +the user has been mentioned in them, e.g. for an extension to +[MSC3575 (sliding sync)](https://github.com/matrix-org/matrix-spec-proposals/pull/3575). ### New push rules @@ -99,17 +108,19 @@ Two new push rule conditions and corresponding default push rule are proposed: The `is_user_mention` push condition is defined as[^5]: -> This matches messages where the `content.mentions` is an array containing the -> owner’s Matrix ID in the first 10 entries. This condition has no parameters. -> If the `mentions` key is absent or not an array then the rule does not match; -> any array entries which are not strings are ignored, but count against the limit. +> This matches messages where the `m.mentions` field of the `content` contains a +> field `user_ids` which is an array containing the owner’s Matrix ID in the first +> 10 entries. This condition has no parameters. If the `m.mentions` key is absent +> or not an object or does not contain a `user_ids` field (or contains it, but it +> is not an array) then the rule does not match; any array entries which are not +> strings are ignored, but count against the limit. Duplicate entries are ignored. The `is_room_mention` push condition is defined as[^5]: -> This matches messages where the `content.mentions` is an array containing the -> string `"@room"` in the first 10 entries. This condition has no parameters. -> If the `mentions` key is absent or not an array then the rule does not match; -> any array entries which are not strings are ignored, but count against the limit. +> This matches messages where the `m.mentions` field of the `content` contains +> a field `room` set to `true`. This condition has no parameters. If the +> `m.mentions` key is absent or not an object or does not contain a `room` field +> (or contains it, but it is not an boolean value) then the rule does not match. The proposed `.m.rule.is_user_mention` override push rule would appear directly before the `.m.rule.contains_display_name` push rule: @@ -177,7 +188,10 @@ and `.m.rule.is_room_mention` is provided below: "format": "org.matrix.custom.html", "formatted_body": "This is an example mention Alice", "msgtype": "m.text", - "mentions": ["@alice:example.org", "@room"] + "m.mentions": { + "user_ids": ["@alice:example.org"], + "room": true + } }, "event_id": "$143273582443PhrSn:example.org", "origin_server_ts": 1432735824653, @@ -198,7 +212,7 @@ and [`.m.rule.roomnotif`](https://spec.matrix.org/v1.5/client-server-api/#defaul push rules are to be removed. To avoid unintentional mentions these rules are modified to only apply when the -`mentions` field is missing and clients should provide the `mentions` field on +`m.mentions` field is missing; clients should provide the `m.mentions` field on every message to avoid the unintentional mentions discussed above. If users wish to continue to be notified of messages containing their display name @@ -223,7 +237,7 @@ it is recommended that clients create a specific keyword rule for this, e.g. a } ``` -After acceptance, it is likely for some disagreement about which push rules are +After acceptance, it is likely for there to be disagreement about which push rules are implemented: legacy clients and homeservers may not yet have deprecated the `.m.rule.contains_display_name`, `.m.rule.contains_user_name`, and `.m.rule.roomnotif` push rules, while up-to-date clients and homeservers will support the @@ -236,41 +250,45 @@ at worst case should simply result in the current behavior (documented in the pr ### Abuse potential This proposal makes it trivial to "hide" mentions since it does not require the -mentioned Matrix IDs to be part of the displayed text. Depending on the use this -can be a feature (similar to CCing a user on an e-mail) or an abuse vector. Overall, -this MSC does not enable additional malicious behavior than what is possible today. +mentioned Matrix IDs to be part of the displayed text. This is only a limitation +for current clients: mentions could be exposed in the user interface directly. +For example, a de-emphasized "notified" list could be shown on messages, similar +to CCing users on an e-mail. -From discussions and research while writing this MSC there are some benefits to -using a separate field for mentions: +Although not including mentions in the displayed text could be used as an abuse +vector, it does not enable additional malicious behavior than what is possible +today. From discussions and research while writing this MSC there are moderation +benefits to using a separate field for mentions: -* The number of mentions is trivially limited. +* The number of mentions is trivially limited by moderation tooling. * Various forms of "mention bombing" are no longer possible. -* It is simpler to collect data on how many users are being mentioned (without - having to process the textual `body` for every user's display name and local +* It is simpler to collect metrics on how mentions are being used (it is no longer + necessary to process the textual `body` for every user's display name and local part). -Nonetheless, the conversations did result in some ideas to combat the potential -for abuse, many of which apply regardless of whether this proposal is implemented. +Overall this proposal seems to be neutral or positive in the ability to combat +malicious behavior. + +## Future extensions -Clients could expose *why* an event has caused a notification and give users inline -tools to combat potential abuse. For example, a client might show a tooltip: +### Combating abuse + +Some ideas for combating abuse came from our discussion and research which are +worth sharing. These ideas are not part of this MSC, but also do not depend on it. + +It was recommended that clients could expose *why* an event has caused a notification +and give users inline tools to combat abuse. For example, a client might show a tooltip: > The sender of the message (`@alice:example.org`) mentioned you in this event. > > Block `@alice:example.org` from sending you messages? `[Yes]` `[No]` -It could also be worth exposing user interface showing all of the mentions in a -message, especially if those users are not included in the message body. One way -to do this could be a deemphasized "CC" list. Additionally, it would be useful for -moderators to quickly find messages which mention many users. +Additionally, tooling to for moderators to quickly find messages which mention +many users would be useful. A future MSC might wish to explore features for trusted contacts or soft-ignores to give users more control over who can generate notifications. -Overall the proposal does not seem to increase the potential for malicious behavior. - -## Future extensions - ### Configurable mentions limits It maybe desirable for room administrators to configure the number of allowed @@ -294,6 +312,18 @@ Some clients attempt to create a "pill" out of `@room` mentions, but this is not a requirement of the Matrix specification. The current [user and rooms mentions](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) section could be expanded for this situation. +### Extensible events + +The `m.mentions` field can be considered a "mixin" as part of extensible events +([MSC1767](https://github.com/matrix-org/matrix-doc/pull/1767)) with no needed +changes. + +### Role mentions + +It is possible to add additional fields to the `m.mentions` object, e.g. a foreseeable +usecase would be a `roles` field which could include values such as `admins` or +`mods`. Defining this in detail is left to a future MSC. + ## Alternatives ### Prior proposals @@ -340,6 +370,16 @@ There's a few prior proposals which tackle subsets of the above problem: +The last two proposals use a similar idea of attempting to find "pills" in the +`formatted_body`, this has some downsides though: + +* It doesn't allow for hidden mentions, which can be useful in some situations. +* It does not work for event types other than `m.room.message`. + +It also adds significant implementation complexity since the HTML messages must +now be parsed for notifications. This is expensive and introduces potential +security issues. + ### Room version for backwards compatibility Alternative backwards compatibility suggestions included using a new room version, @@ -348,16 +388,16 @@ for extensible events. This does not seem like a good fit since room versions ar not usually interested in non-state events. It would additionally require a stable room version before use, which would unnecessarily delay usage. -### Encrypting the `mentions` array +### Encrypting the `m.mentions` field -Encrypting the `mentions` array would solve some unintentional mentions, but +Encrypting the `m.mentions` field would solve some unintentional mentions, but will not help with???. -Allowing an encrypted `mentions` array on a per-message basis could allow users +Allowing an encrypted `m.mentions` field on a per-message basis could allow users to choose, but would result in inconsistencies and complications: -* What happens if the cleartext `mentions` field does not match the plaintext - `mentions` field? +* What happens if the cleartext `m.mentions` field does not match the plaintext + `m.mentions` field? * Do push rules get processed on both the cleartext and plaintext message? It may be argued that clients need to decrypt all messages anyway to handle @@ -385,18 +425,18 @@ significantly increase metadata leakage. During development the following mapping will be used: -| What | Stable | Unstable | -|---------------------|---------------------------|---------------------------------------| -| Push rule ID | `.m.rule.*` | `.org.matrix.msc3952.*` | -| Push condition kind | `is_user_mention` | `org.matrix.msc3952.is_user_mention` | -| Push condition kind | `is_room_mention` | `org.matrix.msc3952.is_room_mention` | - +| What | Stable | Unstable | +|---------------------|-------------------|--------------------------------------| +| Event field | `m.mentions` | `org.matrix.msc3952.mentions` | +| Push rule ID | `.m.rule.*` | `.org.matrix.msc3952.*` | +| Push condition kind | `is_user_mention` | `org.matrix.msc3952.is_user_mention` | +| Push condition kind | `is_room_mention` | `org.matrix.msc3952.is_room_mention` | If a client sees this rule available it can choose to apply the custom logic discussed in the [backwards compatibility](#backwards-compatibility) section. If a client sees the *stable* identifiers available, they should apply the new -logic and start creating events with the `mentions` array. +logic and start creating events with the `m.mentions` field. ## Dependencies From 415c70b13217f4b28776a7159c3695e10be2712c Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 24 Jan 2023 11:45:25 -0500 Subject: [PATCH 13/55] Remove protocol limitation on the number of mentions. --- proposals/3952-intentional-mentions.md | 38 +++++++++----------------- 1 file changed, 13 insertions(+), 25 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 2ac864a5c92..98d774495c4 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -77,9 +77,8 @@ optional fields: It is valid to include both the `user_ids` and `room` fields. -To limit the potential for abuse, only the first 10 items in the array should be -considered. It is recommended that homeservers reject locally created events with -more than 10 mentions with an error with a status code of `400` and an errcode of +It is recommended that homeservers reject locally created events with an invalid +`m.mentions` field with an error with a status code of `400` and an errcode of `M_INVALID_PARAM`. Clients should add a Matrix ID to the `user_ids` array whenever composing a message which @@ -88,10 +87,6 @@ includes an intentional [user mention](https://spec.matrix.org/v1.5/client-serve room-wide announcement. Clients may also set these values at other times when it is obvious the user intends to explicitly mention a user. -If a user generates a message with more than 10 mentions, the client may wish to -show a warning message to the user; it may silently limit the number sent in the -message to 10 or force the user to remove some mentions. - The `m.mentions` field is part of the cleartext event body and should **not** be encrypted into the ciphertext for encrypted events. This exposes slightly more metadata to homeservers, but allows mentions to work properly. It allows the @@ -109,18 +104,18 @@ Two new push rule conditions and corresponding default push rule are proposed: The `is_user_mention` push condition is defined as[^5]: > This matches messages where the `m.mentions` field of the `content` contains a -> field `user_ids` which is an array containing the owner’s Matrix ID in the first -> 10 entries. This condition has no parameters. If the `m.mentions` key is absent -> or not an object or does not contain a `user_ids` field (or contains it, but it -> is not an array) then the rule does not match; any array entries which are not -> strings are ignored, but count against the limit. Duplicate entries are ignored. +> field `user_ids` which is an array containing the owner’s Matrix ID. This +> condition has no parameters. If the `m.mentions` key is absent; not an object; +> does not contain a `user_ids` field; or contains a non-array `user_ids` field +> then the rule does not match. Any non-string array entries are ignored and +> duplicate entries are ignored. The `is_room_mention` push condition is defined as[^5]: > This matches messages where the `m.mentions` field of the `content` contains -> a field `room` set to `true`. This condition has no parameters. If the -> `m.mentions` key is absent or not an object or does not contain a `room` field -> (or contains it, but it is not an boolean value) then the rule does not match. +> a `room` field set to `true`. This condition has no parameters. If the +> `m.mentions` key is absent; not an object; does not contain a `room` field; +> or contains a non-boolean `room` field then the rule does not match. The proposed `.m.rule.is_user_mention` override push rule would appear directly before the `.m.rule.contains_display_name` push rule: @@ -260,7 +255,9 @@ vector, it does not enable additional malicious behavior than what is possible today. From discussions and research while writing this MSC there are moderation benefits to using a separate field for mentions: -* The number of mentions is trivially limited by moderation tooling. +* The number of mentions is trivially limited by moderation tooling, e.g. it may + be appropriate for a community room to only allow 10 mentions. Events not abiding + by this could be rejected automatically (or users could be banned automatically). * Various forms of "mention bombing" are no longer possible. * It is simpler to collect metrics on how mentions are being used (it is no longer necessary to process the textual `body` for every user's display name and local @@ -289,15 +286,6 @@ many users would be useful. A future MSC might wish to explore features for trusted contacts or soft-ignores to give users more control over who can generate notifications. -### Configurable mentions limits - -It maybe desirable for room administrators to configure the number of allowed -mentions in a message, e.g. a conference may want to mention 50 people at once -or it may be appropriate for a kudos room to mention the 15 people on your team. -Since it is easy enough to work around the limit of 10 mentions in socially -appropriate situations (by sending multiple messages) it does not seem worth -the technical complexity of allowing this number to be configurable. - ### Muted except for mentions push rules By leaving the mentions array unencrypted it allows for a "muted-except-for-mentions" From 3ef87c7f5fbb08e2b3e50944ae37de37a31d067e Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 24 Jan 2023 14:40:53 -0500 Subject: [PATCH 14/55] Clarify footnote. Co-authored-by: Brad Murray --- proposals/3952-intentional-mentions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 98d774495c4..32b9c220347 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -436,7 +436,7 @@ proposes removing reply fallbacks which would solve it. Although as noted in may require landing [MSC3664](https://github.com/matrix-org/matrix-doc/pull/3664) in order to receive notifications of replies. -[^2]: Note that this MSC does not attempt to solve the problem of issues generating +[^2]: Note that this MSC does not attempt to solve the problem of edits generating spurious notifications. [^3]: The `body` is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes): From f2f4b7a7db644ed3b43751105219fc5d7ecca2f8 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 24 Jan 2023 14:44:14 -0500 Subject: [PATCH 15/55] Fix footnote formatting. --- proposals/3952-intentional-mentions.md | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 32b9c220347..784995e6dd1 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -439,15 +439,11 @@ in order to receive notifications of replies. [^2]: Note that this MSC does not attempt to solve the problem of edits generating spurious notifications. -[^3]: The `body` is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes): - -> When [the `format` field is set to `org.matrix.custom.html`], a formatted_body -> with the HTML must be provided. The plain text version of the HTML should be -> provided in the `body`. - -But there is no particular algorithm to convert from HTML to plain text *except* +[^3]: It is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes) +the the "plain text version of the HTML [`formatted_body`] should be provided in the `body`", +but there is no particular algorithm to convert from HTML to plain text *except* when converting mentions, where the -[plain text version uses the text, not the link](https://spec.matrix.org/v1.5/client-server-api/#client-behaviour-26). +[plain text version uses the link anchor, not the link](https://spec.matrix.org/v1.5/client-server-api/#client-behaviour-26). [^4]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). From 0ccdff2c9bdf5f4b3a773456f4170f8cfcdc8892 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 27 Jan 2023 08:47:44 -0500 Subject: [PATCH 16/55] Add another alternative. --- proposals/3952-intentional-mentions.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 784995e6dd1..9813b824cab 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -399,6 +399,22 @@ those notifications, is reasonable. In particular, custom keywords is more of a power-user feature while user mentions working properly (and promptly) is a table-stakes feature for a messaging application. +### For the `is_room_mention` condition + +The `is_room_mention` could be replaced with the `exact_event_match` condition +from [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758) +(or the changes to `event_match` in [MSC3862](https://github.com/matrix-org/matrix-spec-proposals/pull/3862)), +although either of those would potentially run into ambiguity in the `key`, +as described in [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): + +```json +{ + "kind": "exact_event_match", + "key": "m.mentions.room", + "value": true +} +``` + ## Security considerations Including the mentioned users in cleartext is a small metadata leak, but the tradeoff From 4d683141a412b6505741fc6163a32dac48592f1b Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 31 Jan 2023 12:53:16 -0500 Subject: [PATCH 17/55] Fix typos. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3952-intentional-mentions.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 9813b824cab..4a6385be9b0 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -34,7 +34,7 @@ This results in some unexpected behavior and bugs: ["pills" are converted to display names](https://github.com/matrix-org/matrix-spec/issues/714), this can result in missed messages. [^3] -There is also some other related bugs: +There are also some other related bugs: * Matrix users will append emoji or other unique text in their display names to avoid unintentional pings. @@ -68,7 +68,7 @@ augment the push rules to search for the current user's Matrix ID. ### New event field -A new `m.mentions` field is added to the event content, it is an object with two +A new `m.mentions` field is added to the event content; it is an object with two optional fields: * `user_ids`: an array of strings consisting of Matrix IDs to mention. @@ -316,7 +316,7 @@ usecase would be a `roles` field which could include values such as `admins` or ### Prior proposals -There's a few prior proposals which tackle subsets of the above problem: +There are a few prior proposals which tackle subsets of the above problem: * [MSC1796](https://github.com/matrix-org/matrix-spec-proposals/pull/1796): extremely similar to the proposal in this MSC, but limited to encrypted events. @@ -417,7 +417,7 @@ as described in [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pu ## Security considerations -Including the mentioned users in cleartext is a small metadata leak, but the tradeoff +Including the mentioned users in cleartext is a small metadata leak, but we consider that the tradeoff of mentions working properly and the savings in bandwidth and battery makes it worthwhile. The additional metadata leaked is minor since the homeserver already knows the From 5bfebcf8ac67287c6e4a0b0460d43ea9feadf502 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 31 Jan 2023 12:54:56 -0500 Subject: [PATCH 18/55] Correct example. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3952-intentional-mentions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 4a6385be9b0..7c3c0476df6 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -410,7 +410,7 @@ as described in [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pu ```json { "kind": "exact_event_match", - "key": "m.mentions.room", + "key": "content.m.mentions.room", "value": true } ``` From bd215e01f33f582c416ee12661c1faa5906c73d2 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 31 Jan 2023 12:59:29 -0500 Subject: [PATCH 19/55] s/field/property/g --- proposals/3952-intentional-mentions.md | 58 +++++++++++++------------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 7c3c0476df6..38b3be28760 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -47,7 +47,7 @@ There are also some other related bugs: ## Background Mentions are powered by two of the default push rules that search an event's -`content.body` field for the current user's display name +`content.body` property for the current user's display name ([`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules)) or the localpart of their Matrix ID ([`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules)). @@ -63,22 +63,22 @@ of the message should be colored differently: ## Proposal Instead of searching a message's `body` for the user's display name or the localpart -of their Matrix ID, it is proposed to use a field specific to mentions,[^4] and +of their Matrix ID, it is proposed to use a property specific to mentions,[^4] and augment the push rules to search for the current user's Matrix ID. -### New event field +### New event property -A new `m.mentions` field is added to the event content; it is an object with two -optional fields: +A new `m.mentions` property is added to the event content; it is an object with two +optional properties: * `user_ids`: an array of strings consisting of Matrix IDs to mention. * `room`: a boolean, true indicates an "@room" mention. Any other value or the - field missing is interpreted as not an "@room" mention. + property missing is interpreted as not an "@room" mention. -It is valid to include both the `user_ids` and `room` fields. +It is valid to include both the `user_ids` and `room` properties. It is recommended that homeservers reject locally created events with an invalid -`m.mentions` field with an error with a status code of `400` and an errcode of +`m.mentions` property with an error with a status code of `400` and an errcode of `M_INVALID_PARAM`. Clients should add a Matrix ID to the `user_ids` array whenever composing a message which @@ -87,7 +87,7 @@ includes an intentional [user mention](https://spec.matrix.org/v1.5/client-serve room-wide announcement. Clients may also set these values at other times when it is obvious the user intends to explicitly mention a user. -The `m.mentions` field is part of the cleartext event body and should **not** be +The `m.mentions` property is part of the cleartext event body and should **not** be encrypted into the ciphertext for encrypted events. This exposes slightly more metadata to homeservers, but allows mentions to work properly. It allows the server to properly handle push notifications (which is a requirement for a usable @@ -103,19 +103,19 @@ Two new push rule conditions and corresponding default push rule are proposed: The `is_user_mention` push condition is defined as[^5]: -> This matches messages where the `m.mentions` field of the `content` contains a -> field `user_ids` which is an array containing the owner’s Matrix ID. This +> This matches messages where the `m.mentions` property of the `content` contains a +> property `user_ids` which is an array containing the owner’s Matrix ID. This > condition has no parameters. If the `m.mentions` key is absent; not an object; -> does not contain a `user_ids` field; or contains a non-array `user_ids` field +> does not contain a `user_ids` property; or contains a non-array `user_ids` property > then the rule does not match. Any non-string array entries are ignored and > duplicate entries are ignored. The `is_room_mention` push condition is defined as[^5]: -> This matches messages where the `m.mentions` field of the `content` contains -> a `room` field set to `true`. This condition has no parameters. If the -> `m.mentions` key is absent; not an object; does not contain a `room` field; -> or contains a non-boolean `room` field then the rule does not match. +> This matches messages where the `m.mentions` property of the `content` contains +> a `room` property set to `true`. This condition has no parameters. If the +> `m.mentions` key is absent; not an object; does not contain a `room` property; +> or contains a non-boolean `room` property then the rule does not match. The proposed `.m.rule.is_user_mention` override push rule would appear directly before the `.m.rule.contains_display_name` push rule: @@ -207,7 +207,7 @@ and [`.m.rule.roomnotif`](https://spec.matrix.org/v1.5/client-server-api/#defaul push rules are to be removed. To avoid unintentional mentions these rules are modified to only apply when the -`m.mentions` field is missing; clients should provide the `m.mentions` field on +`m.mentions` property is missing; clients should provide the `m.mentions` property on every message to avoid the unintentional mentions discussed above. If users wish to continue to be notified of messages containing their display name @@ -253,7 +253,7 @@ to CCing users on an e-mail. Although not including mentions in the displayed text could be used as an abuse vector, it does not enable additional malicious behavior than what is possible today. From discussions and research while writing this MSC there are moderation -benefits to using a separate field for mentions: +benefits to using a separate property for mentions: * The number of mentions is trivially limited by moderation tooling, e.g. it may be appropriate for a community room to only allow 10 mentions. Events not abiding @@ -302,14 +302,14 @@ section could be expanded for this situation. ### Extensible events -The `m.mentions` field can be considered a "mixin" as part of extensible events +The `m.mentions` property can be considered a "mixin" as part of extensible events ([MSC1767](https://github.com/matrix-org/matrix-doc/pull/1767)) with no needed changes. ### Role mentions -It is possible to add additional fields to the `m.mentions` object, e.g. a foreseeable -usecase would be a `roles` field which could include values such as `admins` or +It is possible to add additional properties to the `m.mentions` object, e.g. a foreseeable +usecase would be a `roles` property which could include values such as `admins` or `mods`. Defining this in detail is left to a future MSC. ## Alternatives @@ -325,7 +325,7 @@ There are a few prior proposals which tackle subsets of the above problem: users. * [MSC3517](https://github.com/matrix-org/matrix-spec-proposals/pull/3517): searches for Matrix IDs (instead of display name / localpart) and only searches - specific event types & fields. + specific event types & properties. * [Custom push rules](https://o.librepush.net/aux/matrix_reitools/pill_mention_rules.html) to search for matrix.to links instead of display name / localpart. @@ -376,16 +376,16 @@ for extensible events. This does not seem like a good fit since room versions ar not usually interested in non-state events. It would additionally require a stable room version before use, which would unnecessarily delay usage. -### Encrypting the `m.mentions` field +### Encrypting the `m.mentions` property -Encrypting the `m.mentions` field would solve some unintentional mentions, but +Encrypting the `m.mentions` property would solve some unintentional mentions, but will not help with???. -Allowing an encrypted `m.mentions` field on a per-message basis could allow users +Allowing an encrypted `m.mentions` property on a per-message basis could allow users to choose, but would result in inconsistencies and complications: -* What happens if the cleartext `m.mentions` field does not match the plaintext - `m.mentions` field? +* What happens if the cleartext `m.mentions` property does not match the plaintext + `m.mentions` property? * Do push rules get processed on both the cleartext and plaintext message? It may be argued that clients need to decrypt all messages anyway to handle @@ -431,7 +431,7 @@ During development the following mapping will be used: | What | Stable | Unstable | |---------------------|-------------------|--------------------------------------| -| Event field | `m.mentions` | `org.matrix.msc3952.mentions` | +| Event property | `m.mentions` | `org.matrix.msc3952.mentions` | | Push rule ID | `.m.rule.*` | `.org.matrix.msc3952.*` | | Push condition kind | `is_user_mention` | `org.matrix.msc3952.is_user_mention` | | Push condition kind | `is_room_mention` | `org.matrix.msc3952.is_room_mention` | @@ -440,7 +440,7 @@ If a client sees this rule available it can choose to apply the custom logic dis in the [backwards compatibility](#backwards-compatibility) section. If a client sees the *stable* identifiers available, they should apply the new -logic and start creating events with the `m.mentions` field. +logic and start creating events with the `m.mentions` property. ## Dependencies From e11847d532c42c8501e789ead41ce3ff7c5bc604 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 31 Jan 2023 13:01:28 -0500 Subject: [PATCH 20/55] Fix incomplete sentence. --- proposals/3952-intentional-mentions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 38b3be28760..810626df307 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -379,7 +379,7 @@ room version before use, which would unnecessarily delay usage. ### Encrypting the `m.mentions` property Encrypting the `m.mentions` property would solve some unintentional mentions, but -will not help with???. +leaving it unencrypted allows for solving a wider range of issues. Allowing an encrypted `m.mentions` property on a per-message basis could allow users to choose, but would result in inconsistencies and complications: From 6269d0d83d22d29f2fd7ab68fc4b27da58665249 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 31 Jan 2023 13:02:11 -0500 Subject: [PATCH 21/55] Room notifs shouldn't have sound. --- proposals/3952-intentional-mentions.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 810626df307..0a8ac343b18 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -162,10 +162,6 @@ before the `.m.rule.roomnotif` push rule: ], "actions": [ "notify", - { - "set_tweak": "sound", - "value": "default" - }, { "set_tweak": "highlight" } From 425d1d61a8c494ee2bb9e9f47336f497ba9ee8a2 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 31 Jan 2023 13:05:25 -0500 Subject: [PATCH 22/55] Clarify sentence. --- proposals/3952-intentional-mentions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 0a8ac343b18..0b57f80f02a 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -385,9 +385,9 @@ to choose, but would result in inconsistencies and complications: * Do push rules get processed on both the cleartext and plaintext message? It may be argued that clients need to decrypt all messages anyway to handle -user-specific keywords, the process for doing this is costly (either receiving -every message via push notifications or backpaginating every room fully, in case -of a gappy sync) in terms of bandwidth, CPU, and battery. +user-specific keywords. This is a costly process in terms of bandwidth, CPU, and +battery since the client must either receive every message via push notifications +or backpaginate every room fully, in case of a gappy sync. It is asserted that the use-cases for custom keywords and mentions are sufficiently different that having different solutions, and different urgencies to receiving From ced04e19730283b6c56a8ef903d20634cde63363 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 8 Feb 2023 07:43:38 -0500 Subject: [PATCH 23/55] Replace is_room_mention with exact_event_match. --- proposals/3952-intentional-mentions.md | 35 +++++++------------------- 1 file changed, 9 insertions(+), 26 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 0b57f80f02a..ddae5882d75 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -99,7 +99,7 @@ the user has been mentioned in them, e.g. for an extension to ### New push rules -Two new push rule conditions and corresponding default push rule are proposed: +A new push rule condition and two new default push rule are proposed: The `is_user_mention` push condition is defined as[^5]: @@ -110,13 +110,6 @@ The `is_user_mention` push condition is defined as[^5]: > then the rule does not match. Any non-string array entries are ignored and > duplicate entries are ignored. -The `is_room_mention` push condition is defined as[^5]: - -> This matches messages where the `m.mentions` property of the `content` contains -> a `room` property set to `true`. This condition has no parameters. If the -> `m.mentions` key is absent; not an object; does not contain a `room` property; -> or contains a non-boolean `room` property then the rule does not match. - The proposed `.m.rule.is_user_mention` override push rule would appear directly before the `.m.rule.contains_display_name` push rule: @@ -153,7 +146,9 @@ before the `.m.rule.roomnotif` push rule: "enabled": true, "conditions": [ { - "kind": "is_room_mention" + "kind": "exact_event_match", + "key": "content.m\\.mentions.room", + "value": true }, { "kind": "sender_notification_permission", @@ -395,22 +390,6 @@ those notifications, is reasonable. In particular, custom keywords is more of a power-user feature while user mentions working properly (and promptly) is a table-stakes feature for a messaging application. -### For the `is_room_mention` condition - -The `is_room_mention` could be replaced with the `exact_event_match` condition -from [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758) -(or the changes to `event_match` in [MSC3862](https://github.com/matrix-org/matrix-spec-proposals/pull/3862)), -although either of those would potentially run into ambiguity in the `key`, -as described in [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): - -```json -{ - "kind": "exact_event_match", - "key": "content.m.mentions.room", - "value": true -} -``` - ## Security considerations Including the mentioned users in cleartext is a small metadata leak, but we consider that the tradeoff @@ -440,7 +419,11 @@ logic and start creating events with the `m.mentions` property. ## Dependencies -N/A +This depends on two MSCs which, at the time of writing, have not yet been accepted +into the specification: + +* [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758): Add `exact_event_match` push rule condition kind +* [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): event_match dotted keys [^1]: This MSC does not attempt to solve this problem, but [MSC2781](https://github.com/matrix-org/matrix-spec-proposals/pull/2781) proposes removing reply fallbacks which would solve it. Although as noted in From dc91cc15b9cdb9b7e1c8c024bccb09bd386def54 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 14 Feb 2023 09:56:40 -0500 Subject: [PATCH 24/55] Replace is_user_mention condition with MSC3966. --- proposals/3952-intentional-mentions.md | 20 ++++++-------------- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index ddae5882d75..1ebc4a0123f 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -99,16 +99,7 @@ the user has been mentioned in them, e.g. for an extension to ### New push rules -A new push rule condition and two new default push rule are proposed: - -The `is_user_mention` push condition is defined as[^5]: - -> This matches messages where the `m.mentions` property of the `content` contains a -> property `user_ids` which is an array containing the owner’s Matrix ID. This -> condition has no parameters. If the `m.mentions` key is absent; not an object; -> does not contain a `user_ids` property; or contains a non-array `user_ids` property -> then the rule does not match. Any non-string array entries are ignored and -> duplicate entries are ignored. +Two new default push rule are proposed: The proposed `.m.rule.is_user_mention` override push rule would appear directly before the `.m.rule.contains_display_name` push rule: @@ -120,7 +111,9 @@ before the `.m.rule.contains_display_name` push rule: "enabled": true, "conditions": [ { - "kind": "is_user_mention" + "kind": "exact_event_property_contains", + "key": "content.m\\.mentions.user_ids", + "value": "[the user's Matrix ID]" } ], "actions": [ @@ -406,10 +399,8 @@ During development the following mapping will be used: | What | Stable | Unstable | |---------------------|-------------------|--------------------------------------| -| Event property | `m.mentions` | `org.matrix.msc3952.mentions` | +| Event property | `m.mentions` | `org.matrix.msc3952.mentions` | | Push rule ID | `.m.rule.*` | `.org.matrix.msc3952.*` | -| Push condition kind | `is_user_mention` | `org.matrix.msc3952.is_user_mention` | -| Push condition kind | `is_room_mention` | `org.matrix.msc3952.is_room_mention` | If a client sees this rule available it can choose to apply the custom logic discussed in the [backwards compatibility](#backwards-compatibility) section. @@ -424,6 +415,7 @@ into the specification: * [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758): Add `exact_event_match` push rule condition kind * [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): event_match dotted keys +* [MSC3966](https://github.com/matrix-org/matrix-spec-proposals/pull/3966): `exact_event_property_contains` push rule condition [^1]: This MSC does not attempt to solve this problem, but [MSC2781](https://github.com/matrix-org/matrix-spec-proposals/pull/2781) proposes removing reply fallbacks which would solve it. Although as noted in From 86bf972c2c8ef04dc849ada5bbcb986ac990a7a3 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Feb 2023 08:04:44 -0500 Subject: [PATCH 25/55] Updates for dependent MSCs. --- proposals/3952-intentional-mentions.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 1ebc4a0123f..3759b8f209a 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -111,7 +111,7 @@ before the `.m.rule.contains_display_name` push rule: "enabled": true, "conditions": [ { - "kind": "exact_event_property_contains", + "kind": "event_property_contains", "key": "content.m\\.mentions.user_ids", "value": "[the user's Matrix ID]" } @@ -139,7 +139,7 @@ before the `.m.rule.roomnotif` push rule: "enabled": true, "conditions": [ { - "kind": "exact_event_match", + "kind": "event_property_is", "key": "content.m\\.mentions.room", "value": true }, @@ -413,9 +413,9 @@ logic and start creating events with the `m.mentions` property. This depends on two MSCs which, at the time of writing, have not yet been accepted into the specification: -* [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758): Add `exact_event_match` push rule condition kind +* [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758): Add `event_property_is` push rule condition kind * [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): event_match dotted keys -* [MSC3966](https://github.com/matrix-org/matrix-spec-proposals/pull/3966): `exact_event_property_contains` push rule condition +* [MSC3966](https://github.com/matrix-org/matrix-spec-proposals/pull/3966): `event_property_contains` push rule condition [^1]: This MSC does not attempt to solve this problem, but [MSC2781](https://github.com/matrix-org/matrix-spec-proposals/pull/2781) proposes removing reply fallbacks which would solve it. Although as noted in From f0a1f6ad184788814c45d58370248b8052142171 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Feb 2023 10:12:36 -0500 Subject: [PATCH 26/55] Revert changes about keeping the mentions property in cleartext. Manually reverts 4de784893e53995991b18f44329ca074b54547ae. --- proposals/3952-intentional-mentions.md | 58 ++++++-------------------- 1 file changed, 13 insertions(+), 45 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 3759b8f209a..aa1fe063363 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -87,15 +87,8 @@ includes an intentional [user mention](https://spec.matrix.org/v1.5/client-serve room-wide announcement. Clients may also set these values at other times when it is obvious the user intends to explicitly mention a user. -The `m.mentions` property is part of the cleartext event body and should **not** be -encrypted into the ciphertext for encrypted events. This exposes slightly more -metadata to homeservers, but allows mentions to work properly. It allows the -server to properly handle push notifications (which is a requirement for a usable -chat application) and could result in bandwidth and battery savings (see the -[future extensions](#muted-except-for-mentions-push-rules) section). Additionally, -it may be useful for the homeserver to filter or prioritize rooms based on whether -the user has been mentioned in them, e.g. for an extension to -[MSC3575 (sliding sync)](https://github.com/matrix-org/matrix-spec-proposals/pull/3575). +The `m.mentions` property is part of the plaintext event body and should be encrypted +into the ciphertext for encrypted events. ### New push rules @@ -272,11 +265,14 @@ to give users more control over who can generate notifications. ### Muted except for mentions push rules -By leaving the mentions array unencrypted it allows for a "muted-except-for-mentions" -push rule. This is particularly useful for large (encrypted) rooms where you would -like to receive push notifications for mentions *only*. It is imperative for -the homeserver to be able to send a push in this case since some mobile platforms, -e.g. iOS, do not sync regularly in the background. +It might be desirable to have a "muted-except-for-mentions" feature for large (encrypted) +rooms. This is particularly useful on iOS where a push notification can be decrypted +via a background process but *cannot* be suppressed. This means it is not possible +for the client to handle this feature and it must be handled on the server, unfortunately +this would not be possible with the current proposal since the `m.mentions` +property is encrypted (and the server cannot act on it). + +Solving this problem is left to a future MSC. ### Pillifying `@room` @@ -303,7 +299,8 @@ usecase would be a `roles` property which could include values such as `admins` There are a few prior proposals which tackle subsets of the above problem: * [MSC1796](https://github.com/matrix-org/matrix-spec-proposals/pull/1796): - extremely similar to the proposal in this MSC, but limited to encrypted events. + similar to the proposal in this MSC, but limited to encrypted events (and kept + in cleartext). * [MSC2463](https://github.com/matrix-org/matrix-spec-proposals/pull/2463): excludes searching inside a Matrix ID for localparts / display names of other users. @@ -360,38 +357,9 @@ for extensible events. This does not seem like a good fit since room versions ar not usually interested in non-state events. It would additionally require a stable room version before use, which would unnecessarily delay usage. -### Encrypting the `m.mentions` property - -Encrypting the `m.mentions` property would solve some unintentional mentions, but -leaving it unencrypted allows for solving a wider range of issues. - -Allowing an encrypted `m.mentions` property on a per-message basis could allow users -to choose, but would result in inconsistencies and complications: - -* What happens if the cleartext `m.mentions` property does not match the plaintext - `m.mentions` property? -* Do push rules get processed on both the cleartext and plaintext message? - -It may be argued that clients need to decrypt all messages anyway to handle -user-specific keywords. This is a costly process in terms of bandwidth, CPU, and -battery since the client must either receive every message via push notifications -or backpaginate every room fully, in case of a gappy sync. - -It is asserted that the use-cases for custom keywords and mentions are sufficiently -different that having different solutions, and different urgencies to receiving -those notifications, is reasonable. In particular, custom keywords is more of a -power-user feature while user mentions working properly (and promptly) is a table-stakes -feature for a messaging application. - ## Security considerations -Including the mentioned users in cleartext is a small metadata leak, but we consider that the tradeoff -of mentions working properly and the savings in bandwidth and battery makes it worthwhile. - -The additional metadata leaked is minor since the homeserver already knows the -room members, how often users are sending messages, when users are receiving messages -(via read receipts, read markers, and sync), etc. Exposing explicit mentions does not -significantly increase metadata leakage. +None foreseen. ## Unstable prefix From e346fbb27bf1374f0d58b38ea2d1b1694397c305 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Feb 2023 10:21:44 -0500 Subject: [PATCH 27/55] Clean-up some of the deprecation text. --- proposals/3952-intentional-mentions.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index aa1fe063363..7be39968d3f 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -181,12 +181,17 @@ and `.m.rule.is_room_mention` is provided below: The [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules), [`.m.rule.contains_user_name`](https://spec.matrix.org/v1.5/client-server-api/#default-content-rules), and [`.m.rule.roomnotif`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules) -push rules are to be removed. +push rules are to be deprecated. To avoid unintentional mentions these rules are modified to only apply when the `m.mentions` property is missing; clients should provide the `m.mentions` property on every message to avoid the unintentional mentions discussed above. +A future room version may wish to disable the legacy push rules: clients would +no longer be required to include the `m.mentions` property on every event. It +maybe convenient to do this when extensible events are adopted (see +[MSC3932](https://github.com/matrix-org/matrix-spec-proposals/pull/3932)). + If users wish to continue to be notified of messages containing their display name it is recommended that clients create a specific keyword rule for this, e.g. a `content` rule of the form: @@ -370,11 +375,8 @@ During development the following mapping will be used: | Event property | `m.mentions` | `org.matrix.msc3952.mentions` | | Push rule ID | `.m.rule.*` | `.org.matrix.msc3952.*` | -If a client sees this rule available it can choose to apply the custom logic discussed -in the [backwards compatibility](#backwards-compatibility) section. - -If a client sees the *stable* identifiers available, they should apply the new -logic and start creating events with the `m.mentions` property. +If a client sees this rule available it can choose to apply the deprecation logic +discussed in the [backwards compatibility](#backwards-compatibility) section. ## Dependencies From d619f2160703f816823a15cde937a142753df3fd Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Feb 2023 10:46:21 -0500 Subject: [PATCH 28/55] Add a note about explicit mentions & pills. --- proposals/3952-intentional-mentions.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 7be39968d3f..f9b1739e08c 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -85,7 +85,7 @@ Clients should add a Matrix ID to the `user_ids` array whenever composing a mess includes an intentional [user mention](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) (often called a "pill"). Clients should set the `room` value to `true` when making a room-wide announcement. Clients may also set these values at other times when it is -obvious the user intends to explicitly mention a user. +obvious the user intends to explicitly mention a user.[^5] The `m.mentions` property is part of the plaintext event body and should be encrypted into the ciphertext for encrypted events. @@ -404,5 +404,7 @@ when converting mentions, where the [^4]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). -[^5]: A new push condition is necessary since none of the current push conditions -(e.g. `event_match`) can process arrays. +[^5]: Note that this isn't really a change in behavior, it is just making the behavior +explicit. It is expected that users already considered "pilled" users to be mentions, +and it was more unexpected when non-pilled users *were* mentioned. This MSC fixes the +latter case. From 8f2805defcc25865f37e9ddfd94b08553b2bbbe4 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Feb 2023 11:35:13 -0500 Subject: [PATCH 29/55] Add bridge information. --- proposals/3952-intentional-mentions.md | 29 ++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index f9b1739e08c..d95d5c80a59 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -38,8 +38,11 @@ There are also some other related bugs: * Matrix users will append emoji or other unique text in their display names to avoid unintentional pings. -* Bridges [must use display names](https://github.com/matrix-org/matrix-spec/issues/353#issuecomment-1055809364) - as a trick to get pings to work. +* Bridging mentions is suboptimal since they [use display names](https://github.com/matrix-org/matrix-spec/issues/353#issuecomment-1055809364) + as a workaround, e.g.: + * It broke the contract that bridges will not mutate the content of messages. + * For some protocols, bridges need try to figure out if every message contains + any of the possible nicknames of room members. * If a user changes their display name in a room, they might not be mentioned unless the historical display name is used while processing push rules. (TODO I think there's an issue about this?) @@ -222,6 +225,18 @@ push rules, while up-to-date clients and homeservers will support the that both sets of push rules will need to be supported for a period of time, but at worst case should simply result in the current behavior (documented in the preamble). +### Impact on bridging + +This should strengthen the bridging contract as it will allow them to stop +mutating the content of messages during the bridging process (by directly +listing any mentioned users in the `m.mentions` property). + +This should reduce the impedance mismatch when bridging protocols which similarly +list out users to mention (the bridge should simply be able to map from the +remote user ID to the bridged user ID)[^6], but will not help in cases where the +mention information is embedded into the text content (as today's Matrix messages +are).[^7] + ## Potential issues ### Abuse potential @@ -408,3 +423,13 @@ when converting mentions, where the explicit. It is expected that users already considered "pilled" users to be mentions, and it was more unexpected when non-pilled users *were* mentioned. This MSC fixes the latter case. + +[^6]: Some protocols which provide structured data for mentions include +[Twitter](https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/tweet), +[Mastodon](https://docs.joinmastodon.org/entities/Status/#Mention), +[Discord](https://discord.com/developers/docs/resources/channel#message-object), +and [Microsoft Teams](https://learn.microsoft.com/en-us/graph/api/resources/chatmessagemention?view=graph-rest-1.0). + +[^7]: Unfortunately some protocols do *not* provide structured data: the message +itself must be parsed for mentions, e.g. IRC or +[Slack](https://api.slack.com/reference/surfaces/formatting#mentioning-users). From 7de34e42eb0abce53aec842ff7ed11322d406e02 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 28 Feb 2023 15:10:07 -0500 Subject: [PATCH 30/55] Add some info on replies & edits. --- proposals/3952-intentional-mentions.md | 125 ++++++++++++++++++++++++- 1 file changed, 123 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index d95d5c80a59..2f212eb14c0 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -17,9 +17,9 @@ Some situations that result in unintentional mentions include: evaluated. * If the [localpart of your Matrix ID is a common word](https://github.com/matrix-org/matrix-spec-proposals/issues/3011) then the push rule matching usernames (`.m.rule.contains_user_name`) matches - too often (e.g. Travis CI matching if your Matrix ID is `@travis:example.com`). + too often (e.g. Travis CI matching if your Matrix ID is `@travis:example.org`). * If the [localpart or display name of your Matrix ID matches the hostname](https://github.com/matrix-org/matrix-spec-proposals/issues/2735) - (e.g. `@example:example.com` receives notifications whenever `@foo:example.com` + (e.g. `@example:example.org` receives notifications whenever `@foo:example.org` is replied to). As a sender you do not know if including the user's display name or Matrix ID would @@ -225,6 +225,127 @@ push rules, while up-to-date clients and homeservers will support the that both sets of push rules will need to be supported for a period of time, but at worst case should simply result in the current behavior (documented in the preamble). +### Impact on replies + +Users are notified of replies via the `.m.rule.contains_display_name` or the +`.m.rule.contains_user_name` push rule matching the +[rich reply fallback](https://spec.matrix.org/v1.6/client-server-api/#fallbacks-for-rich-replies). +Unfortunately these push rules will be disabled for events which contain the +`m.mentions` property, i.e. all newly created events (see +[above](#backwards-compatibility)). It is proposed that clients should include +the sender of the event being replied to as well as any users (except themself) +mentioned in that event in the new event's `m.mentions` property. The `room` +property should not be copied over. + +For example, if there is an event: + +```json5 +{ + "sender": "@dan:example.org", + "event_id": "$initial_event", + "content": { + "body": "Alice: Have you heard from Bob?", + "m.mentions": { + "user_ids": ["@alice:example.org", "@bob:example.org"] + } + }, + // other fields as required by events +} +``` + +And a reply from Alice: + +```json5 +{ + "content": { + "body": "> <@dan:example.org> Alice: Have you heard from Bob?\n\nNo, but I saw him with Charlie earlier.", + "m.mentions": { + "user_ids": [ + // Include the sender of $initial_event. + "@dan:example.org", + // The users mentioned, minus yourself. + "@bob:example.org", + // New mentions, as normal. + "@charlie:example.org" + ] + }, + "m.relates_to": { + "m.in_reply_to": { + "event_id": "$initial_event" + } + } + }, + // other fields as required by events +} +``` + +This signals that it is the *intention* of the sender to mention all of those people, +clients may wish to allow users to modify the list of people to include, e.g. to +"quote reply" as opposed to replying directly. + +If a user wishes to be notified of *all replies* to their messages, other solutions +should be investigated, such as [MSC3664](https://github.com/matrix-org/matrix-spec-proposals/pull/3664). +This would give more equal power to both senders and receivers of events. + +### Impact on edits + +Similarly to [replies](#impact-on-replies), users are notified of message edits +via the `.m.rule.contains_display_name` or the `.m.rule.contains_user_name` push +rule matching the [fallback content](https://spec.matrix.org/v1.6/client-server-api/#event-replacements). +Generally this is undesirable and users do not need to be notified for the same +message multiple times (e.g. if a user is fixing a typo). It is recommended that +clients include an empty `m.mentions` property when editing an event, *unless* +the edit is significant or if additional users are mentioned. + +The full list of mentioned users should be included in the `m.new_content` property. + +For example, if there is an event: + +```json5 +{ + "sender": "@dan:example.org", + "event_id": "$initial_event", + "content": { + "body": "Helo Alice!", + "m.mentions": { + "user_ids": ["@alice:example.org"] + } + }, + // other fields as required by events +} +``` + +And an edit after realizing that Bob is also in the room: + +```json5 +{ + "content": { + "body": "* Hello Alice & Bob!", + "m.mentions": { + "user_ids": [ + // Include only the newly mentioned user. + "@bob:example.org" + ] + }, + "m.new_content": { + "body": "Hello Alice & Bob!", + "m.mentions": { + "user_ids": [ + // Include all mentioned users. + "@alice:example.org", + "@bob:example.org" + ] + }, + }, + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$initial_event" + } + }, + // other fields as required by events +} +``` + ### Impact on bridging This should strengthen the bridging contract as it will allow them to stop From 040cc31273440edebf28299cf31641b879297c63 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 28 Feb 2023 15:16:33 -0500 Subject: [PATCH 31/55] Update footnotes & edits a bit. --- proposals/3952-intentional-mentions.md | 38 ++++++++++++-------------- 1 file changed, 17 insertions(+), 21 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 2f212eb14c0..926caf7ece0 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -10,8 +10,8 @@ or highlight notification.) Some situations that result in unintentional mentions include: * Replying to a message will re-issue pings from the initial message due to - [fallback replies](https://spec.matrix.org/v1.5/client-server-api/#fallbacks-for-rich-replies). [^1] -* Each time a message is edited the new version will be re-evaluated for mentions. [^2] + [fallback replies](https://spec.matrix.org/v1.5/client-server-api/#fallbacks-for-rich-replies). +* Each time a message is edited the new version will be re-evaluated for mentions. * Mentions occurring [in spoiler contents](https://github.com/matrix-org/matrix-spec/issues/16) or [code blocks](https://github.com/matrix-org/matrix-spec/issues/15) are evaluated. @@ -32,7 +32,7 @@ This results in some unexpected behavior and bugs: (or display name). * Since the relation between `body` and `formatted_body` is ill-defined and ["pills" are converted to display names](https://github.com/matrix-org/matrix-spec/issues/714), - this can result in missed messages. [^3] + this can result in missed messages. [^1] There are also some other related bugs: @@ -66,7 +66,7 @@ of the message should be colored differently: ## Proposal Instead of searching a message's `body` for the user's display name or the localpart -of their Matrix ID, it is proposed to use a property specific to mentions,[^4] and +of their Matrix ID, it is proposed to use a property specific to mentions,[^2] and augment the push rules to search for the current user's Matrix ID. ### New event property @@ -88,7 +88,7 @@ Clients should add a Matrix ID to the `user_ids` array whenever composing a mess includes an intentional [user mention](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) (often called a "pill"). Clients should set the `room` value to `true` when making a room-wide announcement. Clients may also set these values at other times when it is -obvious the user intends to explicitly mention a user.[^5] +obvious the user intends to explicitly mention a user.[^3] The `m.mentions` property is part of the plaintext event body and should be encrypted into the ciphertext for encrypted events. @@ -346,6 +346,11 @@ And an edit after realizing that Bob is also in the room: } ``` +This should limit duplicate, unnecssary notifications for users. If a user wishes +to receive notifications for edits of events they were mentioned in then they +could setup a push rule for the `content.m\\.new_content.m\\.mentions` property +or potentially leverage [MSC3664](https://github.com/matrix-org/matrix-spec-proposals/pull/3664). + ### Impact on bridging This should strengthen the bridging contract as it will allow them to stop @@ -354,9 +359,9 @@ listing any mentioned users in the `m.mentions` property). This should reduce the impedance mismatch when bridging protocols which similarly list out users to mention (the bridge should simply be able to map from the -remote user ID to the bridged user ID)[^6], but will not help in cases where the +remote user ID to the bridged user ID)[^4], but will not help in cases where the mention information is embedded into the text content (as today's Matrix messages -are).[^7] +are).[^5] ## Potential issues @@ -523,34 +528,25 @@ into the specification: * [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): event_match dotted keys * [MSC3966](https://github.com/matrix-org/matrix-spec-proposals/pull/3966): `event_property_contains` push rule condition -[^1]: This MSC does not attempt to solve this problem, but [MSC2781](https://github.com/matrix-org/matrix-spec-proposals/pull/2781) -proposes removing reply fallbacks which would solve it. Although as noted in -[MSC3676](https://github.com/matrix-org/matrix-spec-proposals/pull/3676) this -may require landing [MSC3664](https://github.com/matrix-org/matrix-doc/pull/3664) -in order to receive notifications of replies. - -[^2]: Note that this MSC does not attempt to solve the problem of edits generating -spurious notifications. - -[^3]: It is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes) +[^1]: It is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes) the the "plain text version of the HTML [`formatted_body`] should be provided in the `body`", but there is no particular algorithm to convert from HTML to plain text *except* when converting mentions, where the [plain text version uses the link anchor, not the link](https://spec.matrix.org/v1.5/client-server-api/#client-behaviour-26). -[^4]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). +[^2]: As proposed in [issue 353](https://github.com/matrix-org/matrix-spec/issues/353). -[^5]: Note that this isn't really a change in behavior, it is just making the behavior +[^3]: Note that this isn't really a change in behavior, it is just making the behavior explicit. It is expected that users already considered "pilled" users to be mentions, and it was more unexpected when non-pilled users *were* mentioned. This MSC fixes the latter case. -[^6]: Some protocols which provide structured data for mentions include +[^4]: Some protocols which provide structured data for mentions include [Twitter](https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/tweet), [Mastodon](https://docs.joinmastodon.org/entities/Status/#Mention), [Discord](https://discord.com/developers/docs/resources/channel#message-object), and [Microsoft Teams](https://learn.microsoft.com/en-us/graph/api/resources/chatmessagemention?view=graph-rest-1.0). -[^7]: Unfortunately some protocols do *not* provide structured data: the message +[^5]: Unfortunately some protocols do *not* provide structured data: the message itself must be parsed for mentions, e.g. IRC or [Slack](https://api.slack.com/reference/surfaces/formatting#mentioning-users). From 6b57caeb03409af9d2613770e83ffe47278197d5 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 1 Mar 2023 09:54:20 -0500 Subject: [PATCH 32/55] Various wording tweaks & typo fixes. Co-authored-by: Travis Ralston --- proposals/3952-intentional-mentions.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 926caf7ece0..444fa1f75ad 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -95,9 +95,9 @@ into the ciphertext for encrypted events. ### New push rules -Two new default push rule are proposed: +Two new default push rule are added: -The proposed `.m.rule.is_user_mention` override push rule would appear directly +The `.m.rule.is_user_mention` override push rule would appear directly before the `.m.rule.contains_display_name` push rule: ```json @@ -125,7 +125,7 @@ before the `.m.rule.contains_display_name` push rule: } ``` -The proposed `.m.rule.is_room_mention` override push rule would appear directly +The `.m.rule.is_room_mention` override push rule would appear directly before the `.m.rule.roomnotif` push rule: ```json @@ -187,7 +187,7 @@ and [`.m.rule.roomnotif`](https://spec.matrix.org/v1.5/client-server-api/#defaul push rules are to be deprecated. To avoid unintentional mentions these rules are modified to only apply when the -`m.mentions` property is missing; clients should provide the `m.mentions` property on +`m.mentions` property is missing; clients should provide at least an empty `m.mentions` property on every message to avoid the unintentional mentions discussed above. A future room version may wish to disable the legacy push rules: clients would @@ -346,7 +346,7 @@ And an edit after realizing that Bob is also in the room: } ``` -This should limit duplicate, unnecssary notifications for users. If a user wishes +This should limit duplicate, unnecessary notifications for users. If a user wishes to receive notifications for edits of events they were mentioned in then they could setup a push rule for the `content.m\\.new_content.m\\.mentions` property or potentially leverage [MSC3664](https://github.com/matrix-org/matrix-spec-proposals/pull/3664). @@ -428,9 +428,8 @@ section could be expanded for this situation. ### Extensible events -The `m.mentions` property can be considered a "mixin" as part of extensible events -([MSC1767](https://github.com/matrix-org/matrix-doc/pull/1767)) with no needed -changes. +Handling of this property in [MSC1767](https://github.com/matrix-org/matrix-doc/pull/1767)-style +extensible events is deliberately left for a future MSC to address, if needed. ### Role mentions @@ -501,11 +500,13 @@ Alternative backwards compatibility suggestions included using a new room versio similar to [MSC3932](https://github.com/matrix-org/matrix-spec-proposals/pull/3932) for extensible events. This does not seem like a good fit since room versions are not usually interested in non-state events. It would additionally require a stable -room version before use, which would unnecessarily delay usage. +room version before use, which would unnecessarily delay usage. Another MSC +can address this concern, such as in the extensible events series, if +desirable to be gated by a room version for a "clean slate" approach. ## Security considerations -None foreseen. +None not already described. ## Unstable prefix @@ -521,7 +522,7 @@ discussed in the [backwards compatibility](#backwards-compatibility) section. ## Dependencies -This depends on two MSCs which, at the time of writing, have not yet been accepted +This depends on the following MSCs which, at the time of writing, have not yet been accepted into the specification: * [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758): Add `event_property_is` push rule condition kind From a2ae719927d8804bcf8b96a516ab585a21bf32a7 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 1 Mar 2023 10:19:19 -0500 Subject: [PATCH 33/55] Add a missing reference. --- proposals/3952-intentional-mentions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 444fa1f75ad..104c59b269f 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -43,9 +43,9 @@ There are also some other related bugs: * It broke the contract that bridges will not mutate the content of messages. * For some protocols, bridges need try to figure out if every message contains any of the possible nicknames of room members. -* If a user changes their display name in a room, they might not be mentioned - unless the historical display name is used while processing push rules. - (TODO I think there's an issue about this?) +* If a user changes their display name in a room, + [they might not be mentioned unless the historical display name](https://github.com/matrix-org/matrix-spec/issues/353#issuecomment-1055809372) + is used while processing push rules. ## Background From 3461ef8f83ac3a5baa433d3e2c954633c06e47b2 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 1 Mar 2023 10:39:57 -0500 Subject: [PATCH 34/55] Tweak proposal intro. --- proposals/3952-intentional-mentions.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 104c59b269f..58f0b2bf26c 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -40,7 +40,7 @@ There are also some other related bugs: avoid unintentional pings. * Bridging mentions is suboptimal since they [use display names](https://github.com/matrix-org/matrix-spec/issues/353#issuecomment-1055809364) as a workaround, e.g.: - * It broke the contract that bridges will not mutate the content of messages. + * It breaks the contract that bridges will not mutate the content of messages. * For some protocols, bridges need try to figure out if every message contains any of the possible nicknames of room members. * If a user changes their display name in a room, @@ -65,9 +65,9 @@ of the message should be colored differently: ## Proposal -Instead of searching a message's `body` for the user's display name or the localpart -of their Matrix ID, it is proposed to use a property specific to mentions,[^2] and -augment the push rules to search for the current user's Matrix ID. +The existing push rules for mentions are deprecated and replaced with rules +which use a property specific for mentions[^2], making the search for a mention +much simpler and more reliable for the user. ### New event property From f5dee4986f9760e098ded131390fc9bf5df42504 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 1 Mar 2023 10:54:29 -0500 Subject: [PATCH 35/55] Add note about removing mentioned users. --- proposals/3952-intentional-mentions.md | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 58f0b2bf26c..2f0be7b7b38 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -346,6 +346,29 @@ And an edit after realizing that Bob is also in the room: } ``` +Mentions can also be removed as part of an edit, in this case top-level `m.mentions` +property would not include any user IDs (you cannot cancel the notification from +the previous event), and the user would be removed from the `m.new_content`'s copy +of `m.mentions`. For example, editing the initial event to offer a generic greeting: + +```json5 +{ + "content": { + "body": "* Hello friends!", + "m.mentions": {}, + "m.new_content": { + "body": "Hello Alice & Bob!", + "m.mentions": {}, + }, + "m.relates_to": { + "rel_type": "m.replace", + "event_id": "$initial_event" + } + }, + // other fields as required by events +} +``` + This should limit duplicate, unnecessary notifications for users. If a user wishes to receive notifications for edits of events they were mentioned in then they could setup a push rule for the `content.m\\.new_content.m\\.mentions` property From c79d36f1e3a5ccf72ad0ecc770a29ae5f52a3068 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 1 Mar 2023 11:04:43 -0500 Subject: [PATCH 36/55] Add more info about combating abuse. --- proposals/3952-intentional-mentions.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 2f0be7b7b38..09cc0454150 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -417,7 +417,8 @@ malicious behavior. ### Combating abuse Some ideas for combating abuse came from our discussion and research which are -worth sharing. These ideas are not part of this MSC, but also do not depend on it. +worth sharing. These ideas are not a requirement for implementing this MSC, and +generally do not depend on it. (They could be implemented today with enough effort.) It was recommended that clients could expose *why* an event has caused a notification and give users inline tools to combat abuse. For example, a client might show a tooltip: @@ -426,8 +427,13 @@ and give users inline tools to combat abuse. For example, a client might show a > > Block `@alice:example.org` from sending you messages? `[Yes]` `[No]` -Additionally, tooling to for moderators to quickly find messages which mention -many users would be useful. +Additionally, if a user sending a message is about to mention many people it can +be useful to confirm whether they wish to do that (or prompt them to do an `@room` +mention instead). + +Moderators may find tooling to quickly find messages which mention many users +useful in combating mention spammers. (Note that this should be made easier by +this MSC.) A future MSC might wish to explore features for trusted contacts or soft-ignores to give users more control over who can generate notifications. From e8694920b936a61f8d2cf9ddf53cea1db7a556c1 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 1 Mar 2023 11:08:38 -0500 Subject: [PATCH 37/55] Separate footnotes from dependencies. --- proposals/3952-intentional-mentions.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 09cc0454150..3d35423aa1a 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -558,6 +558,8 @@ into the specification: * [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): event_match dotted keys * [MSC3966](https://github.com/matrix-org/matrix-spec-proposals/pull/3966): `event_property_contains` push rule condition + + [^1]: It is [defined as](https://spec.matrix.org/v1.5/client-server-api/#mroommessage-msgtypes) the the "plain text version of the HTML [`formatted_body`] should be provided in the `body`", but there is no particular algorithm to convert from HTML to plain text *except* From 04ca14245b955f831b086aa895079ea804526430 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 2 Mar 2023 08:44:54 -0500 Subject: [PATCH 38/55] Clarify edits. --- proposals/3952-intentional-mentions.md | 24 +++--------------------- 1 file changed, 3 insertions(+), 21 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 3d35423aa1a..b0f2861b55c 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -347,27 +347,9 @@ And an edit after realizing that Bob is also in the room: ``` Mentions can also be removed as part of an edit, in this case top-level `m.mentions` -property would not include any user IDs (you cannot cancel the notification from -the previous event), and the user would be removed from the `m.new_content`'s copy -of `m.mentions`. For example, editing the initial event to offer a generic greeting: - -```json5 -{ - "content": { - "body": "* Hello friends!", - "m.mentions": {}, - "m.new_content": { - "body": "Hello Alice & Bob!", - "m.mentions": {}, - }, - "m.relates_to": { - "rel_type": "m.replace", - "event_id": "$initial_event" - } - }, - // other fields as required by events -} -``` +property would not include the removed user IDs (you cannot cancel the notification from +the previous event) or any previously notified users, and the removed user would also be +removed from the `m.new_content` proprerty's copy of `m.mentions`. This should limit duplicate, unnecessary notifications for users. If a user wishes to receive notifications for edits of events they were mentioned in then they From 6ac9392b3605392fa0b018ff2ef969e0d86966c5 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 16 Mar 2023 10:37:34 -0400 Subject: [PATCH 39/55] Minor updates. --- proposals/3952-intentional-mentions.md | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index b0f2861b55c..2b9394f8101 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -11,6 +11,9 @@ Some situations that result in unintentional mentions include: * Replying to a message will re-issue pings from the initial message due to [fallback replies](https://spec.matrix.org/v1.5/client-server-api/#fallbacks-for-rich-replies). + * A user without the power level to send `@room` can abuse this by includin + `@room` in a message and getting a user with the appropriate power levels + to reply to them. * Each time a message is edited the new version will be re-evaluated for mentions. * Mentions occurring [in spoiler contents](https://github.com/matrix-org/matrix-spec/issues/16) or [code blocks](https://github.com/matrix-org/matrix-spec/issues/15) are @@ -528,8 +531,10 @@ During development the following mapping will be used: | Event property | `m.mentions` | `org.matrix.msc3952.mentions` | | Push rule ID | `.m.rule.*` | `.org.matrix.msc3952.*` | -If a client sees this rule available it can choose to apply the deprecation logic -discussed in the [backwards compatibility](#backwards-compatibility) section. +The server will include the `org.matrix.msc3952_intentional_mentions` flag in the +`unstable_features` array of the `/versions` endpoint. If a client sees this flag +it can choose to apply the deprecation logic discussed in the +[backwards compatibility](#backwards-compatibility) section. ## Dependencies From ea1561c8081b545ab3d9c563c35c091779992436 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 21 Mar 2023 11:50:45 -0400 Subject: [PATCH 40/55] Minor clarifications. Co-authored-by: David Baker Co-authored-by: Travis Ralston --- proposals/3952-intentional-mentions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 2b9394f8101..856b2320b85 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -11,7 +11,7 @@ Some situations that result in unintentional mentions include: * Replying to a message will re-issue pings from the initial message due to [fallback replies](https://spec.matrix.org/v1.5/client-server-api/#fallbacks-for-rich-replies). - * A user without the power level to send `@room` can abuse this by includin + * A user without the power level to send `@room` can abuse this by including `@room` in a message and getting a user with the appropriate power levels to reply to them. * Each time a message is edited the new version will be re-evaluated for mentions. @@ -191,7 +191,7 @@ push rules are to be deprecated. To avoid unintentional mentions these rules are modified to only apply when the `m.mentions` property is missing; clients should provide at least an empty `m.mentions` property on -every message to avoid the unintentional mentions discussed above. +every message to avoid the unintentional mentions discussed in the introduction. A future room version may wish to disable the legacy push rules: clients would no longer be required to include the `m.mentions` property on every event. It From ec29452be861e940ba0e04ac889dad626978f2c9 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 21 Mar 2023 12:13:34 -0400 Subject: [PATCH 41/55] Clarify client behavior. Co-authored-by: Travis Ralston --- proposals/3952-intentional-mentions.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 856b2320b85..17c039e7294 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -87,11 +87,11 @@ It is recommended that homeservers reject locally created events with an invalid `m.mentions` property with an error with a status code of `400` and an errcode of `M_INVALID_PARAM`. -Clients should add a Matrix ID to the `user_ids` array whenever composing a message which -includes an intentional [user mention](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions) -(often called a "pill"). Clients should set the `room` value to `true` when making a -room-wide announcement. Clients may also set these values at other times when it is -obvious the user intends to explicitly mention a user.[^3] +Clients add a Matrix ID to the `user_ids` array whenever composing a message which +includes an intentional mention, such as a ["pill"](https://spec.matrix.org/v1.5/client-server-api/#user-and-room-mentions). +Clients set the `room` value to `true` when making a room-wide announcement. Clients +should also set these values at other times when it is obvious the user intends to explicitly +mention a user.[^3] The `m.mentions` property is part of the plaintext event body and should be encrypted into the ciphertext for encrypted events. From c151958357220c8983fd008b8a3a376689f38b40 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 21 Mar 2023 12:14:02 -0400 Subject: [PATCH 42/55] All dependencies have been merged! Co-authored-by: Travis Ralston --- proposals/3952-intentional-mentions.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 17c039e7294..ba0c6d9305a 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -538,8 +538,7 @@ it can choose to apply the deprecation logic discussed in the ## Dependencies -This depends on the following MSCs which, at the time of writing, have not yet been accepted -into the specification: +This depends on the following MSCs: * [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758): Add `event_property_is` push rule condition kind * [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): event_match dotted keys From b9756431eecd2edebe3d4b71da3b0d703a115c7d Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 21 Mar 2023 13:58:48 -0400 Subject: [PATCH 43/55] Clarifications on deprecations * client behavior. --- proposals/3952-intentional-mentions.md | 38 ++++++++++++++++++-------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index ba0c6d9305a..7eb97ff5bd1 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -68,9 +68,9 @@ of the message should be colored differently: ## Proposal -The existing push rules for mentions are deprecated and replaced with rules -which use a property specific for mentions[^2], making the search for a mention -much simpler and more reliable for the user. +The existing push rules for user and room mentions are deprecated and new rules, +which use a property specific for mentions[^2], are added to make mentions simpler +and more reliable for users. ### New event property @@ -182,6 +182,22 @@ and `.m.rule.is_room_mention` is provided below: } ``` +### Client behavior + +The overall user experience is not modified, beyond improving explicitness and +reducing unintended mentions. + +For example, it is common that a client will show an event with a mention in a +different color (and denote the current user's "pill", as a way of showing the +user *why* they were mentioned). This behavior is unchanged. + +There are two variations that clients should take into account when decorating +messages for mentions, however: + +* The presence of a user's "pill" in a message no longer implies it is a mention. +* This makes it easier to mention users without including their "pill" in a + message (see [Abuse Potential](#abuse-potential) for ideas to combat this). + ### Backwards compatibility The [`.m.rule.contains_display_name`](https://spec.matrix.org/v1.5/client-server-api/#default-override-rules), @@ -198,6 +214,14 @@ no longer be required to include the `m.mentions` property on every event. It maybe convenient to do this when extensible events are adopted (see [MSC3932](https://github.com/matrix-org/matrix-spec-proposals/pull/3932)). +After acceptance, it is likely for there to be disagreement about which push rules are +implemented: legacy clients and homeservers may not yet have deprecated the +`.m.rule.contains_display_name`, `.m.rule.contains_user_name`, and `.m.rule.roomnotif` +push rules, while up-to-date clients and homeservers will support the +`.m.rule.is_user_mention` and `.m.rule.is_room_mention` push rules. It is expected +that both sets of push rules will need to be supported for a period of time, but +at worst case should simply result in the current behavior (documented in the preamble). + If users wish to continue to be notified of messages containing their display name it is recommended that clients create a specific keyword rule for this, e.g. a `content` rule of the form: @@ -220,14 +244,6 @@ it is recommended that clients create a specific keyword rule for this, e.g. a } ``` -After acceptance, it is likely for there to be disagreement about which push rules are -implemented: legacy clients and homeservers may not yet have deprecated the -`.m.rule.contains_display_name`, `.m.rule.contains_user_name`, and `.m.rule.roomnotif` -push rules, while up-to-date clients and homeservers will support the -`.m.rule.is_user_mention` and `.m.rule.is_room_mention` push rules. It is expected -that both sets of push rules will need to be supported for a period of time, but -at worst case should simply result in the current behavior (documented in the preamble). - ### Impact on replies Users are notified of replies via the `.m.rule.contains_display_name` or the From 4147590646cfd217f9d318a79c2cf85dd2b4afee Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 21 Mar 2023 14:09:53 -0400 Subject: [PATCH 44/55] Clarify the two properties used when editing events. --- proposals/3952-intentional-mentions.md | 22 ++++++++++++++++++---- 1 file changed, 18 insertions(+), 4 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 7eb97ff5bd1..ddd1736f2ac 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -312,11 +312,18 @@ Similarly to [replies](#impact-on-replies), users are notified of message edits via the `.m.rule.contains_display_name` or the `.m.rule.contains_user_name` push rule matching the [fallback content](https://spec.matrix.org/v1.6/client-server-api/#event-replacements). Generally this is undesirable and users do not need to be notified for the same -message multiple times (e.g. if a user is fixing a typo). It is recommended that -clients include an empty `m.mentions` property when editing an event, *unless* -the edit is significant or if additional users are mentioned. +message multiple times (e.g. if a user is fixing a typo). -The full list of mentioned users should be included in the `m.new_content` property. +Edited events end up with two `m.mentions` properties: + +* One at the top-level of the `content`, this should contain any users to mention + *for this edit*. +* One inside the `m.new_content` property, which should contain the full list of + mentioned users in any version of the event. + +It is recommended that clients use an empty top-level `m.mentions` property when +editing an event, *unless* the edit is significant or if additional users are +mentioned. For example, if there is an event: @@ -467,6 +474,13 @@ It is possible to add additional properties to the `m.mentions` object, e.g. a f usecase would be a `roles` property which could include values such as `admins` or `mods`. Defining this in detail is left to a future MSC. +### Cancelling notifications + +It maybe useful for a future MSC to investigate cancelling notifications if a +user's mentioned is removed while [editing events](#impact-on-edits). This could +be quite difficult as it is unclear if the mentioned user has already receive +the notification or not. + ## Alternatives ### Prior proposals From 59b5fa9061aafa212ebb743b00ee0743d8846395 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 21 Mar 2023 17:09:08 -0400 Subject: [PATCH 45/55] Fix typos. Co-authored-by: Hubert Chathi --- proposals/3952-intentional-mentions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index ddd1736f2ac..a7fe2a61188 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -477,8 +477,8 @@ usecase would be a `roles` property which could include values such as `admins` ### Cancelling notifications It maybe useful for a future MSC to investigate cancelling notifications if a -user's mentioned is removed while [editing events](#impact-on-edits). This could -be quite difficult as it is unclear if the mentioned user has already receive +user's mention is removed while [editing events](#impact-on-edits). This could +be quite difficult as it is unclear if the mentioned user has already received the notification or not. ## Alternatives From 725772f682c42e54b7d96346d4958726d93485a8 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Mar 2023 10:09:36 -0400 Subject: [PATCH 46/55] Clarify bridging section. --- proposals/3952-intentional-mentions.md | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index a7fe2a61188..42d128a791e 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -384,15 +384,16 @@ or potentially leverage [MSC3664](https://github.com/matrix-org/matrix-spec-prop ### Impact on bridging -This should strengthen the bridging contract as it will allow them to stop -mutating the content of messages during the bridging process (by directly -listing any mentioned users in the `m.mentions` property). - -This should reduce the impedance mismatch when bridging protocols which similarly -list out users to mention (the bridge should simply be able to map from the -remote user ID to the bridged user ID)[^4], but will not help in cases where the -mention information is embedded into the text content (as today's Matrix messages -are).[^5] +For protocols with a similar mechanism for listing mentioned users this should +strengthen the bridging contract as it enables bridges to stop mutating the +content of messages. The bridge should be able to map from the remote user ID +to the bridged user ID and include that in the `m.mentions` property of the +Matrix event & the proper field in the bridged protocol[^4]. + +For bridged protocols that do not have this mechanism than the bridge will only +be able to stop mutating content on messages bridged *into* Matrix. Messages +bridged out of Matrix will still need to embed the mention into the text +content.[^5] ## Potential issues From 16251bd4d2897328d3b40478fe5b31998f705b64 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Mar 2023 10:10:15 -0400 Subject: [PATCH 47/55] Clarify dependencies have been accepted. --- proposals/3952-intentional-mentions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 42d128a791e..e3aae23105d 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -569,7 +569,7 @@ it can choose to apply the deprecation logic discussed in the ## Dependencies -This depends on the following MSCs: +This depends on the following (accepted) MSCs: * [MSC3758](https://github.com/matrix-org/matrix-spec-proposals/pull/3758): Add `event_property_is` push rule condition kind * [MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873): event_match dotted keys From 1d1c8f3159134248cc62e111ad9be9b09a403560 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Wed, 22 Mar 2023 11:01:15 -0400 Subject: [PATCH 48/55] Explicit processing of mentioned messages. --- proposals/3952-intentional-mentions.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index e3aae23105d..783ad138275 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -382,6 +382,17 @@ to receive notifications for edits of events they were mentioned in then they could setup a push rule for the `content.m\\.new_content.m\\.mentions` property or potentially leverage [MSC3664](https://github.com/matrix-org/matrix-spec-proposals/pull/3664). +This implies that: + +* If a client highlights a message visually (e.g. by coloring it red), then it + should look at the `m.mentions` under `m.new_content` for edited messages. + Otherwise, in the example above, Alice would not see the message as red, even + though the intent was for her to be mentioned. +* Any sort of processing of push rules, e.g. to display a notification (sound, + toast, push notification), should occur without any special rules. I.e. the + `.m.rule.is_user_mention` and `.m.rule.is_room_mention` should look at the + `m.mentions` directly under `content` and not match for Alice. + ### Impact on bridging For protocols with a similar mechanism for listing mentioned users this should From 71a9e99cb627dc0305663cb92aa9a9e01cca09a2 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Tue, 28 Mar 2023 13:34:08 -0400 Subject: [PATCH 49/55] Minor clarifications. Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com> --- proposals/3952-intentional-mentions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 783ad138275..e992633b6a8 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -314,9 +314,9 @@ rule matching the [fallback content](https://spec.matrix.org/v1.6/client-server- Generally this is undesirable and users do not need to be notified for the same message multiple times (e.g. if a user is fixing a typo). -Edited events end up with two `m.mentions` properties: +Replacement events may have `m.mentions` properties in two locations: -* One at the top-level of the `content`, this should contain any users to mention +* One at the top-level of the `content`, which should contain any users to mention *for this edit*. * One inside the `m.new_content` property, which should contain the full list of mentioned users in any version of the event. @@ -372,7 +372,7 @@ And an edit after realizing that Bob is also in the room: } ``` -Mentions can also be removed as part of an edit, in this case top-level `m.mentions` +Mentions can also be removed as part of an edit. In this case top-level `m.mentions` property would not include the removed user IDs (you cannot cancel the notification from the previous event) or any previously notified users, and the removed user would also be removed from the `m.new_content` proprerty's copy of `m.mentions`. From 147ff7803ca927e8ff132a34125ac18c3766a1e7 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 20 Apr 2023 12:40:37 -0400 Subject: [PATCH 50/55] Cross-link to MSC3996 and clarifications. --- proposals/3952-intentional-mentions.md | 19 +++++++++++++++++-- 1 file changed, 17 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index e992633b6a8..494a045424a 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -432,6 +432,20 @@ benefits to using a separate property for mentions: Overall this proposal seems to be neutral or positive in the ability to combat malicious behavior. +### Encrypted mentions & `/notifications` + +Previous version of this proposal (and the alternative [MSC1796](https://github.com/matrix-org/matrix-spec-proposals/pull/1796)) +suggested leaving the `m.mentions` property in cleartext. This was +[deemed too large of a metadata leak](https://github.com/matrix-org/matrix-spec-proposals/pull/3952#discussion_r1112154200) +and removed from this proposal (and MSC1796 was closed). A downside of this is +that homeservers (still) will not be able to differentiate between notifications +and mentions in many cases. + +This mostly affects how often homeservers push to devices (see +[MSC3996](https://github.com/matrix-org/matrix-spec-proposals/pull/3996) +for more information), but also means that the `/notifications?only=highlight` +API is not useful in encrypted rooms. + ## Future extensions ### Combating abuse @@ -467,7 +481,8 @@ for the client to handle this feature and it must be handled on the server, unfo this would not be possible with the current proposal since the `m.mentions` property is encrypted (and the server cannot act on it). -Solving this problem is left to a future MSC. +Solving this problem is left to a future MSC, such as [MSC3996](https://github.com/matrix-org/matrix-spec-proposals/pull/3996) +which builds on this proposal. ### Pillifying `@room` @@ -488,7 +503,7 @@ usecase would be a `roles` property which could include values such as `admins` ### Cancelling notifications -It maybe useful for a future MSC to investigate cancelling notifications if a +It might be useful for a future MSC to investigate cancelling notifications if a user's mention is removed while [editing events](#impact-on-edits). This could be quite difficult as it is unclear if the mentioned user has already received the notification or not. From 9c6ae32e8e10a64b31be0b93fdc013bdc913dd4d Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 20 Apr 2023 14:18:05 -0400 Subject: [PATCH 51/55] Clean-up information about editing. Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3952-intentional-mentions.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 494a045424a..52d9d47ddf7 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -319,11 +319,12 @@ Replacement events may have `m.mentions` properties in two locations: * One at the top-level of the `content`, which should contain any users to mention *for this edit*. * One inside the `m.new_content` property, which should contain the full list of - mentioned users in any version of the event. + mentioned users in any version of the event, unless a mention is removed + (see below). It is recommended that clients use an empty top-level `m.mentions` property when editing an event, *unless* the edit is significant or if additional users are -mentioned. +mentioned in the latest version. For example, if there is an event: @@ -332,7 +333,7 @@ For example, if there is an event: "sender": "@dan:example.org", "event_id": "$initial_event", "content": { - "body": "Helo Alice!", + "body": "Hello Alice!", "m.mentions": { "user_ids": ["@alice:example.org"] } @@ -372,7 +373,7 @@ And an edit after realizing that Bob is also in the room: } ``` -Mentions can also be removed as part of an edit. In this case top-level `m.mentions` +Mentions can also be removed as part of an edit. In this case, the top-level `m.mentions` property would not include the removed user IDs (you cannot cancel the notification from the previous event) or any previously notified users, and the removed user would also be removed from the `m.new_content` proprerty's copy of `m.mentions`. From 69e99c4c3e259966649a357de12eb9cebd7db275 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 20 Apr 2023 14:19:32 -0400 Subject: [PATCH 52/55] Minor clarifications. Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com> --- proposals/3952-intentional-mentions.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 52d9d47ddf7..186aa5372fe 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -402,7 +402,7 @@ content of messages. The bridge should be able to map from the remote user ID to the bridged user ID and include that in the `m.mentions` property of the Matrix event & the proper field in the bridged protocol[^4]. -For bridged protocols that do not have this mechanism than the bridge will only +For bridged protocols that do not have this mechanism, the bridge will only be able to stop mutating content on messages bridged *into* Matrix. Messages bridged out of Matrix will still need to embed the mention into the text content.[^5] @@ -435,7 +435,7 @@ malicious behavior. ### Encrypted mentions & `/notifications` -Previous version of this proposal (and the alternative [MSC1796](https://github.com/matrix-org/matrix-spec-proposals/pull/1796)) +A previous version of this proposal (and the alternative [MSC1796](https://github.com/matrix-org/matrix-spec-proposals/pull/1796)) suggested leaving the `m.mentions` property in cleartext. This was [deemed too large of a metadata leak](https://github.com/matrix-org/matrix-spec-proposals/pull/3952#discussion_r1112154200) and removed from this proposal (and MSC1796 was closed). A downside of this is From 811c0df5ecbc65333ae99ec3f825ce1175d9c885 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 20 Apr 2023 14:52:09 -0400 Subject: [PATCH 53/55] Add note about double backslashes. --- proposals/3952-intentional-mentions.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 186aa5372fe..48e52b9a639 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -98,7 +98,7 @@ into the ciphertext for encrypted events. ### New push rules -Two new default push rule are added: +Two new default push rule are added. The `.m.rule.is_user_mention` override push rule would appear directly before the `.m.rule.contains_display_name` push rule: @@ -128,6 +128,10 @@ before the `.m.rule.contains_display_name` push rule: } ``` +(Note: `\\.` would become a single logical backslash followed by a dot since the +above is in JSON-representation. See +[MSC3873](https://github.com/matrix-org/matrix-spec-proposals/pull/3873).) + The `.m.rule.is_room_mention` override push rule would appear directly before the `.m.rule.roomnotif` push rule: From 63592e7f63a2f007af91bf6739463a40a3c9c791 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Fri, 21 Apr 2023 11:15:02 -0400 Subject: [PATCH 54/55] Clarify behavior of replies. --- proposals/3952-intentional-mentions.md | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 48e52b9a639..8c5bba68fac 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -255,10 +255,15 @@ Users are notified of replies via the `.m.rule.contains_display_name` or the [rich reply fallback](https://spec.matrix.org/v1.6/client-server-api/#fallbacks-for-rich-replies). Unfortunately these push rules will be disabled for events which contain the `m.mentions` property, i.e. all newly created events (see -[above](#backwards-compatibility)). It is proposed that clients should include -the sender of the event being replied to as well as any users (except themself) -mentioned in that event in the new event's `m.mentions` property. The `room` -property should not be copied over. +[above](#backwards-compatibility)). Clients should include the sender of the event +being replied to as well as any mentioned users in that event (excluding yourself) +in the new event's `m.mentions` property. The `room` property should not be copied over. + +This signals that it is the *intention* of the sender to mention all of those people. +This behavior may not make sense in all situations (e.g. an email-like client could +provide both a "reply" and "reply all", while a microblogging client may wish to +provide a "quote reply", dropping all mentions from the original event) and clients +may wish to allow users to modify the list of mentioned users. For example, if there is an event: @@ -284,9 +289,9 @@ And a reply from Alice: "body": "> <@dan:example.org> Alice: Have you heard from Bob?\n\nNo, but I saw him with Charlie earlier.", "m.mentions": { "user_ids": [ - // Include the sender of $initial_event. + // Include the sender of $initial_event (optional). "@dan:example.org", - // The users mentioned, minus yourself. + // The users mentioned, minus yourself (optional). "@bob:example.org", // New mentions, as normal. "@charlie:example.org" @@ -302,10 +307,6 @@ And a reply from Alice: } ``` -This signals that it is the *intention* of the sender to mention all of those people, -clients may wish to allow users to modify the list of people to include, e.g. to -"quote reply" as opposed to replying directly. - If a user wishes to be notified of *all replies* to their messages, other solutions should be investigated, such as [MSC3664](https://github.com/matrix-org/matrix-spec-proposals/pull/3664). This would give more equal power to both senders and receivers of events. From e0ebbf8af451ead5ab470438364a0eb133f1e917 Mon Sep 17 00:00:00 2001 From: Patrick Cloke Date: Thu, 27 Apr 2023 08:14:40 -0400 Subject: [PATCH 55/55] Clarify that rooms-mentions should not be propogated. Co-authored-by: Alexey Rusakov --- proposals/3952-intentional-mentions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/3952-intentional-mentions.md b/proposals/3952-intentional-mentions.md index 8c5bba68fac..99fd4175d7c 100644 --- a/proposals/3952-intentional-mentions.md +++ b/proposals/3952-intentional-mentions.md @@ -257,7 +257,7 @@ Unfortunately these push rules will be disabled for events which contain the `m.mentions` property, i.e. all newly created events (see [above](#backwards-compatibility)). Clients should include the sender of the event being replied to as well as any mentioned users in that event (excluding yourself) -in the new event's `m.mentions` property. The `room` property should not be copied over. +in the new event's `m.mentions` property. The `room` property MUST NOT be copied over. This signals that it is the *intention* of the sender to mention all of those people. This behavior may not make sense in all situations (e.g. an email-like client could