Skip to content

Commit

Permalink
Replace "Pagination" section in C-S API with text in appendices (#3366)
Browse files Browse the repository at this point in the history
The Pagination section in the C-S API was, basically, full of rubbish. I think that anything of any value it contained was repeated either directly on the API definitions or in the text specific to syncing at https://spec.matrix.org/unstable/client-server-api/#syncing.

The conventions I've added to the Appendices are based on the discussions in #1898. They are there because I don't want to have to go through it all again next time we add a paginated API.

Fixes: #1898
Fixes: #2268
  • Loading branch information
richvdh authored Aug 27, 2021
1 parent da51f7f commit 5e3ea11
Show file tree
Hide file tree
Showing 4 changed files with 29 additions and 84 deletions.
File renamed without changes.
1 change: 1 addition & 0 deletions changelogs/client_server/newsfragments/3366.clarification
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Remove the inaccurate 'Pagination' section.
30 changes: 28 additions & 2 deletions content/appendices.md
Original file line number Diff line number Diff line change
Expand Up @@ -1064,8 +1064,8 @@ The event signing algorithm should emit the following signed event:
## Conventions for Matrix APIs

This section is intended primarily to guide API designers when adding to Matrix,
setting guidelines to follow for how those APIs should work. This is important to
maintain consistency with the Matrix protocol, and thus improve developer
setting guidelines to follow for how those APIs should work. This is important to
maintain consistency with the Matrix protocol, and thus improve developer
experience.

### HTTP endpoint and JSON property naming
Expand All @@ -1079,3 +1079,29 @@ The key names in JSON objects passed over the API also follow this convention.
There are a few historical exceptions to this rule, such as `/createRoom`.
These inconsistencies may be addressed in future versions of this specification.
{{% /boxes/note %}}

### Pagination

REST API endpoints which can return multiple "pages" of results should adopt the
following conventions.

* If more results are available, the endpoint should return a property named
`next_batch`. The value should be a string token which can be passed into
a subsequent call to the endpoint to retrieve the next page of results.

If no more results are available, this is indicated by *omitting* the
`next_batch` property from the results.

* The endpoint should accept a query-parameter named `from` which the client
is expected to set to the value of a previous `next_batch`.

* Some endpoints might support pagination in two directions (example:
`/messages`, which can be used to move forward or backwards in the timeline
from a known point). In this case, the endpoint should return a `prev_batch`
property which can be passed into `from` to receive the previous page of
results.

Avoid having a separate "direction" parameter, which is generally redundant:
the tokens returned by `next_batch` and `prev_batch` should contain enough
information for subsequent calls to the API to know which page of results
they should return.
82 changes: 0 additions & 82 deletions content/client-server-api/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -1214,88 +1214,6 @@ using an `unstable` version.
When this capability is not listed, clients should use `"1"` as the
default and only stable `available` room version.

## Pagination

{{% boxes/note %}}
The paths referred to in this section are not actual endpoints. They
only serve as examples to explain how pagination functions.
{{% /boxes/note %}}

Pagination is the process of dividing a dataset into multiple discrete
pages. Matrix makes use of pagination to allow clients to view extremely
large datasets. These datasets are not limited to events in a room (for
example clients may want to paginate a list of rooms in addition to
events within those rooms). Regardless of what is being paginated, there
is a common approach which is used to give clients an easy way of
selecting subsets of a potentially changing dataset. Each endpoint that
uses pagination may use different parameters. However the theme among
them is that they take a `from` and `to` token, and occasionally a
`limit` and `dir`. Together, these parameters describe the position in a
data set, where `from` and `to` are known as "stream tokens" matching
the regular expression `[a-zA-Z0-9.=_-]+`. If supported, the `dir`
defines the direction of events to return: either forwards (`f`) or
backwards (`b`). The response may contain tokens that can be used for
retrieving results before or after the returned set. These tokens may be
called <span class="title-ref">start</span> or <span
class="title-ref">prev\_batch</span> for retrieving the previous result
set, or <span class="title-ref">end</span>, <span
class="title-ref">next\_batch</span> or <span
class="title-ref">next\_token</span> for retrieving the next result set.

In the following examples, 'START' and 'END' are placeholders to signify
the start and end of the data sets respectively.

For example, if an endpoint had events E1 -&gt; E15. The client wants
the last 5 events and doesn't know any previous events:

```
S E
|-E1-E2-E3-E4-E5-E6-E7-E8-E9-E10-E11-E12-E13-E14-E15-|
| | |
| _____| <--backwards-- |
|__________________ | | ________|
| | | |
GET /somepath?to=START&limit=5&dir=b&from=END
Returns:
E15,E14,E13,E12,E11
```

Another example: a public room list has rooms R1 -&gt; R17. The client
is showing 5 rooms at a time on screen, and is on page 2. They want to
now show page 3 (rooms R11 -&gt; 15):

```
S E
| 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | stream token
|-R1-R2-R3-R4-R5-R6-R7-R8-R9-R10-R11-R12-R13-R14-R15-R16-R17| room
|____________| |________________|
| |
Currently |
viewing |
|
GET /roomslist?from=9&to=END&limit=5
Returns: R11,R12,R13,R14,R15
```

Note that tokens are treated in an *exclusive*, not inclusive, manner.
The end token from the initial request was '9' which corresponded to
R10. When the 2nd request was made, R10 did not appear again, even
though from=9 was specified. If you know the token, you already have the
data.

Responses for pagination-capable endpoints SHOULD have a `chunk` array
alongside the applicable stream tokens to represent the result set.

In general, when the end of a result set is reached the applicable
stream token will be excluded from the response. For example, if a user
was backwards-paginating events in a room they'd eventually reach the
first event in the room. In this scenario, the `prev_batch` token would
be excluded from the response. Some paginated endpoints are open-ended
in one direction, such as endpoints which expose an event stream for an
active room. In this case, it is not possible for the client to reach
the true "end" of the data set and therefore should always be presented
with a token to keep moving forwards.

## Filtering

Filters can be created on the server and can be passed as a parameter to
Expand Down

0 comments on commit 5e3ea11

Please sign in to comment.