Cannot use search_after with field collapse

[jorge-marques]

The elasticsearch documentation indicates that it is possible to use search_after with field collapse, as long the results are sorted exclusively by the collapsing field.

Is this possible with spring-data-elasticsearch? A cursory look at the code suggests it's not possible as _shard_doc is always added to the sort criteria as a tiebreaker

[sothawo]

I assume you are talking about calls to ReactiveSearchOperations.search(Query,...) as this is the only place where in the implementation search_after is used?

So you are setting a FieldCollapse on a NativeQuery and want to add a Sort for that same field and that fails then?

The solution would probably be to not add the shard_doc when there is a FieldCollapse set with a field name being the same as the one in the only Sort passed in.

[jorge-marques]

That's exactly right, elasticsearch returns the following error:
Cannot use [collapse] in conjunction with [search_after] unless the search is sorted on the same field. Multiple sort fields are not allowed

I believe your proposed solution would work well.

[sothawo]

Implemented on main branch and backported to current 5.3.x.