optimize query pattern used by storage filter

[icewind1991]

To filter searches based on the storages a user has access to search queries often include multiple (storage = $1 and path like "$2/%") (for folder shares/groupfolders) and (storage = $1 and path = $2) (for single file shares) parts.

This attempts to optimize those queries by

• grouping multiple storage comparisons together: (storage = $1 and $path like $2) or (storage = $1 and $path like $3) -> storage = $1 and ($path like $2 or $path like $3)
• combining multiple = on the same field with an in: (path = $1 or path = $2 or $path = 3) -> path in ($1,$2,$3)

todo:

• chunk large in statements

[icewind1991]

An example search query with a number of single file and folder shares from 2 users and a few groupfolders

From

SELECT `file`.`fileid`,
       `storage`,
       `path`,
       `path_hash`,
       `file`.`parent`,
       `file`.`name`,
       `mimetype`,
       `mimepart`,
       `size`,
       `mtime`,
       `storage_mtime`,
       `encrypted`,
       `etag`,
       `permissions`,
       `checksum`,
       `unencrypted_size`
FROM `*PREFIX*filecache` `file`
WHERE (`file`.`name` ILIKE :"%tesa%")
  AND (((`storage` = :1) AND
        ((`path` = :"files") OR (`path` LIKE :"files\/%"))) OR
       ((`storage` = :2) AND ((`path` = :"__groupfolders\/1") OR
                              (`path` LIKE :"\\_\\_groupfolders\/1\/%"))) OR
       ((`storage` = :2) AND ((`path` = :"__groupfolders\/2") OR
                              (`path` LIKE :"\\_\\_groupfolders\/2\/%"))) OR
       ((`storage` = :4) AND (`path` = :"files\/cv.md")) OR
       ((`storage` = :3) AND (`path` = :"files\/boilerplate.md")) OR
       ((`storage` = :3) AND ((`path` = :"files\/testfolder") OR
                              (`path` LIKE :"files\/testfolder\/%"))) OR
       ((`storage` = :3) AND (`path` = :"files\/oc-react.md")) OR
       ((`storage` = :3) AND (`path` = :"files\/Patcher.md")) OR
       ((`storage` = :3) AND (`path` = :"files\/storage.md")) OR
       ((`storage` = :3) AND (`path` = :"files\/welcome.txt")) OR
       ((`storage` = :3) AND (`path` = :"files\/oc-news-feed.md")) OR
       ((`storage` = :4) AND (`path` = :"files\/rssrun.md")) OR
       ((`storage` = :4) AND (`path` = :"files\/Replay.md")) OR
       ((`storage` = :4) AND (`path` = :"files\/regen.md")) OR
       ((`storage` = :3) AND ((`path` = :"files\/sharefolder2") OR
                              (`path` LIKE :"files\/sharefolder2\/%"))) OR
       ((`storage` = :3) AND ((`path` = :"files\/sharefolder3") OR
                              (`path` LIKE :"files\/sharefolder3\/%"))))
ORDER BY `mtime` + :0 desc LIMIT 5

to

SELECT `file`.`fileid`,
       `storage`,
       `path`,
       `path_hash`,
       `file`.`parent`,
       `file`.`name`,
       `mimetype`,
       `mimepart`,
       `size`,
       `mtime`,
       `storage_mtime`,
       `encrypted`,
       `etag`,
       `permissions`,
       `checksum`,
       `unencrypted_size`
FROM `*PREFIX*filecache` `file`
WHERE (`file`.`name` ILIKE :"%tes%")
  AND (((`storage` = :1) AND
        ((`path` = :"files") OR (`path` LIKE :"files\/%"))) OR
       ((`storage` = :2) AND
        ((`path` IN (:["__groupfolders\/1", "__groupfolders\/2"])) OR
         (`path` LIKE :"\\_\\_groupfolders\/1\/%") OR
         (`path` LIKE :"\\_\\_groupfolders\/2\/%"))) OR ((`storage` = :4) AND
                                                         (`path` IN
                                                          (:["files\/cv.md",
                                                           "files\/rssrun.md",
                                                           "files\/Replay.md",
                                                           "files\/regen.md"]))) OR
       ((`storage` = :3) AND ((`path` IN
                               (:["files\/boilerplate.md", "files\/testfolder",
                                "files\/oc-react.md", "files\/Patcher.md",
                                "files\/storage.md", "files\/welcome.txt",
                                "files\/oc-news-feed.md", "files\/sharefolder2",
                                "files\/sharefolder3"])) OR
                              (`path` LIKE :"files\/testfolder\/%") OR
                              (`path` LIKE :"files\/sharefolder2\/%") OR
                              (`path` LIKE :"files\/sharefolder3\/%"))))
ORDER BY `mtime` + :0 desc LIMIT 5

[XueSheng-GIT]

@icewind1991 Any plans for an updated pull for NC28? Search for files is back to more than 60 seconds on my instance (thus, search is practically unusable).