[oc_filecache] Loading one million files

[dariodsa]

How to use GitHub

• Please use the 👍 reaction to show that you are interested into the same feature.
• Please don't comment if you have no relevant information to add. It's just extra noise for everyone subscribed to this issue.
• Subscribe to receive notifications on status change and new comments.

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

Describe the solution you'd like
A clear and concise description of what you want to happen.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

As you already know, Nextcloud has issues with loading a huge number of files. As I saw, reason is that for each file, Nextcloud does SELECT query on table oc_filecache. My first question i, what could be stored in oc_filecache that Nextcloud couldn't see from the disk? Second question, I would like to refactor a code so that if file_cache is set to 0 in config.php, Nextcloud wouldn't use oc_filecache at all. So I would like your opinion on that matter? That would probably load page slower, but in some case faster when number of files is huge.

Thanks,
Dario Sindicic

[dariodsa]

#23930

[SimplyCorbett]

Nextcloud is the metadata server for your files. All files use oc_filecache because it stores the file path among other details in that table.

I personally have not experienced issues with having a huge number of files. I've had millions of files on nextcloud with no issue.

The issue you linked to #23930 concerns loading a lot of files in one folder within the WebGUI... this is not the same as nextcloud being unable to handle millions of files.

Further, if you are using webdav with nextcloud you can easily load folders with millions of files. I've tested this on linux with dolphin. The problem is your (local) computer is going to have RAM and CPU usage through the roof trying to browse that folder.

tl;dr explain your problem in more detail.

[dariodsa]

I analyse more detailed my environment and I saw that I am having performance issue with file locking using Redis. When file locking is disabled, Nextcloud renders page really quickly, but when file locking is enabled it simply times out(on folder with more than 7000 files). Is there something special which I should put in my Redis configuration?

[szaimen]

Lets track this in #7312