Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wrong user quota in the user administration #38874

Closed
5 of 8 tasks
Tracked by #2686
ghost opened this issue Jun 18, 2023 · 54 comments · Fixed by #40605 or #42186
Closed
5 of 8 tasks
Tracked by #2686

wrong user quota in the user administration #38874

ghost opened this issue Jun 18, 2023 · 54 comments · Fixed by #40605 or #42186

Comments

@ghost
Copy link

ghost commented Jun 18, 2023

⚠️ This issue respects the following points: ⚠️

Bug description

Entering 10GB will allocate 10.7GB as the storage size. 1GB becomes 1.1GB, 500 becomes 575.5GB.

Steps to reproduce

  1. Go to user management
    2.Specify an Arbitrary size for the user disk space and see how it becomes a different size.

Expected behavior

Installation method

Community Docker image

Nextcloud Server version

27

Operating system

None

PHP engine version

None

Web server

None

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

  • Default user-backend (database)
  • LDAP/ Active Directory
  • SSO - SAML
  • Other

Configuration report

No response

List of activated Apps

No response

Nextcloud Signing status

No response

Nextcloud Logs

No response

Additional info

No response

@ghost ghost added 0. Needs triage Pending check for reproducibility or if it fits our roadmap bug labels Jun 18, 2023
@jalil-salame
Copy link

I can replicate this using the NixOS configuration for nextcloud27

@LuXc-NL
Copy link

LuXc-NL commented Jun 30, 2023

Encountered the same issue..

@dgedgedge
Copy link

More or less same issue :

I have a user with 850GB of files.
If I try to put 1000GB then it goes to unlimited and / or 1GB...
I saw both.

Server configuration detail

Operating system: Linux 6.1.0-9-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.27-1 (2023-05-08) x86_64

Webserver: Apache (fpm-fcgi)

Database: mysql 10.11.3

PHP version: 8.2.7

Modules loaded: Core, date, libxml, openssl, pcre, zlib, filter, hash, json, random, Reflection, SPL, session, standard, sodium, cgi-fcgi, mysqlnd, PDO, xml, apcu, bcmath, bz2, calendar, ctype, curl, dom, mbstring, FFI, fileinfo, ftp, gd, gettext, gmp, iconv, igbinary, imagick, intl, exif, msgpack, mysqli, pdo_mysql, Phar, posix, readline, redis, shmop, SimpleXML, sockets, sysvmsg, sysvsem, sysvshm, tokenizer, xmlreader, xmlwriter, xsl, zip, memcached, Zend OPcache

Nextcloud version: 26.0.3 - 26.0.3.2

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: unknown

Signing status

[]

List of activated apps

Enabled:

  • activity: 2.18.0
  • admin_audit: 1.16.0
  • analytics: 4.9.4
  • announcementcenter: 6.6.1
  • bookmarks: 13.0.1
  • bruteforcesettings: 2.6.0
  • calendar: 4.4.3
  • carnet: 0.24.7
  • checksum: 1.2.2
  • circles: 26.0.0
  • cloud_federation_api: 1.9.0
  • collectives: 2.5.0
  • comments: 1.16.0
  • contacts: 5.3.2
  • contactsinteraction: 1.7.0
  • cookbook: 0.10.2
  • dashboard: 7.6.0
  • dav: 1.25.0
  • deck: 1.9.2
  • drawio: 2.1.1
  • federatedfilesharing: 1.16.0
  • files: 1.21.1
  • files_external: 1.18.0
  • files_mindmap: 0.0.28
  • files_pdfviewer: 2.7.0
  • files_rightclick: 1.5.0
  • files_sharing: 1.18.0
  • files_trashbin: 1.16.0
  • files_versions: 1.19.1
  • firstrunwizard: 2.15.0
  • geoblocker: 0.5.11
  • keeweb: 0.6.13
  • logreader: 2.11.0
  • lookup_server_connector: 1.14.0
  • mail: 3.2.3
  • maps: 1.0.2
  • music: 1.8.4
  • nextcloud_announcements: 1.15.0
  • notes: 4.8.0
  • notifications: 2.14.0
  • oauth2: 1.14.0
  • password_policy: 1.16.0
  • photos: 2.2.0
  • previewgenerator: 5.3.0
  • privacy: 1.10.0
  • provisioning_api: 1.16.0
  • recommendations: 1.5.0
  • related_resources: 1.1.0-alpha1
  • serverinfo: 1.16.0
  • settings: 1.8.0
  • sharebymail: 1.16.0
  • spreed: 16.0.4
  • support: 1.9.0
  • survey_client: 1.14.0
  • systemtags: 1.16.0
  • tasks: 0.15.0
  • text: 3.7.2
  • theming: 2.1.1
  • twofactor_backupcodes: 1.15.0
  • updatenotification: 1.16.0
  • user_status: 1.6.0
  • viewer: 1.10.0
  • weather_status: 1.6.0
  • workflowengine: 2.8.0
    Disabled:
  • encryption
  • federation: 1.10.1
  • suspicious_login: 4.3.0
  • twofactor_totp
  • user_ldap
Configuration (config/config.php)

{
"instanceid": "REMOVED SENSITIVE VALUE",
"passwordsalt": "REMOVED SENSITIVE VALUE",
"secret": "REMOVED SENSITIVE VALUE",
"trusted_domains": [


],
"datadirectory": "***REMOVED SENSITIVE VALUE***",
"overwrite.cli.url": "******",
"dbtype": "mysql",
"version": "26.0.3.2",
"dbname": "***REMOVED SENSITIVE VALUE***",
"dbhost": "***REMOVED SENSITIVE VALUE***",
"dbport": "",
"dbtableprefix": "oc_",
"dbuser": "***REMOVED SENSITIVE VALUE***",
"dbpassword": "***REMOVED SENSITIVE VALUE***",
"loglevel": 2,
"logtimezone": "Europe\/Paris",
"logfile": "\/var\/log\/nextcloud.log",
"log_rotate_size": 0,
"installed": true,
"memcache.local": "\\OC\\Memcache\\APCu",
"share_folder": "\/SharedWithMe",
"mail_from_address": "***REMOVED SENSITIVE VALUE***",
"mail_smtpmode": "smtp",
"mail_domain": "***REMOVED SENSITIVE VALUE***",
"mail_smtphost": "***REMOVED SENSITIVE VALUE***",
"maintenance": false,
"theme": "",
"default_phone_region": "FR",
"updater.release.channel": "stable",
"mail_smtpauth": 1,
"mail_sendmailmode": "smtp",
"mail_smtpauthtype": "LOGIN",
"mail_smtpname": "***REMOVED SENSITIVE VALUE***",
"mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
"mysql.utf8mb4": true,
"trashbin_retention_obligation": "auto, 100",
"has_rebuilt_cache": true,
"app_install_overwrite": [
    "files_reader"
],
"bulkupload.enabled": false

}

Cron Configuration: Array
(
[backgroundjobs_mode] => cron
[lastcron] => 1688364922
)

External storages: yes

External storage configuration

No mounts configured

Encryption: no

User-backends:

OC\User\Database

Talk configuration:

STUN servers


TURN servers

turn:*******- udp,tcp

Signaling servers (mode: default):

no custom server configured

Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:109.0) Gecko/20100101 Firefox/114.0

@Zeroth42
Copy link

Zeroth42 commented Jul 4, 2023

Is there a fix for that? I have the same problem.

@albig
Copy link

albig commented Jul 4, 2023

1. Entering manually "50 GB" ...

2023-07-04_16-33

2. ... results in "unlimited"

2023-07-04_16-33_1

3. ... which is shown as "0" after saving.

2023-07-04_16-33_2


Selecting 10 GB from drop down results in 10,7 GB.

2023-07-04_16-36

@Zeroth42
Copy link

Zeroth42 commented Jul 4, 2023

If you calculate, it is always 7.4% higher than that what you type in

@dgedgedge
Copy link

dgedgedge commented Jul 5, 2023

I have it illimited :
Capture d’écran 2023-07-05 à 09 27 39

Previously, it was consistent to have a warning because I gave the account only 900GB.
I tried to put 1000GB, but nextcloud understood 1GB => I did put it back to unlimited.

Overall is also OK :

Capture d’écran 2023-07-05 à 09 32 20

Note: It's an installation upgraded from release to release since multiple years.
Recently I upgraded from bullsseye to bookworm then the nextcloud.

@kurjajuur666
Copy link

kurjajuur666 commented Jul 6, 2023

Got the same issue on a couple of Nextcloud 27 instances and an instance running 27.0.0 RC1.
e: same in 27.0.1rc1

@florianwgnr
Copy link

Same thing for me, all quotas set to 10GB are 10.7GB after the update. Also i noticed that more space is used. Did something within the calculation change?

@arnowelzel
Copy link
Contributor

arnowelzel commented Jul 19, 2023

It looks like the value entered will be treated as binary based values (2^x) but the display will be the decimal based value (10^x). Also when editing the quota value, the input field becomes empty (at least in my installation with NC 27) even when the quota is saved.

For example:

10 GB can be interpreted as decimal based value which is 10*1000*1000*1000 Bytes = 10,000,000,000 Bytes.

However it can also be: 10*1024*1024*1024 Bytes = 10,737,418,240 Bytes.

When the resulting value is then displayed as "decimal based" GB it will be become "10,7 GB" even the user has entered "10 GB" as value.

I would suggest to interpret units like "GB" or "MB" always as binary based since many operating systems and applications see "1 GB" as 1*1024*1024*1024 and not as 1*1000*1000*1000 - even if this is not completely correct since the unit for binary based values is "GiB" and not "GB". If no unit is entered but only a plain number, then treat the number as it is but still display it as "KB", "MB", "GB" depending on how large the value is.

So when I input "10 GB" this should mean 10*1024*1024*1024 and when I input "1000000" this will mean 1000000 bytes or 976.56 KB (maybe we also do a internal rounding for a maximum of two digits for fractional parts).

About displaying the quota values:

  • When the quota is less then 1*1024 bytes: display as it is, e.g. "500".
  • When the quota is between 1*1024 and 1024*1024-1 bytes: display value/1024 as "KB", e.g. 2048 -> "2 KB" (2048/1024)
  • When quota is between 1024*1024 and 1024*1024*1024-1 bytes: display value/1024/1024 as "MB", e.g. 52428800 -> "50 MB" (52428800/1024/1024)
  • When quota is 1024*1024*1204 or more: display value/1024/1024/1024 as "GB", e.g. 10737418240 -> "10 GB" (10737418240/1024/1024/1024)

@arnowelzel
Copy link
Contributor

1. Entering manually "50 GB" ...

2023-07-04_16-33

2. ... results in "unlimited"

2023-07-04_16-33_1

3. ... which is shown as "0" after saving.

2023-07-04_16-33_2

Selecting 10 GB from drop down results in 10,7 GB.

2023-07-04_16-36

Nextcloud will store the limit corectly, just the initial view update is wrong. So if you enter "50 GB", press enter and then save the user, it will display "unlimited" first - but when you refresh the user list, you will see "53,68 GB" as value (because 50 GB input will be interpreted as 50*1024*1024*1024 bytes but the display will then show the value devided as 50/1000/1000/1000.

@microniko
Copy link

Nextcloud will store the limit corectly, just the initial view update is wrong. So if you enter "50 GB", press enter and then save the user, it will display "unlimited" first - but when you refresh the user list, you will see "53,68 GB" as value (because 50 GB input will be interpreted as 50*1024*1024*1024 bytes but the display will then show the value devided as 50/1000/1000/1000.
You're right but it is not very intuitive ;-)

@szaimen
Copy link
Contributor

szaimen commented Jul 19, 2023

cc @nextcloud/server

@susnux
Copy link
Contributor

susnux commented Jul 19, 2023

I noticed that unit issue too, GB entered will be treated as GiB (gibibyte, base 2 (1024³)) but shown as GB (gigabyte, base 10 (1000³)).

@bguzzardi
Copy link

I tried changing my quota a few different ways on Nextcloud 27.0.1 with macOS client version 3.9.1.

input method input value Admin GUI Files app occ mac client
Admin GUI 1 TB 1.2 TB 1.2 TB 1.1 TB 1,126 GB
Admin GUI 1000 GB 1.2 TB 1.2 TB 1.1 TB 1,126 GB
Admin GUI 1000000000000 1.1 TB 1.1 TB 1 TB 1,024 GB
Admin GUI 1000000000000 B 1.1 TB 1.1 TB 1 TB 1,024 GB
occ 1TB 1.1 TB 1.1 TB 1TB 1,024 GB
occ "1\ TB" 1.1 TB 1.1 TB 1 TB 1,024 GB
occ "1\ TiB" 1 B 1 B 1 TiB 1 B
occ 1000000000000 1 TB 1 TB 1000000000000 931 GB
occ "1000000000000\ B" 1 TB 1 TB 1000000000000 B 931 GB

Using the web interface never gives a correct result: "1 TB" is of course ambiguous; but while the user could have meant 1000 GB or 1024 GiB, "1,126 GB" is wrong. "1000000000000 B" is unambiguous; it's 10^12 bytes, 1000 GB, or 931 GiB, but "1,024 GB" is wrong.

Using occ is better: If I set "1 TB" it's reasonable if not technically correct to get "1,024 GB" in the desktop client.

The only unambiguous way to set a quota, then, appears to be using occ and a value in bytes.

occ user:setting <username> files quota 1000000000000

does the right thing, but the results are displayed inconsistently. The web UI displays the technically correct "1 TB" while the desktop client shows the incorrect-but-excusable "931 GB."

Anyway, I hope this is helpful to someone. Please let me know if I can be of assistance.

@ThowZzy
Copy link

ThowZzy commented Aug 3, 2023

Just want to add that its not only the quota, but its just the way its shown on the nextcloud website.
For example a folder that is really "99,7GB" on Windows will show "107,1GB" on the website.

@Pilzinsel64
Copy link

Yea, this happens not only in admin user settings page but also in Files app on user quota display and folder display.
But good news: On the users account profile settings the quota is shown correctly!

@ThowZzy
Copy link

ThowZzy commented Aug 3, 2023

Yea, this happens not only in admin user settings page but also in Files app on user quota display and folder display. But good news: On the users account profile settings the quota is shown correctly!

You'r right

@manjarooo
Copy link

manjarooo commented Aug 7, 2023

I have the same issues since I've updated Nextcloud from version 26 to 27...
When I enter "100 GB" for example as user quota,

  • the user quota input field jumps to "unlimited" after pressing "Enter", but seems to store a value (if correct, I don't know)
  • the "files" app says "70.7 GB of 114,9 GB used"
  • the user account profile (Personal info > Details) says "You are using 65.8 GB of 107 GB"

So both UI outputs are wrong, or different from what I've entered.

But looking into the real "data/user/files" folder with 'ncdu' and 'du -sh', I get:
ncdu : 65,8 GiB
du -sh : 66G

So yes, the user account UI seems "more correct", but I didn't enter 107 GB as quota, I did enter 100 GB! And even if it wouldn't matter because of just a different calculation, I think this is confusing for most users.

Bye... :-)

@jjasoncool
Copy link

This issue doen't in milestone 27.0.2
https://github.com/nextcloud/server/milestone/230
I don't know the detail, but is this hard to fix? Or this is php8.2's issue?

@arnowelzel
Copy link
Contributor

I don't know the detail, but is this hard to fix? Or this is php8.2's issue?

No, it does not have anything to do with PHP 8.2. It happens with PHP 8.1 as well.

@bguzzardi
Copy link

bguzzardi commented Aug 7, 2023

My PHP is extremely rusty, and I've never written TypeScript or Vue before. I think I can submit a patch for this, but I would like some advice on strategy.

Here's what I think is happening:

  1. There's a humanFileSize() which converts from a number of bytes like 1048576 to a pretty string like "1 MB"
  2. There's a computerFileSize() which does the inverse, it takes "1 MB" and gives "1048576".
  3. There are PHP and TypeScript version of each of these.
  4. The TypeScript version of humanFileSize() was replaced by a reference to "formatFileSize" from "@nextcloud/files" which uses SI prefixes, not binary ones.
  5. When a quota is set in the web interface, the input value is given to computerFileSize() as a validity check, then the result of that conversion is given to humanFileSize() to go back to human-readable, and that result is given to the server, which repeats the same process (in PHP this time).

Example:

  1. Admin sets a user's quota to "1 GB"
  2. (JS) computerFileSize("1 GB") gives "1073741824" (This function uses 2^30 bytes per GB)
  3. (JS) humanFileSize("1073741824") gives "1.1 GB" (This function uses 10^9 bytes per GB)
  4. "1.1 GB" is sent to the backend to be set as the user's quota.
  5. (PHP) computerFileSize("1.1 GB") gives "1181116006" (This function uses 2^30 bytes per GB)
  6. (PHP) humanFileSize("1181116006") gives "1.2 GB" "1.1 GB" (This function uses 2^30 bytes per GB)
  7. "1.2 GB" "1.1 GB" is stored as the user's quota.

Edit: The bad conversion only happens once before the value is stored, not twice.

See:
core/src/OC/util.js:69
core/src/OC/util.js:81
lib/private/legacy/OC_Helper.php:80
lib/private/legacy/OC_Helper.php:117
apps/settings/src/components/Users/UserSettingsDialog.vue:437

It seems that the possible solutions are:

  1. Perform the computerFileSize() conversion, but send that result to the backend before the humanFileSize conversion. If the user enters "1 GB" send "1000000000 B" or "1073741824 B" to the backend.
  2. Update the TypeScript and PHP versions of computerFileSize and the PHP version of humanFileSize to use 1000 instead of 1024.
  3. Same as (2), but add support for KiB/MiB/GiB etc.
  4. Revert formatFileSize to 1024-multiples.

I personally like (1) because the frontend and backend are both computers, so it seems that there's no real reason for them to be exchanging human-friendly units.

@manjarooo
Copy link

@jjasoncool:

This issue doen't in milestone 27.0.2

Good to know, then maybe (hopefully) it is already fixed in version 27.0.2...?
I am using stable updates, so I'm still on version 27.0.1.

@bguzzardi:
Thank you for digging into the source code...! :-)
I fully agree, also would prefer solution (1).

@ThowZzy
Copy link

ThowZzy commented Aug 10, 2023

Still not fixed on 27.0.2 ...

@arnowelzel
Copy link
Contributor

Still not fixed on 27.0.2 ...

Sorry - but there is no need to comment about missing fixes. If the issue will be fixed, then you will see at least a pull request here for the code changes and issue itself would be closed as well.

@l3ochan
Copy link

l3ochan commented Sep 18, 2023

#39500

I can confirm it's my exact issue. Thanks for mentioning it.

@AndyXheli
Copy link

Still an issue on NC 27.1.1

@susnux
Copy link
Contributor

susnux commented Sep 25, 2023

Fixed with #40605

@AndyXheli
Copy link

Hey @susnux just a quick update so i updated to NC 27.1.2 and the user quota under the user tab is now looking good but the quote under this section is still wrong

image

@ThowZzy
Copy link

ThowZzy commented Oct 5, 2023

Hey @susnux just a quick update so i updated to NC 27.1.2 and the user quota under the user tab is now looking good but the quote under this section is still wrong

The fix should have been for the entire webui, because even the values in the file apps are wrong, can you confirm the sizes in your files app are also wrong? And that only the quota page is good

@AndyXheli
Copy link

Hmm so under the IOS app it now shows 1TB and that is correct

Android app shows 1024GB and that should be 1TB

I cleared browser cache too and same results.

@susnux
Copy link
Contributor

susnux commented Oct 5, 2023

@AndyXheli that happens because the files app is not using the global function but the library directly.
There will be a follow up.

@dinosmm
Copy link

dinosmm commented Oct 27, 2023

So this issue is not fixed yet, shouldn't it be marked open again? I have the same on 27.1.2. The quota in the user admin page is correct, but the quota on the files app is incorrect as per above descriptions.

@ryester19
Copy link

So this issue is not fixed yet, shouldn't it be marked open again? I have the same on 27.1.2. The quota in the user admin page is correct, but the quota on the files app is incorrect as per above descriptions.

Newest is 27.1.3, and it's fixed over here

@ryester19
Copy link

The issue has come back for the user's sidebar when I upgraded to 28.0.0. The admin's user page is still displaying the quota correctly

@AndyXheli
Copy link

Same mine is showing 180.4 GB used from 1.1TB

Admin portal shows 168 used from 1TB the admin portal is the correct one

@susnux
Copy link
Contributor

susnux commented Dec 13, 2023

yes for the files app this is a regression of 32c1aeb

@Menelion
Copy link

Menelion commented Oct 9, 2024

Sorry, what's the current status of this issue? for me, if I enter 100 GB manually, it returns 93.1 GB. the only way to have it at 100 Gb is to enter 1073741824, i.e. 1024 * 1024 * 1024.

@RikkiBC
Copy link

RikkiBC commented Oct 9, 2024

Sorry, what's the current status of this issue? for me, if I enter 100 GB manually, it returns 93.1 GB. the only way to have it at 100 Gb is to enter 1073741824, i.e. 1024 * 1024 * 1024.

This originally affected me too but was fixed for a long time, the version after the commit linked by Susnux when they closed this in December 15 2023 (just above your comment). I just tested and it's still fixed for me now too.

@Menelion - what version of Nextcloud are you on?

@Menelion
Copy link

Menelion commented Oct 9, 2024

@RikkiBC I'm on the latest version, I've just installed it three days ago or so. From system info text file:

Nextcloud version: 30.0.0 - 30.0.0.14

And the Files app is at 2.0.0.

@RikkiBC
Copy link

RikkiBC commented Oct 10, 2024

@RikkiBC I'm on the latest version, I've just installed it three days ago or so. From system info text file:

Nextcloud version: 30.0.0 - 30.0.0.14

And the Files app is at 2.0.0.

Ah! Might be worth opening another issue then. I'm on latest stable AIO version (Nextcloud 29.0.7) so if you're getting it now it's worth discussing in a new bug report separately as a regression I believe.

@susnux
Copy link
Contributor

susnux commented Oct 10, 2024

I can not reproduce this issue is still fixed also for 30.

And the Files app is at 2.0.0.

This then sounds more like your update process is broken, files is shipped with Nextcloud and in Nextcloud 30 the files version is 2.2.0.

@Menelion
Copy link

Oh sorry, my bad, it's 2.2.0 actually:

Enabled:
  - activity: 3.0.0
  - bruteforcesettings: 3.0.0
  - circles: 30.0.0-dev
  - cloud_federation_api: 1.13.0
  - comments: 1.20.1
  - contactsinteraction: 1.11.0
  - dashboard: 7.10.0
  - dav: 1.31.1
  - federatedfilesharing: 1.20.0
  - federation: 1.20.0
  - files: 2.2.0
  - files_downloadlimit: 3.0.0
[...]

@susnux
Copy link
Contributor

susnux commented Oct 10, 2024

Latest 30.0.1 RC2:

Bildschirmaufnahme_20241010_155340.webm

I can not reproduce, maybe try to clear the cache. But this should already be fixed long ago.

@Menelion
Copy link

@susnux I can't see the video since I'm blind, but I believe you of course. How can we debug? It's a fresh installation, Apache, MariaDB, Ubuntu, performed with install script.

@susnux
Copy link
Contributor

susnux commented Oct 11, 2024

Basically what I am doing in the video is:

  1. Go to account settings
  2. Set quota of user to "100 GB" (it is then already shown as 100GB in the settings)
  3. Go to files app
  4. The quota information in the app navigation is showing "0B of 100GB used" as expected

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment