Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some users can't login after upgrade from 9.0.4 to 9.1.0 #25616

Closed
ksmets opened this issue Jul 26, 2016 · 18 comments
Closed

Some users can't login after upgrade from 9.0.4 to 9.1.0 #25616

ksmets opened this issue Jul 26, 2016 · 18 comments
Assignees
@PVince81
Labels
performance sev1-critical Type:Bug

Comments

@ksmets
Copy link

ksmets commented Jul 26, 2016

Steps to reproduce

  1. sudo apt-get upgrade owncloud owncloud-files owncloud-deps-php5
  2. sudo -u www-data php occ upgrade
  3. sudo -u www-data php occ maintenance:mode --off

Expected behaviour

All user should be able to login.

Actual behaviour

Some users land on a 502 page after successfully login in.

Server configuration

Operating system:
Ubuntu 14.04.4 LTS

Web server:
nginx 1.4.6-1ubuntu3.5

Database:
mysql Ver 14.14 Distrib 5.5.50, for debian-linux-gnu (x86_64) using readline 6.3

PHP version:
PHP 5.5.9-1ubuntu4.17 (cli) (built: May 19 2016 19:05:57)

ownCloud version: (see ownCloud admin page)
9.1.0

Updated from an older ownCloud or fresh install:
9.0.4

Where did you install ownCloud from:
apt-get

Signing status (ownCloud 9.0 and above):
No errors have been found.

List of activated apps:

Enabled:
  - activity: 2.3.2
  - calendar: 1.2.2
  - comments: 0.3.0
  - dav: 0.2.5
  - federatedfilesharing: 0.3.0
  - federation: 0.1.0
  - files: 1.5.1
  - files_pdfviewer: 0.8.1
  - files_sharing: 0.10.0
  - files_texteditor: 2.1
  - files_trashbin: 0.9.0
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 15.0.0
  - notifications: 0.3.0
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - templateeditor: 0.1
  - updatenotification: 0.2.1
Disabled:
  - encryption
  - external
  - files_antivirus
  - files_external
  - user_external
  - user_ldap

The content of config/config.php:

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "owncloud.bloomingfeld.com"
        ],
        "datadirectory": "\/home\/owncloud\/data",
        "overwrite.cli.url": "https:\/\/owncloud.bloomingfeld.com",
        "dbtype": "mysql",
        "version": "9.1.0.15",
        "dbname": "owncloud",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "UTC",
        "installed": true,
        "instanceid": "oce3vg2opr4k",
        "mail_smtpmode": "smtp",
        "mail_from_address": "noreply",
        "mail_domain": "bloomingfeld.com",
        "loglevel": "1",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "updatechecker": false,
        "has_internet_connection": true,
        "maintenance": false,
        "theme": "",
        "htaccess.RewriteBase": "\/"
    }
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Logs

Web server error log

2016/07/26 12:36:17 [error] 4350#0: *314 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 84.198.87.64, server: owncloud.bloomingfeld.com, request: "POST /index.php/login HTTP/1.1", upstream: "fastcgi://unix:/var/run/php5-fpm.sock:", host: "owncloud.bloomingfeld.com"

php5-fpm.log

26-Jul-2016 12:28:38] NOTICE: [pool www] child 5029 started
[26-Jul-2016 12:36:18] WARNING: [pool www] child 5029 exited on signal 11 (SIGSEGV - core dumped) after 459.858090 seconds from start

coredump

ksmets@bf1:~$ gdb /usr/sbin/php5-fpm /tmp/coredump-php5-fpm.5029
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/php5-fpm...(no debugging symbols found)...done.
[New LWP 5029]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `php-fpm: pool www                                                       '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f5e5f726a4c in php_json_encode () from /usr/lib/php5/20121212/json.so
(gdb) bt
#0  0x00007f5e5f726a4c in php_json_encode () from /usr/lib/php5/20121212/json.so
#1  0x00007f5e5f72774b in ?? () from /usr/lib/php5/20121212/json.so
#2  0x00007f5e5f726b01 in php_json_encode () from /usr/lib/php5/20121212/json.so
#3  0x00007f5e5f727331 in ?? () from /usr/lib/php5/20121212/json.so
#4  0x00000000006dd35b in dtrace_execute_internal ()
#5  0x000000000079c2e0 in ?? ()
#6  0x0000000000716c18 in execute_ex ()
#7  0x00000000006dd259 in dtrace_execute_ex ()
#8  0x000000000079c930 in ?? ()
#9  0x0000000000716c18 in execute_ex ()
#10 0x00000000006dd259 in dtrace_execute_ex ()
#11 0x000000000079c930 in ?? ()
#12 0x0000000000716c18 in execute_ex ()
#13 0x00000000006dd259 in dtrace_execute_ex ()
#14 0x000000000079c930 in ?? ()
#15 0x0000000000716c18 in execute_ex ()
#16 0x00000000006dd259 in dtrace_execute_ex ()
#17 0x000000000079c930 in ?? ()
#18 0x0000000000716c18 in execute_ex ()
#19 0x00000000006dd259 in dtrace_execute_ex ()
#20 0x000000000079c930 in ?? ()
#21 0x0000000000716c18 in execute_ex ()
#22 0x00000000006dd259 in dtrace_execute_ex ()
#23 0x000000000079c930 in ?? ()
#24 0x0000000000716c18 in execute_ex ()
#25 0x00000000006dd259 in dtrace_execute_ex ()
#26 0x000000000079c930 in ?? ()
#27 0x0000000000716c18 in execute_ex ()
#28 0x00000000006dd259 in dtrace_execute_ex ()
#29 0x000000000079c930 in ?? ()
#30 0x0000000000716c18 in execute_ex ()
#31 0x00000000006dd259 in dtrace_execute_ex ()
#32 0x000000000079c930 in ?? ()
#33 0x0000000000716c18 in execute_ex ()
---Type <return> to continue, or q <ret
@ghost
Copy link

ghost commented Jul 26, 2016

@ksmets

Whatever the problem is, it probably does not lie with Owncloud as a script should in no circumstance cause an interpreter to segfault. The problem most likely lies with either your specific PHP version or an
extension you have installed, you could try to get a backtrace or a core dump to find out exactly where it segfaults.

Have you considered moving to PHP7? It is 'the future', after all. It
is maturing rapidly and works well on older hardware.

http://owncloud.10557.n7.nabble.com/php-fpm-hanging-segfault-SIGSEGV-CPUs-100-tp17535p17536.html

The user there solved such an issue with segmentation faults by updating some operating system components:

http://owncloud.10557.n7.nabble.com/php-fpm-hanging-segfault-SIGSEGV-CPUs-100-tp17535p17546.html

@ksmets
Copy link
Author

ksmets commented Jul 26, 2016

I installed php-fpm7.0, but this results in another coredump for the same problematic users:

ksmets@bf1 /tmp> gdb /usr/sbin/php-fpm7.0 coredump-php-fpm7.0.8840
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/php-fpm7.0...(no debugging symbols found)...done.
[New LWP 8840]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `php-fpm: pool www                                                            '.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f8a6adf51bc in dtrace_execute_ex ()
(gdb) bt
#0  0x00007f8a6adf51bc in dtrace_execute_ex ()
#1  0x00007f8a6ae7e510 in ?? ()
#2  0x00007f8a6ae40ddb in execute_ex ()
#3  0x00007f8a6adf5219 in dtrace_execute_ex ()
#4  0x00007f8a6adf6b53 in zend_call_function ()
#5  0x00007f8a6ae20964 in zend_call_method ()
#6  0x00007f8a6ae392ca in ?? ()
#7  0x00007f8a6ae68dda in ?? ()
#8  0x00007f8a6ae40ddb in execute_ex ()
#9  0x00007f8a6adf5219 in dtrace_execute_ex ()
#10 0x00007f8a6ae7e510 in ?? ()
#11 0x00007f8a6ae40ddb in execute_ex ()
#12 0x00007f8a6adf5219 in dtrace_execute_ex ()
#13 0x00007f8a6ae7e510 in ?? ()
#14 0x00007f8a6ae40ddb in execute_ex ()
#15 0x00007f8a6adf5219 in dtrace_execute_ex ()
#16 0x00007f8a6ae7e510 in ?? ()
#17 0x00007f8a6ae40ddb in execute_ex ()
#18 0x00007f8a6adf5219 in dtrace_execute_ex ()
#19 0x00007f8a6ae7e510 in ?? ()
#20 0x00007f8a6ae40ddb in execute_ex ()
#21 0x00007f8a6adf5219 in dtrace_execute_ex ()
#22 0x00007f8a6ae7e510 in ?? ()
#23 0x00007f8a6ae40ddb in execute_ex ()
#24 0x00007f8a6adf5219 in dtrace_execute_ex ()
#25 0x00007f8a6ae7e510 in ?? ()
#26 0x00007f8a6ae40ddb in execute_ex ()
#27 0x00007f8a6adf5219 in dtrace_execute_ex ()
#28 0x00007f8a6ae7e510 in ?? ()
#29 0x00007f8a6ae40ddb in execute_ex ()
#30 0x00007f8a6adf5219 in dtrace_execute_ex ()
#31 0x00007f8a6ae7e510 in ?? ()
#32 0x00007f8a6ae40ddb in execute_ex ()
#33 0x00007f8a6adf5219 in dtrace_execute_ex ()

@ksmets
Copy link
Author

ksmets commented Jul 26, 2016

I restored the database from before the upgrade and I'm able to reproduce the issue when upgrading from 9.0.4 to 9.1.0 with

  • both php5 as php7.0, and
  • with or without \OC\Memcache\APCu enabled,
  • upgrading via console or via webbrowser.

I highly suspect a data-driven issue. Any ideas how I can debug this issue further?

@ksmets
Copy link
Author

ksmets commented Jul 26, 2016

Note that when I log in with an administrator account, I can't see a listing of any users. So it seems also related to #25594.

@protree
Copy link

protree commented Jul 31, 2016

And maybe #25557

@takenek
Copy link

takenek commented Aug 1, 2016

Same problem here some users cant login to server, i cant list users via https://hostname/index.php/settings/users

Best Regards
Marek

@takenek
Copy link

takenek commented Aug 1, 2016 •
edited
Loading

This is more info from my instalation:
When i try login to user: TaKeN i get in logs:
owncloud.log
{"reqId":"HZKo8KxlPjp4Pv3h5V2Y","remoteAddr":"127.0.0.1","app":"core","message":"File with id "7146" has not been found.","level":3,"time":"2016-08-01T14:11:47+00:00","method":"POST","url":"/index.php/login?user=TaKeN","user":"TaKeN"}

{"reqId":"HZKo8KxlPjp4Pv3h5V2Y","remoteAddr":"127.0.0.1","app":"core","message":"File with id "6714" has not been found.","level":3,"time":"2016-08-01T14:11:47+00:00","method":"POST","url":"/index.php/login?user=TaKeN","user":"TaKeN"}

{"reqId":"HZKo8KxlPjp4Pv3h5V2Y","remoteAddr":"127.0.0.1","app":"PHP","message":"Allowed memory size of 536870912 bytes exhausted (tried to allocate 2003968 bytes) at /var/www/owncloud/apps/files_sharing/lib/sharedstorage.php#82","level":3,"time":"2016-08-01T14:12:05+00:00","method":"POST","url":"/index.php/login?user=TaKeN","user":"TaKeN"}

In my Apache error.log:
[Mon Aug 01 16:11:05 2016] [error] [client 127.0.0.1] PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 2003968 bytes) in /var/www/owncloud/apps/files_sharing/lib/sharedstorage.php on line 82
zend_mm_heap corrupted
[Mon Aug 01 16:12:05 2016] [error] [client 127.0.0.1] PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 2003968 bytes) in /var/www/owncloud/apps/files_sharing/lib/sharedstorage.php on line 82
zend_mm_heap corrupted

Any patch for test? got it this on VMWare so i can do snapshot and test it.

Best Regards

@SergioBertolinSG
Copy link
Contributor

Did the users which can't login have something in common?

Reshares? federated shares? something like that?

@ksmets
Copy link
Author

ksmets commented Aug 3, 2016

They all have shared files among each other. Possibly deleted some of those files.

@o87h
Copy link

o87h commented Aug 9, 2016

I appear to have run into this too. Ubuntu 15.10, PHP 5.6.11-1ubuntu3.4, nginx/1.11.3. Upgraded to 9.10 and some days later deleted shared folders without removing the share first.

@PVince81
Copy link
Contributor

PVince81 commented Aug 9, 2016

Very likely related to #25557

@PVince81
Copy link
Contributor

Please follow up here and try the patches on a test system: #25557 (comment)

The patches aren't production ready but it would be good to know if that's the issue you are having.

You could also try enabling the PHP XDebug module and see if it produces a useful stack trace with infinite loops / recursions. If yes, then your problem is #25557

@PVince81
Copy link
Contributor

The patches have been merged and will be available in tomorrow's dailies (and the upcoming RCs)

I'll keep this ticket open, hoping to get feedback whether the patches solved the issue for you or whether this is a completely different one.

@PVince81 PVince81 self-assigned this Aug 16, 2016
@PVince81
Copy link
Contributor

Are you guys still seeing this issue with 9.1.1 which contains the patches ?

@ksmets
Copy link
Author

ksmets commented Oct 24, 2016

Upgraded to ownCloud 9.1.1 (stable) without any problems.

@ksmets ksmets closed this as completed Oct 24, 2016
@466
Copy link

466 commented Nov 17, 2016 •
edited
Loading

Does 9.1.0 already contain this patch?

I encountered this problem today, and was unable to solve it till now.
I'm running owncloud 9.1.0 on a raspberry, with Raspbian 8 installed.

sudo -u www-data php occ -V
ownCloud version 9.1.0

sudo -u www-data php occ upgrade
ownCloud is already latest version

Only the admin user is able to login, all the other (4) users do get "502 Bad Gateway" once they login.
( a wrong password leads to reloading the start page)

I was syncing about 15GB of data, when this happened.
All users do share files with each other, I am not sure about the admin, though i think the admin user does not share stuff with anyone.

@ghost
Copy link

ghost commented Nov 17, 2016

@466

Does 9.1.0 already contain this patch?

No:

Are you guys still seeing this issue with 9.1.1 which contains the patches ?
#25616 (comment)

If you need further support please use https://central.owncloud.org/

@lock
Copy link

lock bot commented Aug 3, 2019

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@lock lock bot locked as resolved and limited conversation to collaborators Aug 3, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@PVince81 PVince81

Labels
performance sev1-critical Type:Bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@PVince81 @466 @ksmets @takenek @protree @SergioBertolinSG @o87h