PostgreSQL => MySQL db:convert-type with calendar/contacts #3179

daishi · 2017-01-20T03:54:36Z

Steps to reproduce

Have a Nextcloud installation using PostgreSQL with calendar and/or contacts data.
Convert the database from PostgresQL to MySQL using the steps described in https://docs.nextcloud.com/server/10/admin_manual/configuration_database/db_conversion.html.
The conversion does not show errors, and appears to succeed.

Expected behaviour

Contacts and calendar entries should be converted/transferred to MySQL.

Actual behaviour

Contacts and calendars are not correctly migrated.

nextcloud.log contains (truncated, more complete version pasted below):

"Exception: {\"Message\":\"This parser\
 only supports VCARD and VCALENDAR files\",\"Exception\":\"Sabre\\\\VObject\\\\ParseException\",\"Code\":0,\"Trace\":\"#0 \\\
\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/vobject\\\/lib\\\/Parser\\\/MimeDir.php(63): Sabre\\\\VObject\\\\Pa\
rser\\\\MimeDir->parseDocument()\\n#1 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/vobject\\\/lib\\\/Reader.ph\
p(44): Sabre\\\\VObject\\\\Parser\\\\MimeDir->parse('Resource id #13...', 0)

Inspecting in the converted MySQL database I see:

mysql> select calendardata from oc_calendarobjects limit 10;
+-------------------+
| calendardata      |
+-------------------+
| Resource id #1392 |
| Resource id #749  |
| Resource id #750  |
| Resource id #751  |
| Resource id #752  |
| Resource id #753  |
| Resource id #754  |
| Resource id #755  |
| Resource id #756  |
| Resource id #757  |
+-------------------+
10 rows in set (0.00 sec)

A similar inspection in the original PostgreSQL installation shows actual calendar data.

Server configuration

Operating system:

Debian GNU/Linux 7.11 (wheezy)

Web server:

apache2: 2.2.22-13+deb7u7

Database:

postgresql: 9.1+134wheezy4
mysql: 10.1.20+maria-1~jessie (running in docker)

PHP version:

libapache2-mod-php5: 5.4.45-0+deb7u6

Nextcloud version: (see Nextcloud admin page)

10.0.1.

Updated from an older Nextcloud/ownCloud or fresh install:

Updated from ownCloud (incrementally since v6 or v7, I don't remember)

Where did you install Nextcloud from:

Previously from .deb provided by owncloud. Now from the nextcloud-provided .zip.

Signing status:

Signing status

No errors have been found.

List of activated apps:

App list

Enabled:
  - activity: 2.3.2
  - admin_audit: 1.0.0
  - calendar: 1.4.1
  - comments: 1.0.0
  - contacts: 1.4.0.0
  - dav: 1.0.1
  - federatedfilesharing: 1.0.1
  - federation: 1.0.1
  - files: 1.5.2
  - files_pdfviewer: 0.8.1
  - files_sharing: 1.0.0
  - files_texteditor: 2.1
  - files_trashbin: 1.0.0
  - files_versions: 1.3.0
  - files_videoplayer: 0.9.8
  - firstrunwizard: 1.1
  - gallery: 15.0.0
  - notifications: 0.3.0
  - password_policy: 1.0.0
  - provisioning_api: 1.0.0
  - serverinfo: 1.1.1
  - survey_client: 0.1.5
  - systemtags: 1.0.2
  - templateeditor: 0.1
  - theming: 1.0.1
  - updatenotification: 1.0.1
  - workflowengine: 1.0.1
Disabled:
  - encryption
  - external
  - files_accesscontrol
  - files_automatedtagging
  - files_external
  - files_retention
  - user_external
  - user_ldap
  - user_saml

The content of config/config.php:

Config report

This is the original PostgreSQL configuration.

{
    "system": {
        "instanceid": "51d12709dea51",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "datadirectory": "\/local\/app\/owncloud\/data",
        "dbtype": "pgsql",
        "version": "9.1.1.5",
        "dbname": "owncloud",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "forcessl": true,
        "theme": "",
        "maintenance": false,
        "trusted_domains": [
            "emilb.us",
            "cloud.emilb.us"
        ],
        "secret": "***REMOVED SENSITIVE VALUE***",
        "loglevel": 0,
        "forceSSLforSubdomains": true,
        "updatechecker": false,
        "trashbin_retention_obligation": "auto"
    }
}

Are you using external storage, if yes which one: local

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Client configuration

Browser:

Chrome

Operating system:

Logs

Web server error log

``` Insert your webserver log here ```

Nextcloud log (data/nextcloud.log)

Nextcloud log

``` {"reqId":"F232wjooymJEr73NESwi","remoteAddr":"50.152.147.22","app":"webdav","message":"Exception: {\"Message\":\"This parser only supports VCARD and VCALENDAR files\",\"Exception\":\"Sabre\\\\VObject\\\\ParseException\",\"Code\":0,\"Trace\":\"#0 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/vobject\\\/lib\\\/Parser\\\/MimeDir.php(63): Sabre\\\\VObject\\\\Parser\\\\MimeDir->parseDocument()\\n#1 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/vobject\\\/lib\\\/Reader.php(44): Sabre\\\\VObject\\\\Parser\\\\MimeDir->parse('Resource id #13...', 0)\\n#2 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/CalDAV\\\/Backend\\\/AbstractBackend.php(142): Sabre\\\\VObject\\\\Reader::read('Resource id #13...')\\n#3 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/apps\\\/dav\\\/lib\\\/CalDAV\\\/CalDavBackend.php(819): Sabre\\\\CalDAV\\\\Backend\\\\AbstractBackend->validateFilterForObject(Array, Array)\\n#4 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/apps\\\/dav\\\/lib\\\/CalDAV\\\/Calendar.php(229): OCA\\\\DAV\\\\CalDAV\\\\CalDavBackend->calendarQuery('3', Array)\\n#5 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/CalDAV\\\/Plugin.php(600): OCA\\\\DAV\\\\CalDAV\\\\Calendar->calendarQuery(Array)\\n#6 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/CalDAV\\\/Plugin.php(236): Sabre\\\\CalDAV\\\\Plugin->calendarQueryReport(Object(Sabre\\\\CalDAV\\\\Xml\\\\Request\\\\CalendarQueryReport))\\n#7 [internal function]: Sabre\\\\CalDAV\\\\Plugin->report('{urn:ietf:param...', Object(Sabre\\\\CalDAV\\\\Xml\\\\Request\\\\CalendarQueryReport), 'calendars\\\/daish...')\\n#8 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#9 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/CorePlugin.php(716): Sabre\\\\Event\\\\EventEmitter->emit('report', Array)\\n#10 [internal function]: Sabre\\\\DAV\\\\CorePlugin->httpReport(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#11 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/event\\\/lib\\\/EventEmitterTrait.php(105): call_user_func_array(Array, Array)\\n#12 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(459): Sabre\\\\Event\\\\EventEmitter->emit('method:REPORT', Array)\\n#13 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/dav\\\/lib\\\/DAV\\\/Server.php(248): Sabre\\\\DAV\\\\Server->invokeMethod(Object(Sabre\\\\HTTP\\\\Request), Object(Sabre\\\\HTTP\\\\Response))\\n#14 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/apps\\\/dav\\\/lib\\\/Server.php(184): Sabre\\\\DAV\\\\Server->exec()\\n#15 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/apps\\\/dav\\\/appinfo\\\/v2\\\/remote.php(30): OCA\\\\DAV\\\\Server->exec()\\n#16 \\\/var\\\/www\\\/nextcloud-10.0.1\\\/remote.php(165): require_once('\\\/var\\\/www\\\/nextcl...')\\n#17 {main}\",\"File\":\"\\\/var\\\/www\\\/nextcloud-10.0.1\\\/3rdparty\\\/sabre\\\/vobject\\\/lib\\\/Parser\\\/MimeDir.php\",\"Line\":125,\"User\":\"daishi\"}","level":4,"time":"2017-01-19T01:41:17+00:00","method":"REPORT","url":"\/remote.php\/dav\/calendars\/daishi\/personal_shared_by_deborah\/","user":"daishi"} ```

Browser log

``` Insert your browser log here, this could for example include:

a) The javascript console log
b) The network log
c) ...

</details>

The text was updated successfully, but these errors were encountered:

ghost · 2017-02-22T04:02:31Z

I can confirm this is still occurring on Nextcloud 11. Logs not attached because they're almost exactly the same as @daishi's

Server configuration

Operating system:

Debian GNU/Linux 8.7 (jessie)

Web server:

apache2: 2.4.10-10+deb8u7

Database:

postgresql: 9.4+165+deb8u2
mysql (MariaDB): 10.0.29-0+deb8u1

PHP version:

libapache2-mod-php5: 5.6.30+dfsg-0+deb8u1

Nextcloud version: (see Nextcloud admin page)

11.0.1.2

Updated from an older Nextcloud/ownCloud or fresh install:

Updated from Nextcloud 10

Where did you install Nextcloud from:

bz2 package from NC website

EDIT: I've been able to restore my contacts and calendars by deleting the relating rows in the new DB and then restoring from ics/vcf. SQL below:

DELETE FROM oc_calendarchanges;
DELETE FROM oc_calendarobjects;
DELETE FROM oc_calendars;
DELETE FROM oc_calendarsubscriptions;
DELETE FROM oc_cards
DELETE FROM oc_cards_properties;

nickvergessen · 2017-03-26T15:21:25Z

I just tested conversion from sqlite -> postgres -> mysql -> postgres -> mysql
And they all worked fine.

Since you seem to already have fixed the problem, I will close this issue until it happens for a second user as well.

daishi · 2017-03-26T22:52:28Z

@nickvergessen, the issue was reported by both @danielcotton and me. It sounds to me like @danielcotton worked around the issue by manually removing the DB rows and importing the data from ICS files.

When you tested the conversion how much calendar and contact data did you have in your installation?

nickvergessen · 2017-03-30T12:30:26Z

Maybe I had still #4071 checked out. It should help to solve this issue too. Can you try it?

nickvergessen added bug feature: install and update labels Jan 20, 2017

nickvergessen closed this as completed Mar 26, 2017

nextcloud-bot mentioned this issue Jul 9, 2018

db:convert-mysql-charset and MariaDB 10.3 #10163

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

PostgreSQL => MySQL db:convert-type with calendar/contacts #3179

PostgreSQL => MySQL db:convert-type with calendar/contacts #3179

daishi commented Jan 20, 2017

ghost commented Feb 22, 2017 •

edited by ghost

Loading

nickvergessen commented Mar 26, 2017

daishi commented Mar 26, 2017

nickvergessen commented Mar 30, 2017

PostgreSQL => MySQL db:convert-type with calendar/contacts #3179

PostgreSQL => MySQL db:convert-type with calendar/contacts #3179

Comments

daishi commented Jan 20, 2017

Steps to reproduce

Expected behaviour

Actual behaviour

Server configuration

Client configuration

Logs

Web server error log

Nextcloud log (data/nextcloud.log)

Browser log

ghost commented Feb 22, 2017 • edited by ghost Loading

Server configuration

Operating system:

Web server:

Database:

PHP version:

Nextcloud version: (see Nextcloud admin page)

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from:

nickvergessen commented Mar 26, 2017

daishi commented Mar 26, 2017

nickvergessen commented Mar 30, 2017

ghost commented Feb 22, 2017 •

edited by ghost

Loading