Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Disable xinclude processing in eXide's "download app" command #90

Closed
dizzzz opened this issue Feb 25, 2015 · 7 comments · Fixed by #158
Closed

Disable xinclude processing in eXide's "download app" command #90

dizzzz opened this issue Feb 25, 2015 · 7 comments · Fixed by #158

Comments

@dizzzz
Copy link
Member

dizzzz commented Feb 25, 2015

Moved from eXist-db/exist#403, where @awagner-mainz suggested (a) disabling expansion of XIncludes in eXide's "download app" or (b) at least have an option to turn off XInclude expansion, and added:

After running into this problem for the fiftieth time (or so it feels), I have again lost my xi:include statements somewhere along the download/edit/distribute/deploy path and ended up with flat (and redundant) xml files. On the eXist-db mailing list, other persons have confirmed to have run into this.

@joewiz
Copy link
Member

joewiz commented Mar 12, 2017

The code controlling how apps are downloaded is https://github.com/wolfgangmm/eXide/blob/develop/modules/deployment.xql#L574-L581. It's simply using the compression:zip() function, which has no option for enabling/disabling XInclude.

I see two routes, then, to a solution:

  1. Request the addition of an XInclude flag in compression:zip.
  2. Instead of the current 1st parameter in compression:zip(xs:anyURI($collection), true(), $collection), we could construct all of the <entry> elements and use util:serialize() and its expand-xincludes=no option (see http://exist-db.org/exist/apps/doc/xquery.xml#serialization-10) to ensure xincludes are not suppressed. This option could be toggled in the UI.

Option 2 seems pretty doable, and doesn't require a change to the eXist core.

@awagner-mainz @jensopetersen @dizzzz Opinions? Is there any interest in taking this on?

@duncdrum
Copy link
Contributor

duncdrum commented Mar 12, 2017

Actually the current behavior is doing exactly what I want it to, I.e. It leaves xincludes where I put them. Auto expanding them would break the zip file size limits. Not sure what the OP is experiencing.

@awagner-mainz
Copy link

@duncdrum, my experience was different from yours: it did not leave xincludes but replaced the xinclude statements with the included stuff. I will have to check if it is still this way in v3, but I will not have the time to do so in the next few days. If it is as you say, then the issue is already solved from my POV.

@duncdrum
Copy link
Contributor

duncdrum commented Mar 15, 2017

@awagner-mainz ok I did another round of tests with a fresh 3.1 and yes my includes are being expanded, no clue what i did to my old 2.2. to not do this, but yes this is a problem.

@joewiz Actually there are at least two problems:

  1. it is auto-exapanding in the first place, there is a reason I used xIncludes.
  2. It doesn't delete the xInclude targets, so the app has one huge xml, and lots of duplicate data. Which means installing from this app generates different e.g. search results compared to a package generated via ant.

@joewiz
Copy link
Member

joewiz commented Mar 15, 2017

Yikes! Thanks for confirming!

I think we can give option 2 a try and see how it performs. I'll let you know when a patch / PR is ready to test.

joewiz added a commit that referenced this issue Mar 15, 2017
- exposes a new request parameter, expand-includes, which is now set to “false” by default but could be set to “true” by the application
- closes #90
@joewiz
Copy link
Member

joewiz commented Mar 15, 2017

@awagner-mainz @duncdrum I've submitted a PR. The changes are pretty easy to apply if you just copy and paste the revised modules/deployment.xql file into your own eXide. Attached is the .xar file I tested the PR on, to confirm that the xincludes are not expanded and that all file types are being properly zipped up into the downloaded .xar. (The file will download as xitest-0.1.zip, so rename it as xitest-0.1.xar; GitHub wouldn't accept a file with the .xar file extension.)

download this and rename it as xitest-0.1.xar

In the process of testing this, I also discovered that when eXide executes queries (via Eval), there is no way to prevent it from expanding XInclude elements that are part of the result sequence. (Try running Eval on the xitest.xq file in the attached app package.) Running the same query outside the context of eXist (e.g., by opening http://localhost:8080/exist/apps/xitest/test.xq in its own browser tab) respects the serialization options in the query. I believe eXide's behavior here is a limitation of the XQueryServlet that eXide uses to pipe queries through to eXist for execution. A side effect of this same phenomenon is that if you open an XML file and insert an XQuery block into it, you'd expect eXide to return the XQuery block as a string, but it actually evaluates the XQuery. For example, starting a new XML file in eXide, entering <x>{1 + 2}</x>, and selecting Eval will return <x>3</x>. So basically eXide lets you Eval the contents of any window and treats the contents as XQuery source code, without regard for the mimetype of the editor pane. An XInclude block in an XML file is evaluated the same way - unless, presumably, you disabled XInclude in conf.xml. I couldn't find an option for disabling XInclude in XQueryServlet.

@joewiz
Copy link
Member

joewiz commented Apr 19, 2017

@awagner-mainz @duncdrum Do you have any comments on this PR? If you would be able to test this to make sure that it works for the apps you needed it for, it would certainly give the reviewers confidence/impetus to merge it in. To try it out, you would just need to clone or download the branch with the work, https://github.com/wolfgangmm/eXide/tree/fix/download-app-xincludes. No rush.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants