Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encode collection/path and report proper exception if any #232

Closed
wants to merge 1 commit into from

Conversation

gmella
Copy link
Contributor

@gmella gmella commented Sep 13, 2019

can store back again some filenames with colon or any other encoding requirement
and
error is now properly returned in json response on upload.xql POST (still to be displayed i guess)

@adamretter adamretter self-requested a review September 13, 2019 17:17
Copy link
Member

@adamretter adamretter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - but I can't really say if it will work. For that I think @wolfgangmm or @joewiz are probably the best.

@joewiz
Copy link
Member

joewiz commented Sep 13, 2019

@gmella Thank you! Could you please supply a test case demonstrating why this addition of encode-uri was needed for you?

Reviewers: Do not merge without further study and discussion. This is an area where we have erred in the past. We do not want to double-encode...

@gmella
Copy link
Contributor Author

gmella commented Sep 16, 2019

@joewiz my first intend was just to fix error report since an exception was reported on upload.xql POST with a file that contains datetime info with colon in its filename.
400 return data was :

<exception><path>/db/apps/eXide/modules/upload.xql</path><message>err:XPDY0002 undefined value for variable '$util:exception-message' [at line 108, column 21, source: /db/apps/eXide/modules/upload.xql]</message></exception>

My second modification was the encoding because using err:description then reported :
{ "name" : "toto-12:34:56-titi.txt", "error" : "Invalid value for cast/constructor. failed to convert toto-12:34:56-titi.txt into an XmldbURI: xmldb URI scheme does not start with xmldb:: toto-12:34:56-titi.txt" }

I mixed both concern after proposing a PR on the bad branch... Sorry for my lack of efficiency.

Test case will then only to perform an upload using eXide for a file that contains colon character.

@joewiz
Copy link
Member

joewiz commented Sep 16, 2019

@gmella The issue of special characters in eXist resource names is complex and unfortunately not resolved - see eXist-db/exist#1824. Like the + character that prompted that discussion, your : character is in the list of RFC3986's reserved characters - see https://tools.ietf.org/html/rfc3986#section-2.2. Thanks for raising this issue again, which tends to flare up when someone encounters a strange character, and languish when they find a workaround. At its core, it's not an eXide issue - eXide's error is just a symptom of an issue that affects eXist's core and many systems that stores resources (like file systems, etc.): what characters are reserved, and how to represent those when you really need to? Could you add your :-related comments to issue 1824?

We've tried applying encoding and decoding to collection and resource path URIs in eXide, and unfortunately this resulted in inadvertent double-escaping—likely due to bugs in the encoding/decoding functions and/or due to unanticipated escaping of GET/POST parameters in the HTTP context. Thus, while I'd entertain a PR that encodes every collection and path URI, I would only do so if it were accompanied by full integration tests and test suite showing that characters of every class mentioned in the RFC performed as expected. If you're interested in this area, I'd greatly welcome your participation in issue 1824 and then back here once we've ironed out those issues? This is an important area that deserves care and attention.

@gmella
Copy link
Contributor Author

gmella commented Sep 17, 2019

@joewiz Ok I will add comments on eXist-db/exist#1824.
I propose to close this issue ? to avoid noise related to encoding issue since the original concern of this PR was raised for the replacement of exception report code with the try-catch form that uses $util:exception-message.

Copy link
Member

@line-o line-o left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would still want to see the working error message in the code base.

@gmella
Copy link
Contributor Author

gmella commented Apr 10, 2020

Thanks to #247, I close my old (mixed/unclear) pull request and will try later again the encoding side.

@gmella gmella closed this Apr 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants