-
Notifications
You must be signed in to change notification settings - Fork 758
Bug 7571 - set directory root for debugger.html #7594
Bug 7571 - set directory root for debugger.html #7594
Conversation
The code needs refactoring |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we add a test in addToTree.spec.js or another relevant file?
So, I believe we started using whatwg-url for this reason last month here. https://www.npmjs.com/package/whatwg-url I wonder if using this library would give us a more spec compliant URL parse that would help us... |
Thanks . Will look into this , if It can help. This solution is hacky and I need to refactor this to have more robust solution. |
src/utils/sources-tree/getURL.js
Outdated
case "ng:": | ||
case "webpack:": | ||
const path = pathname.substring(2); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
using whatwg-url will automatically do this. Can I introduce that module in src/utils/url.js ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, we can try that
@jasonLaster This is better than what I did before. Also fixes the absolute path for https://w2x2oypmqk.codesandbox.io/ Still I am not completely satisfied here. The way to create directory has to change if the url is from webpack. Check chrome's webpack folder. It creates folders based on webpack paths. Check image attached. I think we need to re-think on webpack tree. What are your thoughts on this? Note: Will add unit tests for webpack, once I am done with the exact output that we want. |
@jainsneha23 can you give some more examples of the differences between chrome and firefox? I'm not sure I follow... Also, could you share the whatwg example, either by pushing it to this branch or by sharing another branch. by the way, @loganfsmyth is our path expert and will be back on monday. |
Find the screeshot below on the difference on how node and whatwg handles URL differently. With protocols ending with :// for webpack and ng, node URL doesn't seem to return correct path. There are extra slashes. So, using whatwg URL is better. Also we have URLs like
The screenshot above shows how chrome creates folders with . .. for level up folders But in our source-map, we trim all the dots. So, that is not achievable right now. But the path for absolute paths is fixed. I have to fix the unit test though. Will do it tomorrow. May be we can think on should we handle the level up folders for webpack, and raise a separate issue for the same. As that will require changes in source-map as well. |
e03edef
to
7ba75e2
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think my questions on this are really more about the existing architecture, but the big question to me is, why aren't these source URLs already absolute? It seems like we should aim to have an absolute URL computed up front for all sources, and if we have that, the WHATWG URL API pretty much provides everything else we need, and we can avoid pretty much all of the logic in _getURL
.
src/utils/sources-tree/getURL.js
Outdated
return { | ||
...def, | ||
path: pathname, | ||
filename, | ||
group: `${protocol}//` | ||
group: `${protocol}//`, | ||
isAbsolutePath: pathname.startsWith("//") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem right to me, if we're using WHATWG URL to process this, it should just be a single slash.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@loganfsmyth For webpack, we can have urls with double slash, and dots
Please find some webpack urls below:
webpack:///webpack/bootstrap fc4578a0bef29d2a63f3
webpack:///src/component/index.js
webpack:////Users/workspace/github/debugger.html/node_modules/react/index.js
webpack:///./utils/debug.js
webpack:///../node_modules/devtools-environment/test-flag.js
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In these examples, with whatwg-url
, the pathname
should be
/webpack/bootstrap fc4578a0bef29d2a63f3
/src/component/index.js
//Users/workspace/github/debugger.html/node_modules/react/index.js
/utils/debug.js
/node_modules/devtools-environment/test-flag.js
what is throwing me off here is these are already all absolute because they start with a single slash.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@loganfsmyth
Yes with whatwg-url the pathname are as you mentioned. What I meant by absolute was system absolute path.
If you see, all other paths are relative to the debugger.html folder. And only this path //Users/workspace/github/debugger.html/node_modules/react/index.js is system absolute path.
In webpack, the paths are file paths in system, and either they are relative to the path from where the webpack was run like the other 4, or absolute system path.
And this issue raised happened only for the system absolute path. I was also able to see the same issue in https://w2x2oypmqk.codesandbox.io/
Sorry for miscommunication. And we can have a quick chat on this today evening if you want.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think part of what is tripping me up is, aren't those examples just mistakes on the part of Webpack? All URLs like this are absolute. The fact that there is a double slash at the start of one of them just means that one of the path segments is an empty string. It seems like a mistake in whatever configuration generated that URL. It seems unreasonable to expect the devtools to try to interpret what is essentially bad data.
With URLs as
webpack:///webpack/bootstrap fc4578a0bef29d2a63f3
webpack:///src/component/index.js
webpack:////Users/workspace/github/debugger.html/node_modules/react/index.js
webpack:///./utils/debug.js
webpack:///../node_modules/devtools-environment/test-flag.js
I'd expect that the folder structure would be
Webpack (webpack://)
* "webpack"
* "bootstrap fc4578a0bef29d2a63f3"
* "src"
* "components"
* "index.js"
* ""
* "Users"
* "workspace"
* "github"
* "debugger.html"
* "node_modules"
* "react"
* "index.js"
* "utils"
* "debug.js"
* "node_modules"
* "devtools-enviroment"
* "test-flag.js"
src/utils/sources-tree/getURL.js
Outdated
return { | ||
...def, | ||
path: pathname, | ||
filename, | ||
group: `${protocol}//` | ||
group: `${protocol}//`, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm surprised that this doesn't include the hostname. How that we've got WHATWG URL, I'd have expected more of the logic in this function to collapse into
group: `${protocol}//${host || ""}`
@jainsneha23 I wonder how well whatwg does in general... could you try running getURL without all of the special cases and share some findings? diff --git a/src/utils/sources-tree/getURL.js b/src/utils/sources-tree/getURL.js
index 42e1f5227..055912b19 100644
--- a/src/utils/sources-tree/getURL.js
+++ b/src/utils/sources-tree/getURL.js
@@ -39,82 +39,7 @@ function _getURL(source: Source, defaultDomain: string): ParsedURL {
const { pathname, protocol, host } = parse(url);
const filename = getUnicodeUrlPath(getFilenameFromPath(pathname));
-
- switch (protocol) {
- case "javascript:":
- // Ignore `javascript:` URLs for now
- return def;
-
- case "moz-extension:":
- case "resource:":
- return {
- ...def,
- path: pathname,
- filename,
- group: `${protocol}//${host || ""}`
- };
-
- case "webpack:":
- case "ng:":
- return {
- ...def,
- path: pathname,
- filename,
- group: `${protocol}//`
- };
-
- case "about:":
- // An about page is a special case
- return {
- ...def,
- path: "/",
- filename,
- group: url
- };
-
- case "data:":
- return {
- ...def,
- path: "/",
- group: NoDomain,
- filename: url
- };
-
- case "":
- if (pathname && pathname.startsWith("/")) {
- // use file protocol for a URL like "/foo/bar.js"
- return {
- ...def,
- path: pathname,
- filename,
- group: "file://"
- };
- } else if (!host) {
- return {
- ...def,
- path: url,
- group: defaultDomain || "",
- filename
- };
- }
- break;
-
- case "http:":
- case "https:":
- return {
- ...def,
- path: pathname,
- filename,
- group: getUnicodeHostname(host)
- };
- }
-
- return {
- ...def,
- path: pathname,
- group: protocol ? `${protocol}//` : "",
- filename
- };
+ return { ...def, pathname, protocol, host, filename };
} |
By the way, as you can probably tell... this is somewhat of a scary change so it will likely move pretty slow as we try to sort out all of the different scenarios. Thanks (in advance) for your patience :) |
7ba75e2
to
72d4b15
Compare
Here are some differences listed (we can ignore node url for now, as this happens on the browser). Also notice the difference in chrome and firefox. SO, better to use WHATWG for consistency
So, we will still need the special cases for javascript, about, data, and no protocol case (to handle file urls, and relative urls) |
src/utils/sources-tree/addToTree.js
Outdated
@@ -86,8 +86,9 @@ function traverseTree( | |||
tree: TreeDirectory, | |||
debuggeeHost: ?string | |||
): TreeNode { | |||
const parts = url.path.split("/").filter(p => p !== ""); | |||
parts.unshift(url.group); | |||
const parts = url.path.split("/").filter((p, i) => p !== "" || i < 2); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should add a comment here as to what is going on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@darkwing @loganfsmyth Do we really need this filter? Cannot we just render the path as is?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that this filter seems unnecessary to me. My vote would be for
const parts = url.path.split("/");
if (parts[0] === "") parts.shift();
parts.unshift(url.group);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looking over this again, I think we should just keep it simple and do
const parts = url.path.split("/");
parts.unshift(url.group);
and revisit after that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@loganfsmyth Unfortunately the snippet you cited creates the following:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright, that's what I was worried about. Honestly, maybe the better approach would be to just leave the filter for now and change things more in a later pass?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@darkwing in davidwalsh.name, the empty folder you got was because of a path '/', which creates empty node. for this case, we can remove any trailing slash from a url.
otherwise this works fine. can you please have a look at my commit.
@loganfsmyth Since you were the sourcemaps legend, I'd like your take on this; i.e. if this is a viable solution. |
@darkwing what is the test page? |
My blog, of course :D |
I played with this today and it's totally frustrating! I appreciate your efforts @jainsneha23!
|
8a904d1
to
8aa678b
Compare
8aa678b
to
4b82043
Compare
4b82043
to
50e878b
Compare
… into bug-7571-set-dir-root * 'master' of https://github.com/devtools-html/debugger.html: [Sources] Consolidate sources reducer (firefox-devtools#7977) Update maintainer.md (firefox-devtools#7973) Improve EventListener UI (firefox-devtools#7937)
@jasonLaster @darkwing The branch is rebased with latest master, and unit tests are updates for files with no filename, webpack urls, webpack url with system absolute path. Its ready for review. |
so... in terms of getting this library included in the debugger vendors... There are a couple of steps:
diff --git a/package.json b/package.json
index be1988695..77365dc2a 100644
--- a/package.json
+++ b/package.json
@@ -80,7 +80,8 @@
"react-transition-group": "^2.2.1",
"reselect": "^4.0.0",
"svg-inline-react": "^3.0.0",
- "wasmparser": "^0.7.0"
+ "wasmparser": "^0.7.0",
+ "whatwg-url": "..."
},
"private": true,
"workspaces": [
diff --git a/src/vendors.js b/src/vendors.js
index 29660c0b2..bc7792280 100644
--- a/src/vendors.js
+++ b/src/vendors.js
@@ -28,6 +28,7 @@ import * as fuzzaldrinPlus from "fuzzaldrin-plus";
import * as transition from "react-transition-group/Transition";
import * as reactAriaComponentsTabs from "react-aria-components/src/tabs";
import * as reselect from "reselect";
+import whatWg from "whatwg-url";
// Modules imported without destructuring
import classnames from "classnames";
@@ -55,5 +56,6 @@ export const vendored = {
reselect,
// Svg is required via relative paths, so the key is not imported path.
// See .babel/transform-mc.js
- Svg
+ Svg,
+ whatWg
}; |
Functionally this seems good to me. The hard remaining part is definitely getting this into mozilla-central itself. |
I don't mind getting it into MC |
I'd like to get some closure here -- @loganfsmyth @jasonLaster What's the difficulty in getting this into mc? |
@darkwing I hope to do this next week |
Hey @jainsneha23 i'm sorry for the delay, i'm finally getting around to landing this in firefox https://phabricator.services.mozilla.com/D31313 and ufortunately i'm seeing the original issue with the debugger directory. Did you see this as well when working on it or was your version fixed? |
@jasonLaster hey my version was fixed. Can you elaborate on this? Then I will check. Are you seeing the error after rebasing it with current master? Or my branch itself gives the error? |
oh cool. Yeah, i copied your version in the phabricator branch. I would be really interested if this branch works when you rebase it... |
Sure. I will rebase and check this weekend. |
great!! thanks @jasonLaster @darkwing @loganfsmyth |
Fixes #7571
Summary of Changes