Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add testLocationInResults option #4782

Merged
merged 6 commits into from
Nov 4, 2017
Merged

Conversation

SimenB
Copy link
Member

@SimenB SimenB commented Oct 28, 2017

Summary

Fixes #4775. See that for explanation.

I'm not really a fan of how many places a new option has to be specified for it to actually come through.

Test plan

Added integrationtest

if (config.testLocationInResults === true) {
const originalIt = environment.global.it;
environment.global.it = (...args) => {
const stack = callsites()[1];
Copy link
Member Author

@SimenB SimenB Oct 28, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not sure about hardcoding the index. Thoughts? Can traverse all and return the first one which is not inside node_modules instead, perhaps?

Copy link
Contributor

@ljharb ljharb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason this needs to be an option, instead of appearing by default (which would also be semver-minor)? Is it that much slower?

@SimenB
Copy link
Member Author

SimenB commented Oct 29, 2017

A complete test run of jest's own tests, with no cache and without parallelization completes in 260s with this enabled, and 245s without it on my machine.

Not a huge difference. Maybe even the relatively expensive call to get a stacktrace is offset by the overhead inherent to any spec creation call.

I put it behind an option as we looked into adding the same sort of information to a logger at work, and it became 5x slower. Of course that logger does way less than scheduling a spec, so not too relevant, just explaining why I went with the option.

@cpojer is the overhead negligible enough to drop the option?

@ljharb next release is a major anyways (we've dropped node@4), so semver is not a huge issue in this particular case 🙂

@ljharb
Copy link
Contributor

ljharb commented Oct 29, 2017

(That’s unfortunate; every time you drop a platform you make jest a more unacceptable option for non-apps/published projects)

@cpojer
Copy link
Member

cpojer commented Oct 30, 2017

Maybe we motivate the node/v8 teams to speed up actualizing a stack frame or finding the location? This feature would be awesome to have.

@bmeurer
Copy link

bmeurer commented Oct 30, 2017

cc @hashseed @schuay

@hashseed
Copy link

Do I understand correctly that the issue is that formatting a stack trace is expensive? I wonder whether something like caching (code object, code offset) => source position already helps.

@SimenB
Copy link
Member Author

SimenB commented Oct 30, 2017

Not formatting a stack, but getting a stacktrace to begin with, so that I can give a nice message about who called me (in general).

(EDIT: Well, I guess it depends on what you mean when you say "formatting a stack trace". Hopefully there's no ambiguity now 🙂)

Use case can be seen in this issue (we want to point to the place in user code that defined a spec, so other tooling can find it without searching for strings).

The use case I had at work which I alluded to above is that we append a field called LocationInfo to every log statement so it's easy to find in code where the log message came from. Right now it's only added if there's an error being logged, as it has a stacktrace we can look at. Adding a stack on every log message (using callsites) added way too much overhead.

@schuay
Copy link

schuay commented Oct 30, 2017

Are all collected stack traces used, or only a subset? If the latter, you should be able to improve performance by only accessing the stack property once it is needed to enable lazy formatting.

But if I'm reading your previous replies correctly, all traces are used :/ I'm not surprised that this is slow, this path is very much not optimized for performance. I'm sure it could be improved with a bit of work. See https://cs.chromium.org/chromium/src/v8/src/messages.cc?l=956&rcl=82d03a6ec9fe603f2fba2e103f4e31d3db588c8d.

@SimenB
Copy link
Member Author

SimenB commented Oct 30, 2017

Well, I only need 1 of the frames. But which one I need varies, so I have to traverse the stack until I find it.

In my logging case I need the topmost frame. In Jest I'd want the first one not in node_modules. This should always be the second frame (which I've hard coded it to be in this PR), but it's technically possible to wrap more so that'd I'd need to search to find the one I want.

Would it be possible with an api where I provided a predicate function for a stackframe I want so that a full stack trace is not needed at all times? I have no idea if that'd help performance-wise

@@ -189,6 +189,19 @@ Prevent tests from printing messages through the console.

Alias: `-t`. Run only tests and test suites with a name that matches the regex. For example, suppose you want to run only tests related to authorization which will have names like `"GET /api/posts with auth"`, then you can use `jest -t=auth`.

### `--testLocationInResults`
Copy link
Member Author

@SimenB SimenB Oct 31, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

@thymikee thymikee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. The only concern is already mentioned perf, but since it's not a default I'm ok. Maybe worth to leave a todo comment to get back to it after some V8 optimisations?

@orta
Copy link
Member

orta commented Nov 3, 2017

This is super useful ( we have code in jest-editor-support that does this also ) so it's nice to have it centralized correctly 👍

@SimenB
Copy link
Member Author

SimenB commented Nov 3, 2017

Added TODO mentioned by @thymikee.

@cpojer This is ready for merge (or a stricter review :D ) 🙂

we have code in jest-editor-support that does this also

Can that be cleaned up to reuse this?

@codecov-io
Copy link

Codecov Report

Merging #4782 into master will decrease coverage by 0.05%.
The diff coverage is 12.5%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4782      +/-   ##
==========================================
- Coverage   58.94%   58.88%   -0.06%     
==========================================
  Files         199      199              
  Lines        6630     6638       +8     
  Branches        3        4       +1     
==========================================
+ Hits         3908     3909       +1     
- Misses       2722     2729       +7
Impacted Files Coverage Δ
packages/jest-config/src/index.js 23.8% <ø> (ø) ⬆️
packages/jest-util/src/format_test_results.js 60% <ø> (ø) ⬆️
packages/jest-config/src/valid_config.js 100% <ø> (ø) ⬆️
packages/jest-config/src/defaults.js 100% <ø> (ø) ⬆️
packages/jest-config/src/normalize.js 92.68% <ø> (ø) ⬆️
packages/jest-jasmine2/src/index.js 6.15% <0%> (-0.75%) ⬇️
packages/jest-jasmine2/src/reporter.js 75.6% <100%> (+0.6%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5fc3319...7cec947. Read the comment docs.

@cpojer cpojer merged commit 272ff84 into jestjs:master Nov 4, 2017
@cpojer
Copy link
Member

cpojer commented Nov 4, 2017

Let's do it.

@github-actions
Copy link

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Please note this issue tracker is not a help forum. We recommend using StackOverflow or our discord channel for questions.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 13, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Expose the line number of a failing test block in output
10 participants