Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jest should avoid printing the individual tests if beforeAll fails #9901

Open
thernstig opened this issue Apr 27, 2020 · 17 comments · May be fixed by #13273
Open

Jest should avoid printing the individual tests if beforeAll fails #9901

thernstig opened this issue Apr 27, 2020 · 17 comments · May be fixed by #13273

Comments

@thernstig
Copy link
Contributor

thernstig commented Apr 27, 2020

🐛 Bug Report

When beforeAll fails, none of the other tests actually execute, but they all fail with the same reason, and they all print the same error.

Jest should avoid printing the individual tests if beforeAll fails. Right now the implementation just checks if there is an error and if yes, print it and fail the test without actually executing it.

Note that @SimenB asked me to create this #6695 (comment).

To Reproduce

Run a beforeAll() that fails where a couple of tests are executed after. This is the failure received (with the supplied repro below):

 FAIL   src/test.unit.test.js
  test that a 3rd party API remains consistent
    ✕ API function 1
    ✕ API function 2
    ✕ API function 3test that a 3rd party API remains consistent › API function 1

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

  ● test that a 3rd party API remains consistent › API function 2

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

  ● test that a 3rd party API remains consistent › API function 3

    expect(received).toBe(expected) // Object.is equality

    Expected: "successful"
    Received: "login"

      1 | describe('test that a 3rd party API remains consistent', () => {
    > 2 |   beforeAll(() => expect('login').toBe('successful')); // this will fail
        |                                   ^
      3 |   test('API function 1', () => expect(1).toBe(1)); // each...
      4 |   test('API function 2', () => expect(2).toBe(2)); // ...of these...
      5 |   test('API function 3', () => expect(3).toBe(3)); // ...will fail too

      at src/test.unit.test.js:2:35

Test Suites: 1 failed, 1 total
Tests:       3 failed, 3 total
Snapshots:   0 total
Time:        1.037s

Expected behavior

Jest should avoid printing the individual tests if beforeAll fails. It is highly confusing for users.

Link to repl or repo (highly encouraged)

describe('test that a 3rd party API remains consistent', () => {
  beforeAll(() => expect('login').toBe('successful')); // this will fail
  test('API function 1', () => expect(1).toBe(1)); // each...
  test('API function 2', () => expect(2).toBe(2)); // ...of these...
  test('API function 3', () => expect(3).toBe(3)); // ...will be reported as failed too
});

envinfo

  System:
    OS: Linux 4.15 Ubuntu 18.04.4 LTS (Bionic Beaver)
    CPU: (4) x64 Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
  Binaries:
    Node: 12.15.0 - ~/.config/nvm/12.15.0/bin/node
    npm: 6.13.4 - ~/.config/nvm/12.15.0/bin/npm
  npmPackages:
    jest: 25.1.0 => 25.1.0
@vldmrkl
Copy link

vldmrkl commented May 5, 2020

Hi 👋

Can I work on this issue?

@SimenB
Copy link
Member

SimenB commented May 6, 2020

Yeah, that'd be lovely! Feel free to ask questions if you get stuck or want a pointer 🙂

I'd start in jest-circus (jest-jasmine is soft-deprecated, so unless you want to you can ignore it). Maybe the approach in #7201 might help you get started?

@DMeechan
Copy link

When beforeAll fails, it also ignores it.only and logs all tests as failing: https://repl.it/repls/MobileBaggyLearning

Wasn't sure if this bug will already be fixed by #10004 so please let me know if you'd like me to file a new issue!

@PizzaPartyInc
Copy link

Hi! Has there been any progress on this issue? :)

@thernstig
Copy link
Contributor Author

@vldmrkl Looking forward to you working on this issue 😄

@vldmrkl
Copy link

vldmrkl commented Mar 9, 2021

Sorry, can't continue working on this issue.
My latest progress could be found at #10004
I apologize for miscommunication.

@github-actions
Copy link

github-actions bot commented Mar 9, 2022

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Mar 9, 2022
@markdoliner-doma
Copy link

I think this change still makes sense and the issue should remain open.

@github-actions github-actions bot removed the Stale label Mar 9, 2022
@itaizelther
Copy link

Hey, I'd like to work on this issue. :)

@thernstig
Copy link
Contributor Author

@itaizelther I'd say go for it and push a PR, continuing on the one already started.

@itaizelther
Copy link

@thernstig I have no write permissions to the existing PR, so I have created a new one: #13273
It is done, you may review it

@thernstig
Copy link
Contributor Author

I am not a maintainer here, just meant that I don't think anyone minds if you start a new PR. Great that you did!

@vldmrkl
Copy link

vldmrkl commented Sep 17, 2022

@itaizelther thank you 🙏

@github-actions
Copy link

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Sep 17, 2023
@thernstig
Copy link
Contributor Author

Since there is a PR at #13273 I am bumping this

@github-actions github-actions bot removed the Stale label Sep 18, 2023
Copy link

This issue is stale because it has been open for 1 year with no activity. Remove stale label or comment or this will be closed in 30 days.

@github-actions github-actions bot added the Stale label Sep 17, 2024
@thernstig
Copy link
Contributor Author

Since there is a PR at #13273 I am bumping this

@github-actions github-actions bot removed the Stale label Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment