Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Adding VM conformance output to PR checks #1685

Closed
wants to merge 9 commits into from

Conversation

Razican
Copy link
Member

@Razican Razican commented Oct 24, 2021

This PR adds conformance results for the VM branch both for PRs and for the conformance results in GitHub pages (even if these are not currently being shown). I'm not 100% sure how this will really work, as I'm not very used to the syntax to concatenate strings.

@Razican Razican added the test Issues and PRs related to the tests. label Oct 24, 2021
@Razican Razican added this to the v0.14.0 milestone Oct 24, 2021
@codecov
Copy link

codecov bot commented Oct 24, 2021

Codecov Report

Merging #1685 (4433cc3) into main (6262bd9) will increase coverage by 6.51%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1685      +/-   ##
==========================================
+ Coverage   44.40%   50.91%   +6.51%     
==========================================
  Files         203      199       -4     
  Lines       20270    17709    -2561     
==========================================
+ Hits         9000     9016      +16     
+ Misses      11270     8693    -2577     
Impacted Files Coverage Δ
boa_cli/src/main.rs 5.88% <0.00%> (-27.46%) ⬇️
boa/src/lib.rs 77.55% <0.00%> (-15.14%) ⬇️
...tax/parser/expression/assignment/exponentiation.rs 64.00% <0.00%> (-13.78%) ⬇️
boa/src/syntax/lexer/string.rs 56.15% <0.00%> (-4.68%) ⬇️
boa/src/syntax/lexer/number.rs 64.73% <0.00%> (-1.54%) ⬇️
boa/src/object/internal_methods/array.rs 68.05% <0.00%> (-1.51%) ⬇️
...syntax/parser/statement/iteration/for_statement.rs 71.66% <0.00%> (-1.22%) ⬇️
boa/src/value/operations.rs 45.39% <0.00%> (-1.04%) ⬇️
...tement/declaration/hoistable/generator_decl/mod.rs 55.00% <0.00%> (-1.00%) ⬇️
boa/src/value/display.rs 89.81% <0.00%> (-0.84%) ⬇️
... and 101 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6262bd9...4433cc3. Read the comment docs.

@jedel1043
Copy link
Member

Is it possible to run the VM tests only if the vm module has changed?
That would save us a lot of CI time.

@RageKnify
Copy link
Member

Is it possible to run the VM tests only if the vm module has changed? That would save us a lot of CI time.

I think we can have the CI rule only match if the PR's branch name includes the letters vm
That would probably have the same effect while requiring just a little bit of work in our part.

@Razican
Copy link
Member Author

Razican commented Oct 24, 2021

Is it possible to run the VM tests only if the vm module has changed? That would save us a lot of CI time.

I wouldn't do this, as some changes to the non-vm modules will also affect the conformance of the vm feature.

the path of the json with the base results was used, but I'm pretty sure
we can just start with an empty path instead
@github-actions
Copy link

github-actions bot commented Oct 26, 2021

Test262 conformance changes

Non-VM implementation

Test result main count PR count difference
Total 86,464 86,464 0
Passed 39,533 39,533 0
Ignored 18,869 18,869 0
Failed 28,062 28,062 0
Panics 0 0 0
Conformance 45.72% 45.72% 0.00%

VM implementation

Test result main count PR count difference
Total 86,464 86,464 0
Passed 6,677 6,677 0
Ignored 18,869 18,869 0
Failed 60,918 60,918 0
Panics 60,047 60,047 0
Conformance 7.72% 7.72% 0.00%

@Razican
Copy link
Member Author

Razican commented Oct 28, 2021

Looks good to me :) let's see if it works properly going forward!

@RageKnify
Copy link
Member

bors r+

bors bot pushed a commit that referenced this pull request Oct 29, 2021
This PR adds conformance results for the VM branch both for PRs and for the conformance results in GitHub pages (even if these are not currently being shown). I'm not 100% sure how this will really work, as I'm not very used to the syntax to concatenate strings.

Co-authored-by: João Borges <rageknify@gmail.com>
Co-authored-by: RageKnify <RageKnify@gmail.com>
@bors
Copy link

bors bot commented Oct 29, 2021

Build failed:

@Razican
Copy link
Member Author

Razican commented Oct 29, 2021

bors retry

bors bot pushed a commit that referenced this pull request Oct 29, 2021
This PR adds conformance results for the VM branch both for PRs and for the conformance results in GitHub pages (even if these are not currently being shown). I'm not 100% sure how this will really work, as I'm not very used to the syntax to concatenate strings.

Co-authored-by: João Borges <rageknify@gmail.com>
Co-authored-by: RageKnify <RageKnify@gmail.com>
@bors
Copy link

bors bot commented Oct 29, 2021

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title Adding VM conformance output to PR checks [Merged by Bors] - Adding VM conformance output to PR checks Oct 29, 2021
@bors bors bot closed this Oct 29, 2021
@bors bors bot deleted the vm_test262 branch October 29, 2021 20:38
@Razican Razican added the hacktoberfest-accepted PR accepted for Hacktoberfest label Oct 30, 2021
@RageKnify RageKnify added the Internal Category for changelog label Jan 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
hacktoberfest-accepted PR accepted for Hacktoberfest Internal Category for changelog test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants