Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Merged by Bors] - Compile StatementList after parse passes on negative tests #1906

Closed
wants to merge 2 commits into from

Conversation

raskad
Copy link
Member

@raskad raskad commented Mar 7, 2022

This fixes an issue with 262 negative tests, that should produce a syntax errors. Currently we only parse the test code is such cases. If the parsing does not return an error, we do not compile the code further. This caused some panics. Most of them are fixed by now, the last ones will be fixed with #1860.

@raskad raskad added bug Something isn't working test Issues and PRs related to the tests. Internal Category for changelog labels Mar 7, 2022
@raskad raskad added this to the v0.14.0 milestone Mar 7, 2022
@github-actions
Copy link

github-actions bot commented Mar 7, 2022

Test262 conformance changes

VM implementation

Test result main count PR count difference
Total 88,410 88,410 0
Passed 43,293 43,377 +84
Ignored 21,481 21,481 0
Failed 23,636 23,552 -84
Panics 0 6 +6
Conformance 48.97% 49.06% +0.10%
Fixed tests (84):
test/language/expressions/assignmenttargettype/parenthesized-lefthandsideexpression-assignment-assignmentexpression-2.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/parenthesized-lefthandsideexpression-assignment-assignmentexpression-2.js (previously Failed)
test/language/expressions/assignmenttargettype/direct-callexpression-arguments.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/direct-callexpression-arguments.js (previously Failed)
test/language/expressions/assignmenttargettype/parenthesized-lefthandsideexpression-assignment-assignmentexpression-0.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/parenthesized-lefthandsideexpression-assignment-assignmentexpression-0.js (previously Failed)
test/language/expressions/assignmenttargettype/direct-lefthandsideexpression-assignment-assignmentexpression-0.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/direct-lefthandsideexpression-assignment-assignmentexpression-0.js (previously Failed)
test/language/expressions/assignmenttargettype/parenthesized-callexpression-arguments.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/parenthesized-callexpression-arguments.js (previously Failed)
test/language/expressions/assignmenttargettype/direct-lefthandsideexpression-assignment-assignmentexpression-2.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/direct-lefthandsideexpression-assignment-assignmentexpression-2.js (previously Failed)
test/language/expressions/assignmenttargettype/direct-lefthandsideexpression-assignment-assignmentexpression-1.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/direct-lefthandsideexpression-assignment-assignmentexpression-1.js (previously Failed)
test/language/expressions/assignmenttargettype/parenthesized-lefthandsideexpression-assignment-assignmentexpression-1.js [strict mode] (previously Failed)
test/language/expressions/assignmenttargettype/parenthesized-lefthandsideexpression-assignment-assignmentexpression-1.js (previously Failed)
test/language/expressions/object/method-definition/name-param-redecl.js [strict mode] (previously Failed)
test/language/expressions/object/method-definition/name-param-redecl.js (previously Failed)
test/language/expressions/logical-assignment/lgcl-and-assignment-operator-non-simple-lhs.js [strict mode] (previously Failed)
test/language/expressions/logical-assignment/lgcl-and-assignment-operator-non-simple-lhs.js (previously Failed)
test/language/expressions/logical-assignment/lgcl-or-assignment-operator-non-simple-lhs.js [strict mode] (previously Failed)
test/language/expressions/logical-assignment/lgcl-or-assignment-operator-non-simple-lhs.js (previously Failed)
test/language/expressions/logical-assignment/lgcl-nullish-assignment-operator-non-simple-lhs.js [strict mode] (previously Failed)
test/language/expressions/logical-assignment/lgcl-nullish-assignment-operator-non-simple-lhs.js (previously Failed)
test/language/expressions/assignment/target-assignment-inside-function.js [strict mode] (previously Failed)
test/language/expressions/assignment/target-assignment-inside-function.js (previously Failed)
test/language/expressions/assignment/target-assignment.js [strict mode] (previously Failed)
test/language/expressions/assignment/target-assignment.js (previously Failed)
test/language/statements/for-of/head-const-bound-names-dup.js [strict mode] (previously Failed)
test/language/statements/for-of/head-const-bound-names-dup.js (previously Failed)
test/language/statements/for-of/head-let-bound-names-dup.js [strict mode] (previously Failed)
test/language/statements/for-of/head-let-bound-names-dup.js (previously Failed)
test/language/statements/block/labeled-continue.js [strict mode] (previously Failed)
test/language/statements/block/labeled-continue.js (previously Failed)
test/language/statements/continue/S12.7_A6.js [strict mode] (previously Failed)
test/language/statements/continue/S12.7_A6.js (previously Failed)
test/language/statements/continue/S12.7_A1_T1.js [strict mode] (previously Failed)
test/language/statements/continue/S12.7_A1_T1.js (previously Failed)
test/language/statements/continue/S12.7_A5_T2.js (previously Failed)
test/language/statements/continue/S12.7_A8_T1.js [strict mode] (previously Failed)
test/language/statements/continue/S12.7_A8_T1.js (previously Failed)
test/language/statements/continue/S12.7_A5_T1.js (previously Failed)
test/language/statements/continue/S12.7_A5_T3.js (previously Failed)
test/language/statements/continue/S12.7_A8_T2.js [strict mode] (previously Failed)
test/language/statements/continue/S12.7_A8_T2.js (previously Failed)
test/language/statements/continue/S12.7_A1_T2.js [strict mode] (previously Failed)
test/language/statements/continue/S12.7_A1_T2.js (previously Failed)
test/language/statements/continue/S12.7_A1_T4.js [strict mode] (previously Failed)
test/language/statements/continue/S12.7_A1_T4.js (previously Failed)
test/language/statements/continue/S12.7_A1_T3.js [strict mode] (previously Failed)
test/language/statements/continue/S12.7_A1_T3.js (previously Failed)
test/language/statements/switch/syntax/redeclaration/const-name-redeclaration-attempt-with-let.js [strict mode] (previously Failed)
test/language/statements/switch/syntax/redeclaration/const-name-redeclaration-attempt-with-let.js (previously Failed)
test/language/statements/switch/syntax/redeclaration/let-name-redeclaration-attempt-with-const.js [strict mode] (previously Failed)
test/language/statements/switch/syntax/redeclaration/let-name-redeclaration-attempt-with-const.js (previously Failed)
test/language/statements/switch/syntax/redeclaration/const-name-redeclaration-attempt-with-const.js [strict mode] (previously Failed)
test/language/statements/switch/syntax/redeclaration/const-name-redeclaration-attempt-with-const.js (previously Failed)
test/language/statements/switch/syntax/redeclaration/let-name-redeclaration-attempt-with-let.js [strict mode] (previously Failed)
test/language/statements/switch/syntax/redeclaration/let-name-redeclaration-attempt-with-let.js (previously Failed)
test/language/statements/for/S12.6.3_A12.1_T3.js [strict mode] (previously Failed)
test/language/statements/for/S12.6.3_A12.1_T3.js (previously Failed)
test/language/statements/for/S12.6.3_A11_T3.js [strict mode] (previously Failed)
test/language/statements/for/S12.6.3_A11_T3.js (previously Failed)
test/language/statements/for/S12.6.3_A11.1_T3.js [strict mode] (previously Failed)
test/language/statements/for/S12.6.3_A11.1_T3.js (previously Failed)
test/language/statements/for/S12.6.3_A12_T3.js [strict mode] (previously Failed)
test/language/statements/for/S12.6.3_A12_T3.js (previously Failed)
test/language/statements/labeled/continue.js [strict mode] (previously Failed)
test/language/statements/labeled/continue.js (previously Failed)
test/language/statements/break/S12.8_A8_T1.js [strict mode] (previously Failed)
test/language/statements/break/S12.8_A8_T1.js (previously Failed)
test/language/statements/break/S12.8_A5_T1.js (previously Failed)
test/language/statements/break/S12.8_A1_T2.js [strict mode] (previously Failed)
test/language/statements/break/S12.8_A1_T2.js (previously Failed)
test/language/statements/break/S12.8_A6.js [strict mode] (previously Failed)
test/language/statements/break/S12.8_A6.js (previously Failed)
test/language/statements/break/S12.8_A5_T3.js (previously Failed)
test/language/statements/break/S12.8_A1_T4.js [strict mode] (previously Failed)
test/language/statements/break/S12.8_A1_T4.js (previously Failed)
test/language/statements/break/S12.8_A5_T2.js (previously Failed)
test/language/statements/for-in/head-const-bound-names-dup.js [strict mode] (previously Failed)
test/language/statements/for-in/head-const-bound-names-dup.js (previously Failed)
test/language/statements/for-in/head-let-bound-names-dup.js [strict mode] (previously Failed)
test/language/statements/for-in/head-let-bound-names-dup.js (previously Failed)
New panics (6):
test/language/statements/break/S12.8_A1_T3.js [strict mode] (previously Failed)
test/language/statements/break/S12.8_A1_T3.js (previously Failed)
test/language/statements/break/S12.8_A1_T1.js [strict mode] (previously Failed)
test/language/statements/break/S12.8_A1_T1.js (previously Failed)
test/language/statements/break/S12.8_A8_T2.js [strict mode] (previously Failed)
test/language/statements/break/S12.8_A8_T2.js (previously Failed)

@codecov
Copy link

codecov bot commented Mar 7, 2022

Codecov Report

Merging #1906 (05b2751) into main (9f9e36c) will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #1906   +/-   ##
=======================================
  Coverage   46.36%   46.36%           
=======================================
  Files         206      206           
  Lines       16824    16824           
=======================================
  Hits         7801     7801           
  Misses       9023     9023           

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f9e36c...05b2751. Read the comment docs.

@github-actions
Copy link

github-actions bot commented Mar 7, 2022

Benchmark for b1716b4

Click to view benchmark
Test Base PR %
Arithmetic operations (Compiler) 741.3±91.91ns 740.3±60.45ns -0.13%
Arithmetic operations (Execution) 2.6±0.23µs 2.7±0.29µs +3.85%
Arithmetic operations (Parser) 7.0±0.48µs 7.0±0.52µs 0.00%
Array access (Compiler) 1968.5±165.40ns 1888.4±142.60ns -4.07%
Array access (Execution) 13.4±1.09µs 13.2±1.24µs -1.49%
Array access (Parser) 15.6±0.89µs 15.7±1.07µs +0.64%
Array creation (Compiler) 2.5±0.10µs 2.7±0.40µs +8.00%
Array creation (Execution) 4.2±0.33ms 4.4±0.26ms +4.76%
Array creation (Parser) 17.0±1.25µs 17.8±1.62µs +4.71%
Array pop (Compiler) 5.4±0.57µs 5.1±0.36µs -5.56%
Array pop (Execution) 1858.7±124.62µs 1866.7±102.95µs +0.43%
Array pop (Parser) 188.6±27.03µs 187.4±16.91µs -0.64%
Boolean Object Access (Compiler) 1557.8±127.17ns 1551.5±139.39ns -0.40%
Boolean Object Access (Execution) 8.9±1.28µs 7.7±0.40µs -13.48%
Boolean Object Access (Parser) 18.7±1.63µs 18.4±1.65µs -1.60%
Clean js (Compiler) 5.2±0.58µs 4.8±0.29µs -7.69%
Clean js (Execution) 1461.3±391.66µs 1458.9±84.49µs -0.16%
Clean js (Parser) 37.8±3.84µs 38.2±3.63µs +1.06%
Create Realm 374.0±34.20ns 360.9±20.39ns -3.50%
Dynamic Object Property Access (Compiler) 2.4±0.34µs 2.3±0.13µs -4.17%
Dynamic Object Property Access (Execution) 9.9±1.10µs 9.7±0.69µs -2.02%
Dynamic Object Property Access (Parser) 13.7±1.04µs 13.6±1.01µs -0.73%
Fibonacci (Compiler) 3.4±0.30µs 3.4±0.30µs 0.00%
Fibonacci (Execution) 2.5±0.17ms 2.5±0.13ms 0.00%
Fibonacci (Parser) 20.8±1.43µs 20.8±1.74µs 0.00%
For loop (Compiler) 3.0±0.48µs 2.8±0.17µs -6.67%
For loop (Execution) 56.0±2.91µs 56.2±3.15µs +0.36%
For loop (Parser) 18.3±1.75µs 18.1±1.26µs -1.09%
Mini js (Compiler) 4.6±0.38µs 4.6±0.31µs 0.00%
Mini js (Execution) 1283.5±78.62µs 1358.7±113.25µs +5.86%
Mini js (Parser) 33.2±3.32µs 33.0±2.81µs -0.60%
Number Object Access (Compiler) 1527.3±166.78ns 1414.2±85.28ns -7.41%
Number Object Access (Execution) 6.5±0.55µs 6.3±0.47µs -3.08%
Number Object Access (Parser) 14.2±1.01µs 14.5±1.71µs +2.11%
Object Creation (Compiler) 2.2±0.19µs 2.0±0.14µs -9.09%
Object Creation (Execution) 8.8±0.54µs 8.9±0.90µs +1.14%
Object Creation (Parser) 12.2±0.96µs 11.8±0.84µs -3.28%
RegExp (Compiler) 2.2±0.10µs 2.4±0.25µs +9.09%
RegExp (Execution) 18.1±2.03µs 17.3±1.15µs -4.42%
RegExp (Parser) 12.7±1.13µs 13.0±1.57µs +2.36%
RegExp Creation (Compiler) 2.1±0.18µs 2.0±0.14µs -4.76%
RegExp Creation (Execution) 13.1±1.63µs 12.9±1.26µs -1.53%
RegExp Creation (Parser) 11.1±1.07µs 10.7±0.82µs -3.60%
RegExp Literal (Compiler) 2.4±0.36µs 2.3±0.14µs -4.17%
RegExp Literal (Execution) 17.9±2.04µs 17.2±1.35µs -3.91%
RegExp Literal (Parser) 10.4±0.76µs 10.5±0.77µs +0.96%
RegExp Literal Creation (Compiler) 2.1±0.22µs 2.1±0.25µs 0.00%
RegExp Literal Creation (Execution) 13.0±0.95µs 12.9±0.81µs -0.77%
RegExp Literal Creation (Parser) 8.0±0.56µs 7.8±0.36µs -2.50%
Static Object Property Access (Compiler) 2.0±0.17µs 2.1±0.12µs +5.00%
Static Object Property Access (Execution) 9.0±1.01µs 9.0±0.69µs 0.00%
Static Object Property Access (Parser) 12.8±1.43µs 12.6±0.92µs -1.56%
String Object Access (Compiler) 2.2±0.32µs 2.0±0.18µs -9.09%
String Object Access (Execution) 10.6±0.69µs 10.8±1.15µs +1.89%
String Object Access (Parser) 17.9±1.39µs 18.3±1.65µs +2.23%
String comparison (Compiler) 3.2±0.27µs 3.0±0.25µs -6.25%
String comparison (Execution) 8.4±0.60µs 8.1±0.71µs -3.57%
String comparison (Parser) 13.9±1.13µs 14.3±1.30µs +2.88%
String concatenation (Compiler) 2.3±0.06µs 2.4±0.30µs +4.35%
String concatenation (Execution) 7.6±0.88µs 7.4±0.43µs -2.63%
String concatenation (Parser) 9.6±0.70µs 9.9±0.71µs +3.13%
String copy (Compiler) 1898.9±137.33ns 2.0±0.51µs +5.32%
String copy (Execution) 6.9±0.56µs 7.0±0.99µs +1.45%
String copy (Parser) 7.2±0.58µs 7.3±0.85µs +1.39%
Symbols (Compiler) 1380.1±128.96ns 1349.9±76.73ns -2.19%
Symbols (Execution) 6.7±0.56µs 6.5±0.55µs -2.99%
Symbols (Parser) 5.4±0.22µs 5.6±0.65µs +3.70%

@github-actions
Copy link

github-actions bot commented Mar 7, 2022

Benchmark for dfa24be

Click to view benchmark
Test Base PR %
Arithmetic operations (Compiler) 444.8±3.07ns 506.7±1.80ns +13.92%
Arithmetic operations (Execution) 1933.4±2.20ns 1708.5±1.93ns -11.63%
Arithmetic operations (Parser) 5.2±0.01µs 5.2±0.00µs 0.00%
Array access (Compiler) 1262.3±3.09ns 1127.0±3.83ns -10.72%
Array access (Execution) 9.7±0.06µs 9.6±0.04µs -1.03%
Array access (Parser) 10.2±0.01µs 10.3±0.03µs +0.98%
Array creation (Compiler) 1790.7±3.90ns 1590.6±3.71ns -11.17%
Array creation (Execution) 3.2±0.00ms 3.2±0.00ms 0.00%
Array creation (Parser) 11.5±0.03µs 11.4±0.05µs -0.87%
Array pop (Compiler) 3.3±0.01µs 3.3±0.01µs 0.00%
Array pop (Execution) 1381.5±6.26µs 1229.8±6.33µs -10.98%
Array pop (Parser) 115.3±0.39µs 115.1±0.54µs -0.17%
Boolean Object Access (Compiler) 960.4±2.15ns 954.8±1.98ns -0.58%
Boolean Object Access (Execution) 5.7±0.03µs 5.7±0.01µs 0.00%
Boolean Object Access (Parser) 14.0±0.02µs 12.3±0.04µs -12.14%
Clean js (Compiler) 3.3±0.01µs 3.0±0.01µs -9.09%
Clean js (Execution) 1091.5±6.75µs 963.2±3.79µs -11.75%
Clean js (Parser) 28.0±0.05µs 28.1±0.03µs +0.36%
Create Realm 265.7±0.26ns 265.8±0.37ns +0.04%
Dynamic Object Property Access (Compiler) 1583.8±2.60ns 1394.7±5.30ns -11.94%
Dynamic Object Property Access (Execution) 6.6±0.03µs 6.7±0.03µs +1.52%
Dynamic Object Property Access (Parser) 10.3±0.03µs 10.4±0.02µs +0.97%
Fibonacci (Compiler) 2.2±0.00µs 1970.0±17.87ns -10.45%
Fibonacci (Execution) 1734.2±6.02µs 1731.7±3.34µs -0.14%
Fibonacci (Parser) 15.7±0.03µs 14.0±0.12µs -10.83%
For loop (Compiler) 1950.3±3.87ns 1718.3±6.12ns -11.90%
For loop (Execution) 41.8±0.08µs 41.7±0.10µs -0.24%
For loop (Parser) 13.4±0.02µs 13.5±0.01µs +0.75%
Mini js (Compiler) 3.2±0.01µs 3.2±0.01µs 0.00%
Mini js (Execution) 1003.0±7.42µs 1008.8±8.74µs +0.58%
Mini js (Parser) 24.7±0.05µs 24.6±0.04µs -0.40%
Number Object Access (Compiler) 1008.1±2.74ns 902.4±2.03ns -10.49%
Number Object Access (Execution) 4.4±0.02µs 4.5±0.01µs +2.27%
Number Object Access (Parser) 10.8±0.01µs 9.6±0.05µs -11.11%
Object Creation (Compiler) 1191.6±4.90ns 1232.8±4.44ns +3.46%
Object Creation (Execution) 5.3±0.03µs 6.0±0.01µs +13.21%
Object Creation (Parser) 8.9±0.02µs 8.0±0.02µs -10.11%
RegExp (Compiler) 1589.8±7.42ns 1609.3±16.03ns +1.23%
RegExp (Execution) 12.1±0.06µs 11.9±0.06µs -1.65%
RegExp (Parser) 8.6±0.02µs 8.7±0.02µs +1.16%
RegExp Creation (Compiler) 1367.8±6.15ns 1240.5±2.94ns -9.31%
RegExp Creation (Execution) 8.9±0.03µs 8.8±0.03µs -1.12%
RegExp Creation (Parser) 8.2±0.02µs 7.4±0.03µs -9.76%
RegExp Literal (Compiler) 1608.4±2.60ns 1402.9±6.51ns -12.78%
RegExp Literal (Execution) 12.1±0.09µs 12.0±0.07µs -0.83%
RegExp Literal (Parser) 7.0±0.01µs 7.1±0.02µs +1.43%
RegExp Literal Creation (Compiler) 1389.3±5.47ns 1201.6±3.82ns -13.51%
RegExp Literal Creation (Execution) 7.8±0.03µs 8.8±0.03µs +12.82%
RegExp Literal Creation (Parser) 6.3±0.02µs 5.6±0.03µs -11.11%
Static Object Property Access (Compiler) 1405.8±6.12ns 1242.6±7.18ns -11.61%
Static Object Property Access (Execution) 5.5±0.04µs 6.2±0.02µs +12.73%
Static Object Property Access (Parser) 9.6±0.01µs 9.8±0.02µs +2.08%
String Object Access (Compiler) 1271.1±7.29ns 1281.0±5.95ns +0.78%
String Object Access (Execution) 7.6±0.02µs 7.5±0.04µs -1.32%
String Object Access (Parser) 13.7±0.01µs 13.8±0.02µs +0.73%
String comparison (Compiler) 1846.9±10.08ns 1839.3±5.44ns -0.41%
String comparison (Execution) 5.6±0.01µs 5.0±0.02µs -10.71%
String comparison (Parser) 10.6±0.01µs 9.6±0.01µs -9.43%
String concatenation (Compiler) 1430.8±8.19ns 1448.0±5.86ns +1.20%
String concatenation (Execution) 5.1±0.02µs 5.2±0.02µs +1.96%
String concatenation (Parser) 6.4±0.01µs 6.5±0.01µs +1.56%
String copy (Compiler) 1125.2±2.60ns 1152.5±12.22ns +2.43%
String copy (Execution) 4.6±0.02µs 4.1±0.02µs -10.87%
String copy (Parser) 5.4±0.01µs 4.9±0.01µs -9.26%
Symbols (Compiler) 910.5±3.65ns 801.6±7.28ns -11.96%
Symbols (Execution) 4.4±0.01µs 4.4±0.01µs 0.00%
Symbols (Parser) 4.2±0.01µs 4.3±0.05µs +2.38%

@HalidOdat
Copy link
Member

bors r+

bors bot pushed a commit that referenced this pull request Mar 8, 2022
This fixes an issue with 262 negative tests, that should produce a syntax errors. Currently we only parse the test code is such cases. If the parsing does not return an error, we do not compile the code further. This caused some panics. Most of them are fixed by now, the last ones will be fixed with #1860.
@bors
Copy link

bors bot commented Mar 8, 2022

Pull request successfully merged into main.

Build succeeded:

@bors bors bot changed the title Compile StatementList after parse passes on negative tests [Merged by Bors] - Compile StatementList after parse passes on negative tests Mar 8, 2022
@bors bors bot closed this Mar 8, 2022
@bors bors bot deleted the tests-continue-after-parse branch March 8, 2022 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working Internal Category for changelog test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants