Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework environment records #1156

Merged
merged 7 commits into from
Apr 5, 2021
Merged

Conversation

0x7D2B
Copy link
Contributor

@0x7D2B 0x7D2B commented Mar 5, 2021

This Pull Request fixes/closes #989 by changing the environment traversal logic to use outer environments rather than the lexical environment stack.

This is implemented by moving much of LexicalEnvironment logic into EnvironmentRecordTrait and making it recursive, allowing for most of the tasks to be implemented without cloning any Gc values while still keeping the borrow checker happy.

In addition to that, FunctionEnvironmentRecord has been updated to use DeclarativeEnvironmentRecord to remove duplicated code. This is similar to how GlobalEnvironmentRecord currently works.

Finally, an additional test for #989 was added.

@0x7D2B 0x7D2B changed the title Lexical environments Rework environment records Mar 5, 2021
@0x7D2B
Copy link
Contributor Author

0x7D2B commented Mar 5, 2021

Is there a way to run the benchmarks on two different commits to compare them with each other and master? I have a different implementation of the fix in 1d0d73f so it would be nice to check which one is faster.

@jasonwilliams
Copy link
Member

jasonwilliams commented Mar 5, 2021

Not automatically but manually you can yes. You can look at the github workflow for benchmarks. It runs a benchmark, checks out another branch (in that case master) then runs the benchmarks again.

It starts from here:
https://github.com/boa-dev/criterion-compare-action/blob/master/main.js#L23

@0x7D2B
Copy link
Contributor Author

0x7D2B commented Mar 5, 2021

Hmm, seems like the actions don't want to run on my fork. Would adding (or removing) commits make the benchmarks run again? I can try rolling the PR back to 1d0d73f so that happens.

@jasonwilliams
Copy link
Member

You can still view them in https://github.com/boa-dev/boa/pull/1156/checks?check_run_id=2039359835 and check the bottom of the benchmark output. I think for non-members it can't print the results back to this PR @Razican should know more about this issue

@Razican
Copy link
Member

Razican commented Mar 5, 2021

Yes, unfortunately, benchmarks only show as comments for project members due to a GitHub limitation, but here we have them:

┌─────────┬──────────────────────────────────────────────┬─────────────────────┬──────────────────────┬────────────┐
│ (index) │                     name                     │   changesDuration   │    masterDuration    │ difference │
├─────────┼──────────────────────────────────────────────┼─────────────────────┼──────────────────────┼────────────┤
│    0    │     'Arithmetic operations (Execution)'      │   '348.0±7.60ns'    │ '**329.3±14.34ns**'  │   '+6.0'   │
│    1    │        'Arithmetic operations (Full)'        │ '**210.1±7.93µs**'  │    '214.5±9.30µs'    │   '-2.0'   │
│    2    │          'Array access (Execution)'          │    '5.8±0.19µs'     │   '**5.6±0.24µs**'   │   '+2.0'   │
│    3    │            'Array access (Full)'             │ '**240.3±7.02µs**'  │   '241.9±11.38µs'    │  '-0.99'   │
│    4    │         'Array creation (Execution)'         │    '2.8±0.07ms'     │   '**2.7±0.08ms**'   │   '+1.0'   │
│    5    │           'Array creation (Full)'            │    '2.9±0.08ms'     │   '**2.8±0.09ms**'   │   '+4.0'   │
│    6    │           'Array pop (Execution)'            │   '906.7±24.33µs'   │ '**871.6±25.81µs**'  │   '+4.0'   │
│    7    │              'Array pop (Full)'              │  '1311.9±29.90µs'   │ '**1254.7±49.92µs**' │   '+5.0'   │
│    8    │     'Boolean Object Access (Execution)'      │    '5.2±0.18µs'     │   '**4.7±0.19µs**'   │   '+9.0'   │
│    9    │        'Boolean Object Access (Full)'        │ '**229.2±8.13µs**'  │    '245.6±6.10µs'    │   '-6.5'   │
│   10    │            'Clean js (Execution)'            │   '641.1±14.60µs'   │ '**619.2±20.58µs**'  │   '+4.0'   │
│   11    │              'Clean js (Full)'               │ '**857.0±32.09µs**' │   '887.3±36.63µs'    │   '-3.8'   │
│   12    │             'Clean js (Parser)'              │    '40.2±1.44µs'    │  '**39.5±1.72µs**'   │   '+2.0'   │
│   13    │                'Create Realm'                │   '425.7±15.32ns'   │   '425.7±13.82ns'    │   '0.0'    │
│   14    │ 'Dynamic Object Property Access (Execution)' │    '4.7±0.17µs'     │   '**4.6±0.19µs**'   │   '+2.0'   │
│   15    │   'Dynamic Object Property Access (Full)'    │   '236.1±7.26µs'    │  '**230.9±8.66µs**'  │   '+2.0'   │
│   16    │            'Expression (Parser)'             │    '6.4±0.30µs'     │     '6.5±0.26µs'     │   '0.0'    │
│   17    │           'Fibonacci (Execution)'            │ '**744.8±16.00µs**' │   '785.4±29.07µs'    │   '-4.8'   │
│   18    │              'Fibonacci (Full)'              │ '**992.7±32.47µs**' │   '1028.0±40.72µs'   │   '-3.8'   │
│   19    │            'For loop (Execution)'            │  '**20.3±0.65µs**'  │    '20.9±0.85µs'     │   '-2.9'   │
│   20    │              'For loop (Full)'               │   '257.9±6.50µs'    │ '**251.9±10.79µs**'  │   '+2.0'   │
│   21    │             'For loop (Parser)'              │    '19.5±0.67µs'    │  '**19.2±0.81µs**'   │   '+1.0'   │
│   22    │           'Goal Symbols (Parser)'            │    '13.6±0.52µs'    │  '**13.5±0.68µs**'   │   '+1.0'   │
│   23    │            'Hello World (Parser)'            │    '3.8±0.14µs'     │   '**3.7±0.14µs**'   │   '+5.0'   │
│   24    │             'Long file (Parser)'             │   '742.9±23.58ns'   │ '**725.3±23.27ns**'  │   '+2.0'   │
│   25    │            'Mini js (Execution)'             │   '575.0±15.14µs'   │ '**554.5±19.14µs**'  │   '+4.0'   │
│   26    │               'Mini js (Full)'               │ '**787.1±29.30µs**' │   '813.7±39.39µs'    │   '-2.9'   │
│   27    │              'Mini js (Parser)'              │    '35.4±1.27µs'    │  '**35.2±1.58µs**'   │   '+1.0'   │
│   28    │      'Number Object Access (Execution)'      │    '4.1±0.09µs'     │   '**3.7±0.15µs**'   │   '+8.0'   │
│   29    │        'Number Object Access (Full)'         │ '**223.7±8.86µs**'  │    '238.7±4.60µs'    │   '-6.5'   │
│   30    │        'Object Creation (Execution)'         │    '4.0±0.10µs'     │   '**3.9±0.15µs**'   │   '+1.0'   │
│   31    │           'Object Creation (Full)'           │   '234.5±5.67µs'    │  '**225.3±8.87µs**'  │   '+4.0'   │
│   32    │             'RegExp (Execution)'             │    '10.5±0.32µs'    │  '**10.1±0.36µs**'   │   '+4.0'   │
│   33    │               'RegExp (Full)'                │ '**236.4±9.47µs**'  │    '251.5±8.29µs'    │   '-5.7'   │
│   34    │         'RegExp Literal (Execution)'         │    '10.3±0.29µs'    │  '**10.0±0.42µs**'   │   '+3.0'   │
│   35    │           'RegExp Literal (Full)'            │ '**233.7±10.50µs**' │    '238.3±9.49µs'    │   '-2.0'   │
│   36    │    'RegExp Literal Creation (Execution)'     │    '9.1±0.25µs'     │   '**8.7±0.28µs**'   │   '+5.0'   │
│   37    │       'RegExp Literal Creation (Full)'       │   '234.6±7.61µs'    │  '**232.4±8.11µs**'  │   '+1.0'   │
│   38    │ 'Static Object Property Access (Execution)'  │    '4.3±0.14µs'     │   '**4.2±0.15µs**'   │   '+2.0'   │
│   39    │    'Static Object Property Access (Full)'    │   '237.1±6.63µs'    │  '**226.1±9.96µs**'  │   '+5.0'   │
│   40    │      'String Object Access (Execution)'      │    '7.1±0.19µs'     │   '**6.5±0.29µs**'   │   '+10'    │
│   41    │        'String Object Access (Full)'         │ '**231.7±8.85µs**'  │    '236.5±9.66µs'    │   '-2.0'   │
│   42    │       'String comparison (Execution)'        │    '6.1±0.15µs'     │   '**5.9±0.22µs**'   │   '+3.0'   │
│   43    │          'String comparison (Full)'          │ '**232.8±8.99µs**'  │    '244.8±7.56µs'    │   '-4.8'   │
│   44    │      'String concatenation (Execution)'      │    '4.9±0.11µs'     │   '**4.8±0.21µs**'   │   '+3.0'   │
│   45    │        'String concatenation (Full)'         │ '**220.4±9.53µs**'  │    '238.6±5.23µs'    │   '-7.4'   │
│   46    │          'String copy (Execution)'           │  '**3.6±0.11µs**'   │     '3.7±0.15µs'     │   '-2.9'   │
│   47    │             'String copy (Full)'             │ '**216.8±7.68µs**'  │    '230.9±7.67µs'    │   '-6.5'   │
│   48    │            'Symbols (Execution)'             │    '3.1±0.10µs'     │   '**3.0±0.11µs**'   │   '+3.0'   │
│   49    │               'Symbols (Full)'               │   '220.8±4.72µs'    │  '**215.9±9.28µs**'  │   '+2.0'   │
│   50    │                      ''                      │      undefined      │      undefined       │   '+NaN'   │
└─────────┴──────────────────────────────────────────────┴─────────────────────┴──────────────────────┴────────────┘

@0x7D2B
Copy link
Contributor Author

0x7D2B commented Mar 5, 2021

Ouch, increases all across the board. I wonder if this could be improved at least a little bit by overriding the default methods for creating and getting bindings, given that they do runtime checks for environment types right now?

@0x7D2B
Copy link
Contributor Author

0x7D2B commented Mar 6, 2021

Benchmarks for iterator implementation:

┌─────────┬──────────────────────────────────────────────┬──────────────────────┬──────────────────────┬────────────┐
│ (index) │                     name                     │   changesDuration    │    masterDuration    │ difference │
├─────────┼──────────────────────────────────────────────┼──────────────────────┼──────────────────────┼────────────┤
│    0    │     'Arithmetic operations (Execution)'      │ '**381.2±17.79ns**'  │   '405.2±32.43ns'    │   '-5.7'   │
│    1    │        'Arithmetic operations (Full)'        │   '280.8±16.71µs'    │  '**270.6±9.96µs**'  │   '+4.0'   │
│    2    │          'Array access (Execution)'          │     '7.6±0.58µs'     │   '**7.5±0.42µs**'   │   '+1.0'   │
│    3    │            'Array access (Full)'             │   '307.0±21.41µs'    │ '**302.5±16.38µs**'  │   '+1.0'   │
│    4    │         'Array creation (Execution)'         │   '**3.1±0.15ms**'   │     '3.1±0.16ms'     │  '-0.99'   │
│    5    │           'Array creation (Full)'            │     '3.4±0.19ms'     │   '**3.4±0.18ms**'   │   '+3.0'   │
│    6    │           'Array pop (Execution)'            │ '**979.1±34.46µs**'  │   '1018.2±58.59µs'   │   '-3.8'   │
│    7    │              'Array pop (Full)'              │   '1535.1±80.91µs'   │ '**1516.0±75.90µs**' │   '+1.0'   │
│    8    │     'Boolean Object Access (Execution)'      │   '**5.9±0.25µs**'   │     '6.1±0.34µs'     │   '-2.0'   │
│    9    │        'Boolean Object Access (Full)'        │   '300.2±26.82µs'    │   '300.2±45.17µs'    │   '0.0'    │
│   10    │            'Clean js (Execution)'            │   '798.2±47.05µs'    │ '**774.2±77.63µs**'  │   '+3.0'   │
│   11    │              'Clean js (Full)'               │  '1149.9±101.17µs'   │ '**1093.0±63.10µs**' │   '+5.0'   │
│   12    │             'Clean js (Parser)'              │    '50.8±4.36µs'     │  '**49.0±1.94µs**'   │   '+4.0'   │
│   13    │                'Create Realm'                │   '527.9±31.14ns'    │ '**496.8±32.51ns**'  │   '+6.0'   │
│   14    │ 'Dynamic Object Property Access (Execution)' │   '**6.2±0.40µs**'   │     '6.4±0.74µs'     │   '-2.9'   │
│   15    │   'Dynamic Object Property Access (Full)'    │ '**299.4±14.38µs**'  │   '303.3±22.06µs'    │  '-0.99'   │
│   16    │            'Expression (Parser)'             │     '7.8±0.58µs'     │   '**7.5±0.35µs**'   │   '+3.0'   │
│   17    │           'Fibonacci (Execution)'            │ '**1008.9±59.66µs**' │   '1068.5±63.24µs'   │   '-5.7'   │
│   18    │              'Fibonacci (Full)'              │ '**1308.2±70.43µs**' │  '1390.6±134.77µs'   │   '-5.7'   │
│   19    │            'For loop (Execution)'            │  '**28.3±1.48µs**'   │    '28.6±5.40µs'     │  '-0.99'   │
│   20    │              'For loop (Full)'               │   '327.3±17.02µs'    │   '326.6±16.11µs'    │   '0.0'    │
│   21    │             'For loop (Parser)'              │    '24.2±1.51µs'     │  '**23.5±1.12µs**'   │   '+3.0'   │
│   22    │           'Goal Symbols (Parser)'            │  '**16.3±1.14µs**'   │    '16.5±0.99µs'     │   '-2.0'   │
│   23    │            'Hello World (Parser)'            │   '**4.3±0.25µs**'   │     '4.4±0.26µs'     │  '-0.99'   │
│   24    │             'Long file (Parser)'             │   '853.6±49.44ns'    │ '**846.3±44.72ns**'  │   '+1.0'   │
│   25    │            'Mini js (Execution)'             │   '713.0±49.77µs'    │ '**683.3±42.93µs**'  │   '+4.0'   │
│   26    │               'Mini js (Full)'               │   '1032.2±98.54µs'   │ '**989.8±52.57µs**'  │   '+4.0'   │
│   27    │              'Mini js (Parser)'              │    '44.2±2.72µs'     │  '**43.6±2.39µs**'   │   '+1.0'   │
│   28    │      'Number Object Access (Execution)'      │     '4.7±0.42µs'     │   '**4.6±0.21µs**'   │   '+2.0'   │
│   29    │        'Number Object Access (Full)'         │ '**292.1±29.96µs**'  │   '294.9±21.24µs'    │  '-0.99'   │
│   30    │        'Object Creation (Execution)'         │     '5.2±0.32µs'     │     '5.3±0.30µs'     │   '0.0'    │
│   31    │           'Object Creation (Full)'           │   '296.3±16.98µs'    │ '**293.7±19.50µs**'  │   '+1.0'   │
│   32    │             'RegExp (Execution)'             │  '**12.6±0.69µs**'   │    '12.9±1.27µs'     │   '-2.0'   │
│   33    │               'RegExp (Full)'                │   '307.9±12.40µs'    │   '308.8±13.80µs'    │   '0.0'    │
│   34    │         'RegExp Literal (Execution)'         │  '**12.5±0.53µs**'   │    '12.7±0.72µs'     │   '-2.0'   │
│   35    │           'RegExp Literal (Full)'            │   '307.6±13.40µs'    │ '**304.8±21.42µs**'  │   '+1.0'   │
│   36    │    'RegExp Literal Creation (Execution)'     │  '**11.0±0.56µs**'   │    '11.3±1.14µs'     │   '-3.8'   │
│   37    │       'RegExp Literal Creation (Full)'       │   '298.8±17.81µs'    │   '297.9±14.23µs'    │   '0.0'    │
│   38    │ 'Static Object Property Access (Execution)'  │   '**5.6±0.37µs**'   │     '5.6±0.43µs'     │   '-2.0'   │
│   39    │    'Static Object Property Access (Full)'    │   '305.2±18.54µs'    │ '**296.2±15.70µs**'  │   '+3.0'   │
│   40    │      'String Object Access (Execution)'      │     '8.4±0.44µs'     │     '8.4±0.62µs'     │   '0.0'    │
│   41    │        'String Object Access (Full)'         │   '301.0±15.21µs'    │  '**298.2±9.51µs**'  │   '+1.0'   │
│   42    │       'String comparison (Execution)'        │     '8.0±0.55µs'     │   '**7.6±0.33µs**'   │   '+6.0'   │
│   43    │          'String comparison (Full)'          │   '299.2±14.61µs'    │   '297.9±21.44µs'    │   '0.0'    │
│   44    │      'String concatenation (Execution)'      │     '6.4±0.71µs'     │   '**6.2±0.25µs**'   │   '+4.0'   │
│   45    │        'String concatenation (Full)'         │   '289.9±12.11µs'    │   '289.0±12.35µs'    │   '0.0'    │
│   46    │          'String copy (Execution)'           │   '**4.7±0.17µs**'   │     '4.8±1.06µs'     │  '-0.99'   │
│   47    │             'String copy (Full)'             │   '286.0±17.30µs'    │   '287.4±13.88µs'    │   '0.0'    │
│   48    │            'Symbols (Execution)'             │   '**4.2±0.21µs**'   │     '4.2±0.22µs'     │  '-0.99'   │
│   49    │               'Symbols (Full)'               │ '**276.0±12.33µs**'  │   '280.2±13.36µs'    │   '-2.0'   │
│   50    │                      ''                      │      undefined       │      undefined       │   '+NaN'   │
└─────────┴──────────────────────────────────────────────┴──────────────────────┴──────────────────────┴────────────┘

@0x7D2B
Copy link
Contributor Author

0x7D2B commented Mar 6, 2021

I noticed a regression in test262 results, the source of which appeared to originate in the test harness. It seems like #989 was masking a different issue with some parts of Boa not cleaning up the environments that they pushed, particularly when returning early. I made some fixes, but I think this needs a closer look.

After the fixes, test262 results actually improved:

Test262 conformance changes:

Test result master count PR count difference
Total 78,497 78,497 0
Passed 24,986 25,139 +153
Ignored 15,587 15,587 0
Failed 37,924 37,771 -153
Panics 16 18 +2
Conformance 31.83 32.03 +0.19%

Not sure what those two extra panics are, I don't seem to be getting them on my end:
total: 78497, passed: 25143, ignored: 15587, failed: 37767 (panics: 16 ⚠), conformance: 32.03%

@RageKnify
Copy link
Member

The panics might be some panics that seem to appear and disappear in CI, when it's exactly 2 it can be ok to ignore it.

@Razican Razican added this to the v0.12.0 milestone Mar 14, 2021
@Razican Razican added bug Something isn't working execution Issues or PRs related to code execution labels Mar 14, 2021
@0x7D2B 0x7D2B mentioned this pull request Mar 22, 2021
Copy link
Member

@jasonwilliams jasonwilliams left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @0x7D2B thanks so much for taking this on, sorry the review wait was long. Yeah the old system of just iterating through all environments wasn't a great one but got us this far, I'm glad we have something more robust.

@jasonwilliams jasonwilliams merged commit 57bd79a into boa-dev:master Apr 5, 2021
@jasonwilliams
Copy link
Member

jasonwilliams commented Apr 5, 2021

@0x7D2B I've merged this for now, but I think the environment_stack can go, it was only there as an old implementation of environment traversal. As we will only ever get the back element I think it can be replaced by an env placeholder which only holds the current env.

e.g https://github.com/boa-dev/boa/blob/master/boa/src/environment/lexical_environment.rs#L154-L168 both of these can be replaced with self.env

Issue created #1197

Razican pushed a commit that referenced this pull request May 22, 2021
* Add test for function scope
* Make the environment iterator follow outer environments
* Make FunctionEnvironmentRecord reuse code
* Refactor environment records
* Override default methods instead of checking environment types
* Fix various environment cleanup issues
* Fix lint issues
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working execution Issues or PRs related to code execution
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Functions seem to have access to their caller's scope
4 participants