Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated the Test262 submodule #1499

Merged
merged 1 commit into from
Aug 24, 2021
Merged

Updated the Test262 submodule #1499

merged 1 commit into from
Aug 24, 2021

Conversation

Razican
Copy link
Member

@Razican Razican commented Aug 23, 2021

This updates the Test262 submodule to have the latest tests from the official suite.

@Razican Razican added the test Issues and PRs related to the tests. label Aug 23, 2021
@Razican Razican added this to the v0.13.0 milestone Aug 23, 2021
@github-actions
Copy link

Test262 conformance changes:

Test result master count PR count difference
Total 78,897 80,685 +1,788
Passed 31,283 31,410 +127
Ignored 15,612 15,818 +206
Failed 32,002 33,457 +1,455
Panics 2 2 0
Conformance 39.65% 38.93% -0.72%
Broken tests:
test/built-ins/Boolean/prototype/S15.6.3.1_A3.js [strict mode] (previously Passed)

@github-actions
Copy link

Benchmark for cb15ba6

Click to view benchmark
Test PR Benchmark Master Benchmark %
Arithmetic operations (Execution) 316.0±15.08ns 299.6±18.11ns +5.47%
Arithmetic operations (Full) 367.9±22.45µs 360.5±12.86µs +2.05%
Array access (Execution) 8.3±0.44µs 8.3±0.35µs 0.00%
Array access (Full) 398.5±37.15µs 406.0±32.42µs -1.85%
Array creation (Execution) 3.3±0.14ms 3.5±0.36ms -5.71%
Array creation (Full) 3.8±0.18ms 3.7±0.14ms +2.70%
Array pop (Execution) 1102.8±111.90µs 1075.5±51.56µs +2.54%
Array pop (Full) 1668.5±112.29µs 1700.4±152.54µs -1.88%
Boolean Object Access (Execution) 6.4±0.75µs 6.6±0.42µs -3.03%
Boolean Object Access (Full) 391.8±21.76µs 384.7±10.47µs +1.85%
Clean js (Execution) 835.5±68.58µs 839.1±34.05µs -0.43%
Clean js (Full) 1242.0±65.73µs 1242.7±47.58µs -0.06%
Clean js (Parser) 39.9±2.29µs 40.1±2.98µs -0.50%
Create Realm 474.8±43.06ns 459.8±32.10ns +3.26%
Dynamic Object Property Access (Execution) 6.4±0.45µs 6.8±0.55µs -5.88%
Dynamic Object Property Access (Full) 388.8±15.63µs 394.3±19.46µs -1.39%
Expression (Parser) 6.9±0.56µs 6.7±0.24µs +2.99%
Fibonacci (Execution) 993.7±66.15µs 988.1±35.17µs +0.57%
Fibonacci (Full) 1414.4±72.44µs 1433.3±68.34µs -1.32%
For loop (Execution) 23.5±0.86µs 23.8±1.63µs -1.26%
For loop (Full) 397.5±22.23µs 403.0±20.19µs -1.36%
For loop (Parser) 19.9±4.38µs 19.6±2.46µs +1.53%
Goal Symbols (Parser) 13.7±0.63µs 14.0±1.04µs -2.14%
Hello World (Parser) 4.0±0.38µs 3.8±0.15µs +5.26%
Long file (Parser) 880.0±55.01ns 920.7±91.51ns -4.42%
Mini js (Execution) 765.0±34.09µs 763.6±42.23µs +0.18%
Mini js (Full) 1185.3±81.23µs 1170.0±47.04µs +1.31%
Mini js (Parser) 35.1±1.32µs 35.2±2.39µs -0.28%
Number Object Access (Execution) 5.1±0.33µs 5.3±1.57µs -3.77%
Number Object Access (Full) 372.6±13.32µs 386.7±21.01µs -3.65%
Object Creation (Execution) 6.2±0.50µs 5.9±0.33µs +5.08%
Object Creation (Full) 380.1±18.95µs 382.8±21.21µs -0.71%
RegExp (Execution) 15.4±1.01µs 15.6±1.07µs -1.28%
RegExp (Full) 402.3±24.38µs 402.7±32.76µs -0.10%
RegExp Literal (Execution) 15.4±1.33µs 15.5±0.92µs -0.65%
RegExp Literal (Full) 400.1±16.04µs 414.5±29.81µs -3.47%
RegExp Literal Creation (Execution) 11.2±0.53µs 11.2±0.70µs 0.00%
RegExp Literal Creation (Full) 388.8±15.72µs 395.1±26.58µs -1.59%
Static Object Property Access (Execution) 6.3±0.55µs 6.4±0.55µs -1.56%
Static Object Property Access (Full) 380.0±14.27µs 392.8±31.89µs -3.26%
String Object Access (Execution) 8.8±0.55µs 8.8±0.46µs 0.00%
String Object Access (Full) 395.6±28.20µs 393.9±19.32µs +0.43%
String comparison (Execution) 8.3±0.43µs 8.2±0.39µs +1.22%
String comparison (Full) 403.3±42.36µs 387.0±15.32µs +4.21%
String concatenation (Execution) 6.7±0.60µs 6.6±0.38µs +1.52%
String concatenation (Full) 386.6±27.59µs 389.5±25.23µs -0.74%
String copy (Execution) 5.2±0.31µs 5.2±0.28µs 0.00%
String copy (Full) 381.3±24.86µs 389.8±26.66µs -2.18%
Symbols (Execution) 4.4±0.43µs 4.4±0.13µs 0.00%
Symbols (Full) 364.5±26.35µs 360.5±23.16µs +1.11%

@jedel1043
Copy link
Member

jedel1043 commented Aug 23, 2021

Failing test:

// CHECK#1
verifyNotConfigurable(Boolean, "prototype");

assert.throws(TypeError, () => {
  delete Boolean.prototype;
});

Previous version of the same test:

// CHECK#1
verifyNotConfigurable(Boolean, "prototype");

try {
  if (delete Boolean.prototype !== false) {
    $ERROR('#1: Boolean.prototype has the attribute DontDelete');
  }
} catch (e) {
  if (e instanceof Test262Error) throw e;
  assert(e instanceof TypeError);
}

delete should throw an error on strict mode, but on

Node::GetConstField(ref get_const_field) => JsValue::new(
get_const_field
.obj()
.run(context)?
.to_object(context)?
.__delete__(&get_const_field.field().into()),
),
Node::GetField(ref get_field) => {
let obj = get_field.obj().run(context)?;
let field = &get_field.field().run(context)?;
let res = obj
.to_object(context)?
.__delete__(&field.to_property_key(context)?);
return Ok(JsValue::new(res));
}
Node::Identifier(_) => JsValue::new(false),

we don't consider strict mode when deleting.

@HalidOdat
Copy link
Member

@jedel1043 I think we should leave fixing broken tests for a separate PR, we should create an issue for this to keep track of it :)

@jedel1043
Copy link
Member

@jedel1043 I think we should leave fixing broken tests for a separate PR, we should create an issue for this to keep track of it :)

Yep, just commenting to keep track on what we need to fix

@Razican Razican merged commit 2712aaa into master Aug 24, 2021
@jasonwilliams jasonwilliams deleted the test_update branch October 5, 2021 21:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants