Skip to content

Commit

Permalink
Update docs for runtime enabled features and web tests
Browse files Browse the repository at this point in the history 
Change-Id: I7230bd37e7f9c1c136e1c23b6acdc340eb089121
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/2303452
Reviewed-by: Philip Rogers <pdr@chromium.org>
Commit-Queue: Xianzhu Wang <wangxianzhu@chromium.org>
Cr-Commit-Position: refs/heads/master@{#789247}
  • Loading branch information
@wangxianzhu
wangxianzhu authored and Commit Bot committed Jul 16, 2020
1 parent f98aa5d commit adb0670
Show file tree
Hide file tree
Showing 2 changed files with 20 additions and 10 deletions.
16 changes: 12 additions & 4 deletions docs/testing/web_tests.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -217,7 +217,7 @@ on this.

There are two ways to run web tests with additional command-line arguments:

* Using `--additional-driver-flag`:
* Using `--additional-driver-flag` or `--flag-specific`:

```bash
third_party/blink/tools/run_web_tests.py --additional-driver-flag=--blocking-repaint
Expand Down Expand Up @@ -300,16 +300,24 @@ There are two ways to run web tests with additional command-line arguments:
with the flags without creating any virtual tests.

For flags whose implementation is still in progress, virtual test suites and
flag-specific expectations represent two alternative strategies for testing.
Consider the following when choosing between them:
flag-specific expectations represent two alternative strategies for testing both
the enabled code path and not-enabled code path. They are preferred to only
setting a [runtime enabled feature](../../third_party/blink/renderer/platform/RuntimeEnabledFeatures.md)
to `status: "test"` if the feature has substantially different code path from
production because the latter would cause loss of test coverage of the production
code path.

Consider the following when choosing between virtual test suites and
flag-specific expectations:

* The
[waterfall builders](https://dev.chromium.org/developers/testing/chromium-build-infrastructure/tour-of-the-chromium-buildbot)
and [try bots](https://dev.chromium.org/developers/testing/try-server-usage)
will run all virtual test suites in addition to the non-virtual tests.
Conversely, a flag-specific expectations file won't automatically cause the
bots to test your flag - if you want bot coverage without virtual test suites,
you will need to set up a dedicated bot for your flag.
you will need to set up a dedicated bot ([example](https://chromium-review.googlesource.com/c/chromium/src/+/1850255))
for your flag.
* Due to the above, virtual test suites incur a performance penalty for the
commit queue and the continuous build infrastructure. This is exacerbated by
Expand Down
14 changes: 8 additions & 6 deletions third_party/blink/renderer/platform/RuntimeEnabledFeatures.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ Example:
```
The status of the feature controls when it will be enabled in the Blink engine.

| Status Value | Web feature enabled during web tests with content_shell [1] | Web feature enabled as part of web experimental features [2] | Web feature enabled in stable release | Non-web exposed feature enabled through a command line flag [3]
| Status Value | Web feature enabled during [web tests] with content_shell [1] | Web feature enabled as part of web experimental features [2] | Web feature enabled in stable release | Non-web exposed feature enabled through a command line flag [3]
|:---:|:---:|:---:|:---:|:---:|
| <missing\> | No | No | No | Yes |
| `test` | Yes | No | No | No |
Expand Down Expand Up @@ -134,16 +134,17 @@ interface ExistingObject {
**Warning:** You will not be able to change the enabled state of these at runtime as the V8 object templates definitions are created during start up and will not be updated during runtime.
## Web Tests (JavaScript)
Test whether a feature is enabled using:
In [web tests], you can test whether a feature is enabled using:
```javascript
internals.runtimeFlags.amazingNewFeatureEnabled
```
This attribute is read only and cannot be changed.
This attribute is read only and cannot be changed, unless `settable_from_internals: true` is specified for the feature.

**Note:** The `internals` JavaScript API is only available in ContentShell for use by web tests and does not appear in released versions of Chromium.
**Note:** The `internals` JavaScript API is only available in content_shell for use by web tests and does not appear in Chromium. In content_shell's browser mode, `--expose-internals-for-testing` is needed to have the `internals` JavaScript API.

### Running Web Tests
When content_shell is run with `--stable-release-mode` flag, test-only features (ones listed in [runtime_enabled_features.json5] with `test`) are turned off.
When content_shell is run for web tests with `--stable-release-mode` flag, test-only and experimental features (ones listed in [runtime_enabled_features.json5] with `status: "test"` or `status: "experimental"`) are turned off. The [virtual/stable] suite runs with the flag, which is one of the ways to ensure test coverage of production code path for these features.

## Generated Files
[renderer/build/scripts/make_runtime_features.py][make_runtime_features.py] uses [runtime_enabled_features.json5] to generate:
Expand Down Expand Up @@ -173,7 +174,7 @@ After applying most other feature settings, the features requested feature setti
https://groups.google.com/a/chromium.org/d/msg/blink-dev/JBakhu5J6Qs/re2LkfEslTAJ



[web tests]: <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_tests.md>
[supportedPlatforms]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/platform/runtime_enabled_features.json5#36>
[cssProperties]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/core/css/css_properties.json5>
[virtual test suite]: <https://chromium.googlesource.com/chromium/src/+/master/docs/testing/web_tests.md#testing-runtime-flags>
Expand All @@ -187,3 +188,4 @@ https://groups.google.com/a/chromium.org/d/msg/blink-dev/JBakhu5J6Qs/re2LkfEslTA
[runtime_enabled_features.json5]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/platform/runtime_enabled_features.json5>
[make_internal_runtime_flags.py]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/build/scripts/make_internal_runtime_flags.py>
[code_generator_v8.py]: <https://chromium.googlesource.com/chromium/src/+/master/third_party/blink/renderer/bindings/scripts/code_generator_v8.py>
[virtual/stable]: <https://source.chromium.org/chromium/chromium/src/+/master:third_party/blink/web_tests/VirtualTestSuites;drc=9878f26d52d32871ed1c085444196e5453909eec;l=112>

0 comments on commit adb0670

Please sign in to comment.