Skip to content

Commit

Permalink
Adds new optional fields to the JSON results merger script
Browse files Browse the repository at this point in the history
These new fields can be useful for debugging regressions due to
invalid test expectations. It also updates the JSON results documentation
to include the new fields as well as the optional test_name_prefix field
which was added in a previous CL.

Bug: chromium:698902
Change-Id: Icc0edd59a2a3b585fb793a70355ee585cac28fad
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/1546257
Reviewed-by: Kenneth Russell <kbr@chromium.org>
Commit-Queue: Rakib Hasan <rmhasan@google.com>
Cr-Commit-Position: refs/heads/master@{#646000}
  • Loading branch information
Rakib M. Hasan authored and Commit Bot committed Mar 29, 2019
1 parent 81ebfff commit 92075cd
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 1 deletion.
3 changes: 3 additions & 0 deletions docs/testing/json_test_results_format.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,9 @@ object may appear in any order.
| `artifact_permanent_location` | string | **Optional.** The URI of the root location where the artifacts are stored. If present, any artifact locations are taken to be relative to this location. Currently only the `gs://` scheme is supported. |
| `build_number` | string | **Optional.** If this test run was produced on a bot, this should be the build number of the run, e.g., "1234". |
| `builder_name` | string | **Optional.** If this test run was produced on a bot, this should be the builder name of the bot, e.g., "Linux Tests". |
| `test_name_prefix` | string | **Optional.** If a prefix was specified to be removed from test names, then that prefix can be placed here. |
| `expectations_files` | array of strings | **Optional.** If a list of expectations file paths are passed to the test runner, then that list can be placed here. |
| `tags` | array of strings | **Optional.** If a list of test expectations file tags are passed to the test runner, then that list can be placed here. |
| `chromium_revision` | string | **Optional.** The revision of the current Chromium checkout, if relevant, e.g. "356123". |
| `has_pretty_patch` | bool | **Optional, layout test specific, deprecated.** Whether the web tests' output contains PrettyDiff-formatted diffs for test failures. |
| `has_wdiff` | bool | **Optional, layout test specific, deprecated.** Whether the web tests' output contains wdiff-formatted diffs for test failures. |
Expand Down
4 changes: 3 additions & 1 deletion testing/merge_scripts/results_merger.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@
'path_delimiter',
'pixel_tests_enabled',
'random_order_seed',
'test_name_prefix'
'test_name_prefix',
'tags',
'expectations_files'
)

OPTIONAL_IGNORED = (
Expand Down

0 comments on commit 92075cd

Please sign in to comment.