TS Incremental build exclude test files #95610

mshustov · 2021-03-27T10:17:08Z

Summary

part of #90281
Cold run before: 7m 25sec
Cold run after: 5m 20sec

This PR adds a centralized tsconfig.projects.json for all the plugins and excludes all the test files from being compiled when building TS project references. Because of this, bootstrap does less work: test and test utils aren't imported outside of your plugin code, so no need to calculate their public interfaces. Mocks are the only exception to this rule. We can add them back to the incremental builds as soon as we unify the naming schema for mock files.
As a by-product of this work, we created a list of patterns for test file names. Establishing a repo-wide convention for test file names will allow us to have control over them (remove test files from the distributable version, for example) and understand whether changes in files might trigger downstream plugin rebuild (will be used by Bazel build toolchain).

Known problems:

you have to run script/type_check --project tsconfig.json to initiate type check for plugin tests.
scripts/check_ts_projects slowed down on my laptop from 20sec. to 3min.5sec due to a huge number of include and exclude patterns added in this PR. We're going to address the problem when we establish a convention for test-related files in order to reduce the number of pattern names.

mshustov · 2021-03-29T11:30:51Z

src/dev/typescript/project.ts

@@ -26,6 +26,10 @@ function testMatchers(matchers: IMinimatch[], path: string) {
  return matchers.some((matcher) => matcher.match(path));
 }

+const parentProjectFactory = memoize(function (parentConfigPath: string) {


all the plugin config extend tsconfig.project.json, no need to recreate it 123 times.

mshustov · 2021-03-29T11:31:26Z

src/dev/typescript/project.ts


  constructor(
    public tsConfigPath: string,
    options: { name?: string; disableTypeCheck?: boolean } = {}
  ) {
    this.config = parseTsConfig(tsConfigPath);

-    const { files, include, exclude = [] } = this.config as {
+    const { files, include, exclude = [], extends: extendsPath } = this.config as {


extends in a reserved word

mshustov · 2021-03-29T11:32:31Z

x-pack/plugins/maps/public/components/validated_number_input.tsx

-// @ts-expect-error
-import { EuiFieldNumber, EuiFormRow, EuiFormRowDisplayKeys } from '@elastic/eui';
+import { EuiFieldNumber, EuiFormRow } from '@elastic/eui';
+import type { EuiFormRowDisplayKeys } from '@elastic/eui/src/components/form/form_row/form_row';


EuiFormRowDisplayKeys used to be inferred as any, fixed

mshustov · 2021-03-29T11:34:48Z

x-pack/plugins/uptime/public/components/certificates/cert_search.test.tsx

@@ -6,7 +6,7 @@
 */

 import React from 'react';
-import { renderWithRouter, shallowWithRouter } from '../../lib';
+import { renderWithRouter, shallowWithRouter } from '../../lib/helper/enzyme_helpers';


I'm surprised the exclusion of test utils didn't affect the dist size.

spong

Security Solution changes LGTM! Thanks! 🙂

patrykkopycinski

Asset management LGTM

mshustov · 2021-03-30T19:07:41Z

@elasticmachine merge upstream

chrisronline

LGTM!

mshustov · 2021-03-31T13:45:48Z

@elasticmachine merge upstream

mshustov · 2021-03-31T14:51:00Z

@elasticmachine merge upstream

poffdeluxe

Presentation team changes lgtm

mshustov · 2021-04-01T08:38:37Z

@elasticmachine merge upstream

kibanamachine · 2021-04-01T10:54:53Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`expressions`	151	150	-1
`uptime`	607	606	-1
total			-2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`uptime`	880.2KB	880.2KB	-47.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`expressions`	193.0KB	192.7KB	-349.0B

History

💚 Build #116549 succeeded d9fdd3f
💔 Build #116521 failed e7a00fc
💚 Build #116314 succeeded 6651b69
💛 Build #116070 was flaky 04ca8da
💚 Build #115725 succeeded 89912bf

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* add base config for all the TS projects * all the project use new tsconfig.project.json * compile test files in the high-level tsconfig.json * fix TS error in maps plugin * fix TS error in infra plugin * exclude mote test and test until folders * uptime. do not import test code within prod code * expressions. do not import test code within prod code * data: export mocks from high level folder * task_manager: comply with es client typings * infra: remove unused enzyme_helpers * check_ts_project requires "include" key * ts_check should handle parent configs * all ts configs should extend base one * exclude test folders from plugins * update patterns to fix ts_check errors * Apply suggestions from code review Co-authored-by: Constance <constancecchen@users.noreply.github.com> * uptime: MountWithReduxProvider to test helpers Co-authored-by: Constance <constancecchen@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

* TS Incremental build exclude test files (#95610) * add base config for all the TS projects * all the project use new tsconfig.project.json * compile test files in the high-level tsconfig.json * fix TS error in maps plugin * fix TS error in infra plugin * exclude mote test and test until folders * uptime. do not import test code within prod code * expressions. do not import test code within prod code * data: export mocks from high level folder * task_manager: comply with es client typings * infra: remove unused enzyme_helpers * check_ts_project requires "include" key * ts_check should handle parent configs * all ts configs should extend base one * exclude test folders from plugins * update patterns to fix ts_check errors * Apply suggestions from code review Co-authored-by: Constance <constancecchen@users.noreply.github.com> * uptime: MountWithReduxProvider to test helpers Co-authored-by: Constance <constancecchen@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> * fix code plugin tsconfig Co-authored-by: Constance <constancecchen@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

This reverts commit b6e582c.

…stic#96223) This reverts commit b6e582c.

…-nav * 'master' of github.com:elastic/kibana: (106 commits) [Lens] don't use eui variables for zindex (elastic#96117) Remove /src/legacy (elastic#95510) skip flaky suite (elastic#95899) [Dashboard] Fix Lens and TSVB chart tooltip positioning relative to global headers (elastic#94247) fixes a skipped management x-pack test (elastic#96178) [App Search] API logs: Add log detail flyout (elastic#96162) [tech-debt] Remove defunct opacity parameters from EUI shadow functions (elastic#96191) Add Input Controls project configuration (elastic#96238) [file upload] document file upload privileges and provide actionable UI when failures occur (elastic#95883) Revert "TS Incremental build exclude test files (elastic#95610)" (elastic#96223) [App Search] Added Sample Response section to Result Settings (elastic#95971) [Maps] Safe-erase text-field (elastic#94873) [RAC][Alert Triage][TGrid] Update the Alerts Table (TGrid) API to implement `renderCellValue` (elastic#96098) [Maps] Enable all zoom levels for all users (elastic#96093) Use plugin version in its publicPath (elastic#95945) [Enterprise Search] Expose core.chrome.setIsVisible for use in Workplace Search (elastic#95984) [Workplace Search] Add sub nav and fix rendering bugs in Personal dashboard (elastic#96100) [OBS]home page is showing incorrect value of APM throughput (tpm) (elastic#95991) [Observability] Exploratory View initial skeleton (elastic#94426) [KQL] Fixed styles of KQL textarea for the K8 theme (elastic#96190) ... # Conflicts: # x-pack/plugins/snapshot_restore/__jest__/client_integration/helpers/restore_snapshot.helpers.ts

…) (#96281) * Revert "TS Incremental build exclude test files (#95610)" (#96223) This reverts commit b6e582c. * code: use base tsconfig

mshustov added 8 commits March 26, 2021 17:12

add base config for all the TS projects

f7f6837

all the project use new tsconfig.project.json

c25a34c

compile test files in the high-level tsconfig.json

500bb42

fix TS error in maps plugin

70f6c22

fix TS error in infra plugin

34ca2d6

exclude mote test and test until folders

e3764cf

uptime. do not import test code within prod code

7fdd474

expressions. do not import test code within prod code

9482cf4

mshustov added chore v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.13.0 labels Mar 27, 2021

mshustov mentioned this pull request Mar 27, 2021

Exclude tests from incremental builds #92175

Closed

mshustov added 9 commits March 28, 2021 15:05

data: export mocks from high level folder

c92c3ce

Merge branch 'master' into incremental-without-tests

5e3d093

task_manager: comply with es client typings

a7ab3c5

infra: remove unused enzyme_helpers

cec2e04

check_ts_project requires "include" key

46aeb89

ts_check should handle parent configs

4849b49

all ts configs should extend base one

fc814d0

exclude test folders from plugins

dee62cc

update patterns to fix ts_check errors

89912bf

mshustov commented Mar 29, 2021

View reviewed changes

mshustov marked this pull request as ready for review March 29, 2021 17:01

mshustov requested a review from a team as a code owner March 29, 2021 17:01

mshustov requested a review from a team March 29, 2021 17:01

mshustov requested a review from a team as a code owner March 29, 2021 17:01

neptunian approved these changes Mar 30, 2021

View reviewed changes

spong approved these changes Mar 30, 2021

View reviewed changes

patrykkopycinski approved these changes Mar 30, 2021

View reviewed changes

Merge branch 'master' into incremental-without-tests

6651b69

kibanamachine requested a review from a team March 30, 2021 19:07

chrisronline approved these changes Mar 30, 2021

View reviewed changes

Merge branch 'master' into incremental-without-tests

e7a00fc

Merge branch 'master' into incremental-without-tests

d9fdd3f

poffdeluxe approved these changes Mar 31, 2021

View reviewed changes

Merge branch 'master' into incremental-without-tests

f26066a

mshustov merged commit b6e582c into elastic:master Apr 1, 2021

mshustov deleted the incremental-without-tests branch April 1, 2021 12:40

mshustov mentioned this pull request Apr 1, 2021

[7.x] TS Incremental build exclude test files (#95610) #96041

Merged

mshustov added a commit to mshustov/kibana that referenced this pull request Apr 5, 2021

Revert "TS Incremental build exclude test files (elastic#95610)"

767ddbb

This reverts commit b6e582c.

spalger pushed a commit that referenced this pull request Apr 5, 2021

Revert "TS Incremental build exclude test files (#95610)" (#96223)

e457f21

This reverts commit b6e582c.

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Apr 5, 2021

Revert "TS Incremental build exclude test files (elastic#95610)" (ela…

8617c8d

…stic#96223) This reverts commit b6e582c.

kibanamachine mentioned this pull request Apr 5, 2021

[7.x] Revert "TS Incremental build exclude test files (#95610)" (#96223) #96247

Closed

mshustov mentioned this pull request Apr 6, 2021

[7.x] Revert "TS Incremental build exclude test files (#95610)" (#96223) #96281

Merged

mshustov added a commit to mshustov/kibana that referenced this pull request Apr 6, 2021

Revert "TS Incremental build exclude test files (elastic#95610)" (ela…

a26c99b

…stic#96223) This reverts commit b6e582c.

mshustov added the reverted label Apr 6, 2021

mshustov added a commit that referenced this pull request Apr 6, 2021

[7.x] Revert "TS Incremental build exclude test files (#95610)" (#96223…

ad78665

…) (#96281) * Revert "TS Incremental build exclude test files (#95610)" (#96223) This reverts commit b6e582c. * code: use base tsconfig

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TS Incremental build exclude test files #95610

TS Incremental build exclude test files #95610

mshustov commented Mar 27, 2021 •

edited

Loading

mshustov Mar 29, 2021

mshustov Mar 29, 2021

mshustov Mar 29, 2021 •

edited

Loading

mshustov Mar 29, 2021

spong left a comment

patrykkopycinski left a comment

mshustov commented Mar 30, 2021

chrisronline left a comment

mshustov commented Mar 31, 2021

mshustov commented Mar 31, 2021

poffdeluxe left a comment

mshustov commented Apr 1, 2021

kibanamachine commented Apr 1, 2021

TS Incremental build exclude test files #95610

TS Incremental build exclude test files #95610

Conversation

mshustov commented Mar 27, 2021 • edited Loading

Summary

mshustov Mar 29, 2021

Choose a reason for hiding this comment

mshustov Mar 29, 2021

Choose a reason for hiding this comment

mshustov Mar 29, 2021 • edited Loading

Choose a reason for hiding this comment

mshustov Mar 29, 2021

Choose a reason for hiding this comment

spong left a comment

Choose a reason for hiding this comment

patrykkopycinski left a comment

Choose a reason for hiding this comment

mshustov commented Mar 30, 2021

chrisronline left a comment

Choose a reason for hiding this comment

mshustov commented Mar 31, 2021

mshustov commented Mar 31, 2021

poffdeluxe left a comment

Choose a reason for hiding this comment

mshustov commented Apr 1, 2021

kibanamachine commented Apr 1, 2021

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

Page load bundle

History

mshustov commented Mar 27, 2021 •

edited

Loading

mshustov Mar 29, 2021 •

edited

Loading