Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ui/UiApp] sort used modules so entry files can be cache keys #15910

Merged

Conversation

spalger
Copy link
Contributor

@spalger spalger commented Jan 9, 2018

Fixes elastic/kibana-docker#66

The order of appExtensions in uiExports is based on the load order of plugins, and can change even without a plugin being installed. This is currently causing the modules list for each uiApp to come back in slightly different orders every once and a while, which causes the {app}.entry.js files to have slightly different contents, triggering a run of the optimizer. This is especially painful because of how expensive optimization has become.

To fix this, uiApp will now sort the appExtension module ids it collects from uiExports before providing them to the bundle files. They need to be sorted here (rather than in the bundle itself) so that we can verify that the main file for the app is still the first module in the list. The bundles will just render the modules list provided to them by their uiApp.

@spalger spalger added review Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v7.0.0 v6.2.0 v6.1.2 labels Jan 9, 2018
@spalger
Copy link
Contributor Author

spalger commented Jan 9, 2018

The only way I know how to test this is to run bin/kibana in a loop, waiting for it to require optimization on startup.

# patch kibana to exit immediately after optimization
curl https://gist.githubusercontent.com/anonymous/49acf9ef2c8a7c60de93d030b3aa0989/raw/9dd0a0489e6b602f1d3ef43dbf85ab43258fae32/gistfile.diff | git apply

# optimize once to get a baseline
node scripts/kibana

# copy the optimize bundles directory for comparison once the cache is considered invalid
cp -r optimize/bundles optimize/valid_bundles

# run bin/kibana in a loop until it fails to reuse the cache
while true; do echo ""; echo "running..."; node scripts/kibana --verbose || break; done;

# compare the optimize/bundles/*.entry.js files with the optimize/valid_bundles/*.entry.js files

In master this will run a few times and eventually fail to reuse the cache. If you can't get it to happen, try including x-pack. I find it triggers more often when x-pack is installed.

@spalger spalger force-pushed the fix/unnecessary-optimize-invalidation branch from 87c45db to b49258a Compare January 9, 2018 02:10
@spalger spalger force-pushed the fix/unnecessary-optimize-invalidation branch from b49258a to fe3d1dd Compare January 9, 2018 02:11
@spalger
Copy link
Contributor Author

spalger commented Jan 9, 2018

jenkins, test this

Copy link
Contributor

@kimjoar kimjoar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a unit test or two for this? We already have a couple tests for getModules, but they are not capturing this problem.

@LeeDr LeeDr added the blocker label Jan 9, 2018
@spalger spalger force-pushed the fix/unnecessary-optimize-invalidation branch from 35fb6c0 to fd24ae5 Compare January 9, 2018 17:55
@spalger spalger force-pushed the fix/unnecessary-optimize-invalidation branch from fd24ae5 to 8017341 Compare January 9, 2018 17:56
@spalger
Copy link
Contributor Author

spalger commented Jan 9, 2018

Good call @kjbekkelund, added tests for UiApp and also a couple that verify the assumptions this maintains for UiBundle and the appEntryTemplate

@spalger
Copy link
Contributor Author

spalger commented Jan 9, 2018

jenkins, test this

@rhoboat
Copy link

rhoboat commented Jan 9, 2018

This is a nice change, thanks @spalger

@spalger spalger merged commit 2ee17cd into elastic:master Jan 9, 2018
spalger added a commit to spalger/kibana that referenced this pull request Jan 9, 2018
…c#15910)

* [ui/UiApp] sort used modules so entry files can be cache keys

* [ui/bundles+apps] test changes and verify assumptions down to appEntryTemplate
@spalger spalger removed the v6.1.2 label Jan 9, 2018
spalger added a commit that referenced this pull request Jan 9, 2018
…15910) (#15941)

* [ui/UiApp] sort used modules so entry files can be cache keys

* [ui/bundles+apps] test changes and verify assumptions down to appEntryTemplate
@spalger
Copy link
Contributor Author

spalger commented Jan 9, 2018

6.1/6.x: a676f08

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocker review Team:Core Core services & architecture: plugins, logging, config, saved objects, http, ES client, i18n, etc v6.2.0 v7.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimize runs on startup, even with default settings.
4 participants