Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[full-ci] Various performance and structure optimizations #5194

Merged
merged 39 commits into from
Jun 18, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
ccbe013
skip resource loading on personal view route redirect
fschade Jun 2, 2021
71f7c79
add basic support for typescript
fschade Jun 2, 2021
65efc27
update eslint and install some packages to make eslint-config-standar…
fschade Jun 3, 2021
d6b4dba
fix unnecessary usage of var
fschade Jun 3, 2021
d3528ef
configure jest to use typescript specs also
fschade Jun 3, 2021
b8f2a36
basic cache implementation
fschade Jun 3, 2021
255adb0
add elementIsVisible util which enables us to only run functions if a…
fschade Jun 4, 2021
4b40edf
make it possible for visibility observer to observe multiple elements…
fschade Jun 7, 2021
161d50f
implement lazy loading for images and avatars
fschade Jun 9, 2021
e7701f3
forgot to add new files
fschade Jun 9, 2021
472c55b
mock IntersectionObserver
fschade Jun 9, 2021
e5ce4b6
remove unused files
fschade Jun 9, 2021
7fdbeab
enable resource preview cache by default
fschade Jun 9, 2021
7710c20
Use authenticated request for avatarUrl
pascalwengerter Jun 10, 2021
4656491
Rebase on master, remove obsolete loadPreviews action
pascalwengerter Jun 10, 2021
d556f83
fix cache evict and ttl
fschade Jun 10, 2021
b58fedf
add constants for most used image dimensions
fschade Jun 15, 2021
9dacc9a
skip axios mock on shared with me
fschade Jun 16, 2021
02fbedc
fix lint glob
fschade Jun 16, 2021
d7aac09
test avatarUrl caching
fschade Jun 16, 2021
ec21f7e
use lodash debounce instead of own debounce implementation
fschade Jun 17, 2021
bf47164
use ImageDimension.Avatar as default in avatarUrl spec
fschade Jun 17, 2021
926a9a4
check error case in avatarUrl spec
fschade Jun 17, 2021
8d430a0
add cache spec
fschade Jun 17, 2021
65b8932
fix 0 falsy eval in buildQueryString
fschade Jun 17, 2021
127920f
Add changelog drafts for the performance improvements PR
pascalwengerter Jun 17, 2021
1f4a5d1
add tests for privatePreviewBlob
fschade Jun 17, 2021
cad9c19
chaeck image dimensions while retrieving privatePreviewBlob entries f…
fschade Jun 17, 2021
3a19aa6
add tests for publicPreviewUrl
fschade Jun 17, 2021
7e9ed3b
fix client cancel token bug
fschade Jun 17, 2021
7ac1472
remove code smell
fschade Jun 17, 2021
d03566f
fix lint errors
fschade Jun 17, 2021
248f921
rebase and restructure tests
fschade Jun 17, 2021
a9291ba
return from accessibleBreadcrumb_focusAndAnnounceBreadcrumb early if …
fschade Jun 17, 2021
a6bd834
Add another bugfix changelog item
pascalwengerter Jun 17, 2021
8c2de18
Add more changelog items & try to satisfy CI
pascalwengerter Jun 17, 2021
cac97c8
Use share id as resource id
kulmann Jun 17, 2021
758eb69
add debounce wait time to personal view image loading
fschade Jun 17, 2021
e468d99
bring back crore-js to babel to support older environments
fschade Jun 17, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .eslintrc.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
module.exports = {
env: {
browser: true,
es6: true,
amd: true
},
extends: [
'standard',
'prettier/standard',
'plugin:prettier/recommended',
'plugin:jest/recommended'
],
parserOptions: {
parser: 'babel-eslint',
sourceType: 'module'
},
rules: {
/**
* TODO: fix project import issues and then enable it
* 'sort-imports': 'warn',
*/
'require-await': 'warn',
'no-new': 'off',
'node/no-callback-literal': 'off'
},
globals: {
require: false,
requirejs: false
},
plugins: ['jest'],
overrides: [
{
files: ['**/*.vue'],
extends: ['plugin:vue/recommended', 'prettier/vue']
},
{
files: ['**/*.ts'],
parser: '@typescript-eslint/parser',
extends: ['plugin:@typescript-eslint/recommended'],
rules: {
'@typescript-eslint/no-extra-semi': 'off',
'@typescript-eslint/no-explicit-any': 'off'
}
}
]
}
39 changes: 0 additions & 39 deletions .eslintrc.json

This file was deleted.

4 changes: 2 additions & 2 deletions .prettierrc.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"printWidth": 100,
"semi": false,
"singleQuote": true
"singleQuote": true,
"semi": false
}
2 changes: 2 additions & 0 deletions __mocks__/axios.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
import mockAxios from 'jest-mock-axios'
export default mockAxios
1 change: 0 additions & 1 deletion __mocks__/style.js
Original file line number Diff line number Diff line change
@@ -1 +0,0 @@
export default {}
6 changes: 6 additions & 0 deletions changelog/unreleased/bugfix-avoid-duplicate-resource-loading
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Bugfix: Avoid duplicate loading of resources

On the personal route, we had a redirect case where
resources would be loaded twice, which now is fixed.

https://github.com/owncloud/web/pull/5194
6 changes: 6 additions & 0 deletions changelog/unreleased/bugfix-reduced-thumbnail-size
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Bugfix: Reduced Thumbnail Size

We have greatly reduced the size of the images we request from the
backend to display as thumbnail previews in order to minimize loading times.

https://github.com/owncloud/web/pull/5194
7 changes: 7 additions & 0 deletions changelog/unreleased/enhancement-async-image-loading
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Enhancement: Asynchronous loading of images

Thumbnail and avatar images now get loaded in the background
and don't block the main rendering of the user interface.

https://github.com/owncloud/web/issues/4973
https://github.com/owncloud/web/pull/5194
7 changes: 7 additions & 0 deletions changelog/unreleased/enhancement-introduce-img-cache
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Enhancement: Introduce image cache

We have added a (configurable) cache for thumbnails and avatar
images to avoid loading the same files over and over again.

https://github.com/owncloud/web/issues/3098
https://github.com/owncloud/web/pull/5194
9 changes: 9 additions & 0 deletions changelog/unreleased/enhancement-request-cancellation
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
Enhancement: Request cancellation

Requests (e.g. loading of images) can now be pragmatically
cancelled from the client side. Before, obsolete requests
would still create load on the server and return results
that then would be discarded by the web frontend.

https://github.com/owncloud/web/issues/5163
https://github.com/owncloud/web/pull/5194
6 changes: 6 additions & 0 deletions changelog/unreleased/enhancement-typescript-support
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
Enhancement: TypeScript Support

We have added support for TypeScript and started to refactor parts of the codebase.
This will help us provide clearer interfaces and catch bugs earlier.

https://github.com/owncloud/web/pull/5194
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
Change: Use slots in the navigation sidebar
Enhancement: Use slots in the navigation sidebar

In the new sidebar content is defined solely via slots. We've moved all the content into those slots so that the sidebar still gets displayed correctly.

https://github.com/owncloud/web/pull/5105
https://github.com/owncloud/web/pull/5105
7 changes: 7 additions & 0 deletions changelog/unreleased/enhancement-visibility-observer
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
Enhancement: Visibility observer

By adding a visibility observer, we now only load image previews for those files that
are close to the user's viewport. It is also equiped with a short waiting period so
scrolling doesn't lead to an overload of requests.

https://github.com/owncloud/web/pull/5194
39 changes: 24 additions & 15 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@
"build": "rollup -c",
"build:w": "rollup -c -w",
"depcheck": "depcheck",
"lint": "eslint packages/*/src packages/*/tests tests --ext vue --ext js --color",
"lint": "eslint '{packages,tests}/**/*.{js,ts,vue}' --color",
"serve": "SERVER=true yarn build:w",
"test:acceptance:drone": "cucumber-js --retry 1 --require-module @babel/register --require-module @babel/polyfill --require tests/acceptance/setup.js --require tests/acceptance/stepDefinitions --format node_modules/cucumber-pretty",
"test:acceptance:oc10": "cucumber-js --require-module @babel/register --require-module @babel/polyfill --require tests/acceptance/setup.js --require tests/acceptance/stepDefinitions --format node_modules/cucumber-pretty -t \"${TEST_TAGS:-not @skip and not @skipOnOC10}\"",
Expand All @@ -34,15 +34,17 @@
"devDependencies": {
"@babel/core": "^7.12.10",
"@babel/polyfill": "^7.12.1",
"@babel/preset-env": "^7.12.11",
"@babel/preset-env": "^7.14.4",
"@babel/register": "^7.13.16",
"@erquhart/rollup-plugin-node-builtins": "^2.1.5",
"@rollup/plugin-commonjs": "^17.0.0",
"@rollup/plugin-html": "^0.2.0",
"@rollup/plugin-json": "^4.1.0",
"@testing-library/jest-dom": "^5.13.0",
"@testing-library/vue": "^5.6.2",
"@rollup/plugin-typescript": "^8.2.1",
"@types/jest": "^26.0.23",
"@types/jest-axe": "^3.5.1",
"@typescript-eslint/eslint-plugin": "^4.26.0",
"@typescript-eslint/parser": "^4.26.0",
"@vue/test-utils": "^1.1.2",
"archiver": "^3.0.0",
"autoprefixer": "^10.2.4",
Expand All @@ -53,18 +55,17 @@
"chromedriver": "^89.0.0",
"core-js": "^3.10.0",
"cucumber": ">=6.0.5",
"cucumber-pretty": ">=6.0.0",
"cucumber-pretty": "^6.0.0",
"depcheck": "^1.3.1",
"ejs": "^3.1.5",
"eslint": "6.8.0",
"eslint": "^7.27.0",
"eslint-config-prettier": "^6.10.0",
"eslint-config-standard": "^14.1.0",
"eslint-plugin-import": "^2.18.2",
"eslint-config-standard": "^16.0.3",
"eslint-plugin-import": "^2.23.4",
"eslint-plugin-jest": "^24.1.3",
"eslint-plugin-node": "^11.0.0",
"eslint-plugin-prettier": "^3.1.2",
"eslint-plugin-promise": "^4.2.1",
"eslint-plugin-standard": "^4.0.0",
"eslint-plugin-node": "^11.1.0",
"eslint-plugin-prettier": "^3.4.0",
"eslint-plugin-promise": "^5.1.0",
"eslint-plugin-vue": "^6.2.2",
"eslint-plugin-vuejs-accessibility": "^0.6.1",
"focus-trap": "^6.4.0",
Expand All @@ -73,10 +74,10 @@
"jest": "^26.6.3",
"jest-axe": "^4.1.0",
"jest-fetch-mock": "^3.0.3",
"jest-mock-axios": "^4.4.0",
"jest-serializer-vue": "^2.0.2",
"jest-svg-transformer": "^1.0.0",
"join-path": "^1.1.1",
"jsdom": "^16.6.0",
"ldapjs": "^2.2.3",
"lodash": "^4.17.21",
"nightwatch": "1.5.1",
Expand All @@ -85,8 +86,9 @@
"node-fetch": "^2.6.1",
"p-limit": "^3.1.0",
"postcss": "^8.2.15",
"regenerator-runtime": "^0.13.7",
"requirejs": "^2.3.6",
"rollup": "^2.38.5",
"rollup": "^2.51.1",
"rollup-plugin-babel": "^4.4.0",
"rollup-plugin-copy-watch": "^0.0.1",
"rollup-plugin-delete": "^2.0.0",
Expand All @@ -99,8 +101,15 @@
"rollup-plugin-progress": "^1.1.2",
"rollup-plugin-serve": "^1.1.0",
"rollup-plugin-terser": "^7.0.2",
"rollup-plugin-ts": "^1.4.0",
"rollup-plugin-visualizer": "^4.2.0",
"rollup-plugin-vue": "^5.1.4",
"ts-jest": "^26.5.6",
"tslib": "^2.2.0",
"typescript": "^4.3.2",
"@testing-library/jest-dom": "^5.13.0",
"@testing-library/vue": "^5.6.2",
"jsdom": "^16.6.0",
"sync-fetch": "^0.3.0",
"url-search-params-polyfill": "^8.0.0",
"vue-jest": "^3.0.7",
Expand All @@ -109,6 +118,6 @@
"xml-js": "^1.6.11"
},
"engines": {
"node": ">=10 <=15"
"node": ">=10 <=16"
}
}
6 changes: 3 additions & 3 deletions packages/web-app-draw-io/src/App.vue
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ export default {
this.checkPermissions()
window.addEventListener('message', event => {
if (event.data.length > 0) {
var payload = JSON.parse(event.data)
const payload = JSON.parse(event.data)
switch (payload.event) {
case 'init':
this.fileExtension === 'vsdx' ? this.importVisio() : this.load()
Expand Down Expand Up @@ -137,8 +137,8 @@ export default {
return resp.arrayBuffer()
})
.then(arrayBuffer => {
var blob = new Blob([arrayBuffer], { type: 'application/vnd.visio' })
var reader = new FileReader()
const blob = new Blob([arrayBuffer], { type: 'application/vnd.visio' })
const reader = new FileReader()
reader.onloadend = () => {
this.$refs.drawIoEditor.contentWindow.postMessage(
JSON.stringify({
Expand Down
14 changes: 1 addition & 13 deletions packages/web-app-files/src/components/AppBar.vue
Original file line number Diff line number Diff line change
Expand Up @@ -270,12 +270,7 @@ export default {
}
},
methods: {
...mapActions('Files', [
'updateFileProgress',
'removeFilesFromTrashbin',
'loadPreviews',
'loadIndicators'
]),
...mapActions('Files', ['updateFileProgress', 'removeFilesFromTrashbin', 'loadIndicators']),
...mapActions(['openFile', 'showMessage', 'createModal', 'setModalInputErrorMessage']),
...mapMutations('Files', ['UPSERT_RESOURCE']),
...mapMutations(['SET_QUOTA']),
Expand Down Expand Up @@ -510,13 +505,6 @@ export default {
})
}

this.loadPreviews({
resources: [resource],
isPublic: this.publicPage(),
mediaSource: this.mediaSource,
encodePath: this.encodePath
})

const user = await this.$client.users.getUser(this.user.id)

this.SET_QUOTA(user.quota)
Expand Down
4 changes: 4 additions & 0 deletions packages/web-app-files/src/constants.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
export abstract class ImageDimension {
static readonly ThumbNail: [number, number] = [36, 36]
static readonly Avatar: number = 64
}
20 changes: 20 additions & 0 deletions packages/web-app-files/src/helpers/resource/common.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import queryString from 'query-string'

interface BuildQueryStringOptions {
preview?: number
scalingup?: number
a?: number
etag?: string
dimensions?: [number, number]
}

export const buildQueryString = (options: BuildQueryStringOptions): string => {
return queryString.stringify({
scalingup: options.scalingup || 0,
preview: Object.hasOwnProperty.call(options, 'preview') ? options.preview : 1,
a: Object.hasOwnProperty.call(options, 'a') ? options.a : 1,
...(options.etag && { c: options.etag.replaceAll('"', '') }),
...(options.dimensions && options.dimensions[0] && { x: options.dimensions[0] }),
...(options.dimensions && options.dimensions[1] && { y: options.dimensions[1] })
})
}
2 changes: 2 additions & 0 deletions packages/web-app-files/src/helpers/resource/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
export * from './privatePreviewBlob'
export * from './publicPreviewUrl'
Loading