-
Notifications
You must be signed in to change notification settings - Fork 8.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ML] Transforms: Data grid fixes. #59538
Merged
walterra
merged 14 commits into
elastic:master
from
walterra:ml-transforms-data-grid-fix-date-formatting
Mar 10, 2020
Merged
Changes from 12 commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
c862d42
[ML] Fix date formatting.
walterra b68f9f3
[ML] Migrate transform list preview table to data grid.
walterra 8b6f1d0
[ML] Move PivotPreview to components.
walterra b60bd12
[ML] Fix mock path.
walterra ae7e5e7
Merge branch 'master' into ml-transforms-data-grid-fix-date-formatting
elasticmachine c6b9430
[ML] Fix source table sorting.
walterra 16527aa
Merge branch 'ml-transforms-data-grid-fix-date-formatting' of github.…
walterra 9dbbc75
Merge branch 'master' into ml-transforms-data-grid-fix-date-formatting
walterra 7b8af7f
[ML] Fix tests.
walterra 760ae8a
[ML] Fix getNestedProperty for falsey values.
walterra c4da338
Merge branch 'master' into ml-transforms-data-grid-fix-date-formatting
walterra 66161ff
[ML] Improve sorting related error messages.
walterra 19ccc57
[ML] Don't hide data grid when an error occurs fetching the data.
walterra d17b83b
[ML] Tweak reduce.
walterra File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
70 changes: 70 additions & 0 deletions
70
x-pack/plugins/transform/common/utils/object_utils.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,70 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the Elastic License; | ||
* you may not use this file except in compliance with the Elastic License. | ||
*/ | ||
|
||
import { getNestedProperty } from './object_utils'; | ||
|
||
describe('object_utils', () => { | ||
test('getNestedProperty()', () => { | ||
const testObj = { | ||
the: { | ||
nested: { | ||
value: 'the-nested-value', | ||
}, | ||
}, | ||
}; | ||
|
||
const falseyObj = { | ||
the: { | ||
nested: { | ||
value: false, | ||
}, | ||
other_nested: { | ||
value: 0, | ||
}, | ||
}, | ||
}; | ||
|
||
const test1 = getNestedProperty(testObj, 'the'); | ||
expect(typeof test1).toBe('object'); | ||
expect(Object.keys(test1)).toStrictEqual(['nested']); | ||
|
||
const test2 = getNestedProperty(testObj, 'the$'); | ||
expect(typeof test2).toBe('undefined'); | ||
|
||
const test3 = getNestedProperty(testObj, 'the$', 'the-default-value'); | ||
expect(typeof test3).toBe('string'); | ||
expect(test3).toBe('the-default-value'); | ||
|
||
const test4 = getNestedProperty(testObj, 'the.neSted'); | ||
expect(typeof test4).toBe('undefined'); | ||
|
||
const test5 = getNestedProperty(testObj, 'the.nested'); | ||
expect(typeof test5).toBe('object'); | ||
expect(Object.keys(test5)).toStrictEqual(['value']); | ||
|
||
const test6 = getNestedProperty(testObj, 'the.nested.vaLue'); | ||
expect(typeof test6).toBe('undefined'); | ||
|
||
const test7 = getNestedProperty(testObj, 'the.nested.value'); | ||
expect(typeof test7).toBe('string'); | ||
expect(test7).toBe('the-nested-value'); | ||
|
||
const test8 = getNestedProperty(testObj, 'the.nested.value.doesntExist'); | ||
expect(typeof test8).toBe('undefined'); | ||
|
||
const test9 = getNestedProperty(testObj, 'the.nested.value.doesntExist', 'the-default-value'); | ||
expect(typeof test9).toBe('string'); | ||
expect(test9).toBe('the-default-value'); | ||
|
||
const test10 = getNestedProperty(falseyObj, 'the.nested.value'); | ||
expect(typeof test10).toBe('boolean'); | ||
expect(test10).toBe(false); | ||
|
||
const test11 = getNestedProperty(falseyObj, 'the.other_nested.value'); | ||
expect(typeof test11).toBe('number'); | ||
expect(test11).toBe(0); | ||
}); | ||
}); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
117 changes: 117 additions & 0 deletions
117
x-pack/plugins/transform/public/app/components/pivot_preview/common.test.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,117 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the Elastic License; | ||
* you may not use this file except in compliance with the Elastic License. | ||
*/ | ||
|
||
import { EuiDataGridSorting } from '@elastic/eui'; | ||
|
||
import { | ||
getPreviewRequestBody, | ||
PivotAggsConfig, | ||
PivotGroupByConfig, | ||
PIVOT_SUPPORTED_AGGS, | ||
PIVOT_SUPPORTED_GROUP_BY_AGGS, | ||
SimpleQuery, | ||
} from '../../common'; | ||
|
||
import { multiColumnSortFactory, getPivotPreviewDevConsoleStatement } from './common'; | ||
|
||
describe('Transform: Define Pivot Common', () => { | ||
test('multiColumnSortFactory()', () => { | ||
const data = [ | ||
{ s: 'a', n: 1 }, | ||
{ s: 'a', n: 2 }, | ||
{ s: 'b', n: 3 }, | ||
{ s: 'b', n: 4 }, | ||
]; | ||
|
||
const sortingColumns1: EuiDataGridSorting['columns'] = [{ id: 's', direction: 'desc' }]; | ||
const multiColumnSort1 = multiColumnSortFactory(sortingColumns1); | ||
data.sort(multiColumnSort1); | ||
|
||
expect(data).toStrictEqual([ | ||
{ s: 'b', n: 3 }, | ||
{ s: 'b', n: 4 }, | ||
{ s: 'a', n: 1 }, | ||
{ s: 'a', n: 2 }, | ||
]); | ||
|
||
const sortingColumns2: EuiDataGridSorting['columns'] = [ | ||
{ id: 's', direction: 'asc' }, | ||
{ id: 'n', direction: 'desc' }, | ||
]; | ||
const multiColumnSort2 = multiColumnSortFactory(sortingColumns2); | ||
data.sort(multiColumnSort2); | ||
|
||
expect(data).toStrictEqual([ | ||
{ s: 'a', n: 2 }, | ||
{ s: 'a', n: 1 }, | ||
{ s: 'b', n: 4 }, | ||
{ s: 'b', n: 3 }, | ||
]); | ||
|
||
const sortingColumns3: EuiDataGridSorting['columns'] = [ | ||
{ id: 'n', direction: 'desc' }, | ||
{ id: 's', direction: 'desc' }, | ||
]; | ||
const multiColumnSort3 = multiColumnSortFactory(sortingColumns3); | ||
data.sort(multiColumnSort3); | ||
|
||
expect(data).toStrictEqual([ | ||
{ s: 'b', n: 4 }, | ||
{ s: 'b', n: 3 }, | ||
{ s: 'a', n: 2 }, | ||
{ s: 'a', n: 1 }, | ||
]); | ||
}); | ||
|
||
test('getPivotPreviewDevConsoleStatement()', () => { | ||
const query: SimpleQuery = { | ||
query_string: { | ||
query: '*', | ||
default_operator: 'AND', | ||
}, | ||
}; | ||
const groupBy: PivotGroupByConfig = { | ||
agg: PIVOT_SUPPORTED_GROUP_BY_AGGS.TERMS, | ||
field: 'the-group-by-field', | ||
aggName: 'the-group-by-agg-name', | ||
dropDownName: 'the-group-by-drop-down-name', | ||
}; | ||
const agg: PivotAggsConfig = { | ||
agg: PIVOT_SUPPORTED_AGGS.AVG, | ||
field: 'the-agg-field', | ||
aggName: 'the-agg-agg-name', | ||
dropDownName: 'the-agg-drop-down-name', | ||
}; | ||
const request = getPreviewRequestBody('the-index-pattern-title', query, [groupBy], [agg]); | ||
const pivotPreviewDevConsoleStatement = getPivotPreviewDevConsoleStatement(request); | ||
|
||
expect(pivotPreviewDevConsoleStatement).toBe(`POST _transform/_preview | ||
{ | ||
"source": { | ||
"index": [ | ||
"the-index-pattern-title" | ||
] | ||
}, | ||
"pivot": { | ||
"group_by": { | ||
"the-group-by-agg-name": { | ||
"terms": { | ||
"field": "the-group-by-field" | ||
} | ||
} | ||
}, | ||
"aggregations": { | ||
"the-agg-agg-name": { | ||
"avg": { | ||
"field": "the-agg-field" | ||
} | ||
} | ||
} | ||
} | ||
} | ||
`); | ||
}); | ||
}); |
60 changes: 60 additions & 0 deletions
60
x-pack/plugins/transform/public/app/components/pivot_preview/common.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,60 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the Elastic License; | ||
* you may not use this file except in compliance with the Elastic License. | ||
*/ | ||
|
||
import { EuiDataGridSorting } from '@elastic/eui'; | ||
|
||
import { getNestedProperty } from '../../../../common/utils/object_utils'; | ||
|
||
import { PreviewRequestBody } from '../../common'; | ||
|
||
/** | ||
* Helper to sort an array of objects based on an EuiDataGrid sorting configuration. | ||
* `sortFn()` is recursive to support sorting on multiple columns. | ||
* | ||
* @param sortingColumns - The EUI data grid sorting configuration | ||
* @returns The sorting function which can be used with an array's sort() function. | ||
*/ | ||
export const multiColumnSortFactory = (sortingColumns: EuiDataGridSorting['columns']) => { | ||
const isString = (arg: any): arg is string => { | ||
return typeof arg === 'string'; | ||
}; | ||
|
||
const sortFn = (a: any, b: any, sortingColumnIndex = 0): number => { | ||
const sort = sortingColumns[sortingColumnIndex]; | ||
const aValue = getNestedProperty(a, sort.id, null); | ||
const bValue = getNestedProperty(b, sort.id, null); | ||
|
||
if (typeof aValue === 'number' && typeof bValue === 'number') { | ||
if (aValue < bValue) { | ||
return sort.direction === 'asc' ? -1 : 1; | ||
} | ||
if (aValue > bValue) { | ||
return sort.direction === 'asc' ? 1 : -1; | ||
} | ||
} | ||
|
||
if (isString(aValue) && isString(bValue)) { | ||
if (aValue.localeCompare(bValue) === -1) { | ||
return sort.direction === 'asc' ? -1 : 1; | ||
} | ||
if (aValue.localeCompare(bValue) === 1) { | ||
return sort.direction === 'asc' ? 1 : -1; | ||
} | ||
} | ||
|
||
if (sortingColumnIndex + 1 < sortingColumns.length) { | ||
return sortFn(a, b, sortingColumnIndex + 1); | ||
} | ||
|
||
return 0; | ||
}; | ||
|
||
return sortFn; | ||
}; | ||
|
||
export const getPivotPreviewDevConsoleStatement = (request: PreviewRequestBody) => { | ||
return `POST _transform/_preview\n${JSON.stringify(request, null, 2)}\n`; | ||
}; |
7 changes: 7 additions & 0 deletions
7
x-pack/plugins/transform/public/app/components/pivot_preview/index.ts
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
/* | ||
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one | ||
* or more contributor license agreements. Licensed under the Elastic License; | ||
* you may not use this file except in compliance with the Elastic License. | ||
*/ | ||
|
||
export { PivotPreview } from './pivot_preview'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I believe it'd be better to have test per expect here because assertations are not related
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll leave it as is since this is a 1:1 copy of the same file we have in the ML plugin. I'd like to consolidate related files in a shared space for both plugins at some point then we can also refactor the these tests.