Skip to content

Commit

Permalink
update from upstream nextstrain/auspice (#8)
Browse files Browse the repository at this point in the history
* make frequencies tend to 0 in absense of data

* increase padding value for frequencies

* lint appeasment

* set frequencies explicity to 0 if total is too low

* reduce frequency normalizaton threshold to single constant

* remove unassigned variable

* Allow continuous colorings to define anchor points

The schema currently allows datasets to provide a scale for non-
continuous scales where specific trait values are given colour hexes
(missing values are given greys by auspice).

Here we extend this to continuous scales by interpreting the same
data structure as anchor points which we interpolate between using
the same method as we currently use for generating default continuous
color scales (d3's `interpolateRgb`)

* Allow legend entries to be user-defined

This allows continuous colour scales to define custom legend
entries, via a `legend` key in the JSON. This allows control
over the values in the scale which we use as legend elements,
the displayed text, and the range of values which each entry covers.

Bounds are enforced to be non-overlapping. If overlapping bounds
are detected, we revert to Auspice dynamically generating these.
(This is a requirement for future work which will map continuous
tip values to a legend entry, which will allow pie-chart display
using the legend swatches.)

* Legend bound matching is (a, b] for continuous scales

This restores the algorithm used to associate a hovered legend
item to tips for continuous variables. Commit
0f37b1a (Mar 2018) incorrectly changed
this to `tip \in [a, b]` rather than the intended (and documented)
`tip \in (a, b]`.

This takes on more importance given that the previous commit allows
user-defined bounds.

Note that the frequencies panel already used `(a, b]` matching, so now
the legend matching mirrors this.

* Extend user-provided legend info beyond continuous scales

* Use filterOptions to modify search alg

* GitHub Action to create nextstrain.org PR

This action will run on each auspice PR and create a corresponding
PR on nextstrain.org which includes a commit using the version
of auspice from this (auspice) PR. This functionality is extremely
useful for auspice development as it will allow us to use a Heroku
review app to test auspice in the context of nextstrain.org

There are a number of future improvements to implement:

* New auspice releases (tagged commits on `release` branch)
would ideally create a PR on nextstrain.org which could be merged
to update the version of auspice there.

* Other consumers of auspice (e.g. auspice.us) could be added to this
GitHub Action.

* Allow non-continuous scatterplot variables

This implements a requested improvement to the original
scatterplot implementation. The implementation hinges on two changes:
(1) The collection of values for a given variable (e.g. x-var) need
to be computed and passed to PhyloTree to act as the scale's domain.
We reuse the colorScale machinery here, which could be optimised
(see todo messages in code), but this has the advantage that the
domain ordering matches the legend (unless user supplied).
(2) PhyloTree needed to be modified to use non-linear scales, in this
case `pointScale`.

This commit should be fully functional, however there are some
future improvements to be made:

(i) Grid text is obscured and unreadable when there are many entries
in the domain.
(ii) Genotypes and Boolean scales are not yet available.
(iii) Jitter should be added to nodes to avoid obfuscation.

* Layout changes occur via redux thunk

This commit is in preparation for allowing genotype to be a scatterplot
variable. This will complicate the allowable scatterplot variables
and force these to update upon colorBy changes. This is much cleaner
if layout is changed in a thunk.

* Allow genotype to be scatterplot variable

Genotype is treated differently to other colorings in two important
ways: (1) it can change value, for instance when changing the
colorby to another genotype position and (2) it is stored in a
different place to other colorings. These require scatterplot logic
to be more complex as actions are no longer separate - we now require
a NEW_COLOURS action to potentially update the layout which was
formerly within the remit of the CHANGE_LAYOUT actions. This is
achieved through a middleware layer.

This implementation makes it clear that jitter and better domain
spacing are crucial for scatterplots.

* Improve padding for categorical scatterplot variables

This prevents nodes falling on the axis itself or at the very end of
the grid, which was especially noticeable for traits with small domains.

* Add jitter to categorical scatterplots

* Apply clipping to first column of legend

We have had issues in the past with legend values from column 1 overflowing into column 2. For instance, issue nextstrain#899 was fixed by PR nextstrain#914 which implemented a maximum character limit for legend names. This solution can produce misleading views, such as those described in nextstrain#1306.

This solution implements a clipping mask for column 1, avoiding the complication of limiting the string size. Column 2 already has similar behaviour because the SVG element of the legend itself performs the clipping.

* changelog

* version bump to 2.26.0 for release

* Always show regression toggle for clock layout

Fixes a bug where the ability to toggle regression lines was hidden for clock views. (The ability to hide this toggle is only intended for scatter layouts, where we should not expose the toggle unless both axes are showing continuous variables.)

* Adjust grayscale color ramp

The existing grayscale color ramp (used for values absent in an explicitly specified color scale) had values that were too dark and threw off the overall color balance. This commit narrows the grayscale color ramp to be more in line with pastel color ramp.

* Inject a bit of color into the "grayscale" color ramp

This adds a bit of blue into the grayscale color ramp. Still reads as mostly gray, but no colors seem to exist more in the same universe as canonical auspice color ramp.

* changelog

* version bump to 2.27.0 for release

* Styling adjustments to footer text

* Remove metadata download from GISAID datasets

This commit uses dataProvenance in metadata to identify datasets using "GISAID" data. For these datasets, the full metadata download is swapped to an "acknowledgments" download that only includes the following fields:
 - strain
 - gisaid_epi_isl
 - genbank_accession
 - originating_lab
 - submitting_lab
 - author

* Cleanup metadata headers

This commit cleans up naming of metadata headers in downloaded metadata TSV. It does the following:
1. Keeps headers as input into "augur export" rather than renaming by title. Thus it has "originating_lab" rather than "Originating lab", "pango_lineage" rather than "PANGO lineage", etc... This should make it easier for people to process downloaded metadata from Auspice alongside metadata provisioned by Nextstrain (via GISAID or via S3).
2. Makes "date" the second column as this is often what's most important. I couldn't figure out a way to intelligently order remaining fields. My first thought was to use metadata.colorings, but this isn't sorted.
3. Fixes "accession". It had been exporting as "[object Object]".

* Update changelog

* version bump to 2.28.0 for release

Co-authored-by: Richard Neher <richard.neher@unibas.ch>
Co-authored-by: James Hadfield <hadfield.james@gmail.com>
Co-authored-by: james hadfield <jameshadfield@users.noreply.github.com>
Co-authored-by: Muhammad Aditya Hilmy <mhilmy@hey.com>
Co-authored-by: Trevor Bedford <trevor@bedford.io>
  • Loading branch information
6 people authored Jul 6, 2021
1 parent d64a39a commit 6b3eb64
Show file tree
Hide file tree
Showing 10 changed files with 308 additions and 149 deletions.
45 changes: 37 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,38 @@
title: Changelog
---

## version 2.25.1 - 2021/04/07

## version 2.28.0 - 2021/07/05


* Remove ability to download metadata TSV from GISAID datasets. Replace with acknowledgments TSV. See [PR 1366](https://github.com/nextstrain/auspice/pull/1366).
* Cleanup header fields when downloading metadata TSV, including switch to use name (eg `pango_lineage`) instead of title (eg `PANGO lineage`). See [PR 1367](https://github.com/nextstrain/auspice/pull/1367).
* Update styling of footer text. See [PR 1364](https://github.com/nextstrain/auspice/pull/1364).

## version 2.27.0 - 2021/06/05

* Adjust greyscale colour ramp.
See [PR 1353](https://github.com/nextstrain/auspice/pull/1353) for more.
* (Bugfix) Fixed the situation where the regression toggle would not appear for clock layouts.
See [PR 1352](https://github.com/nextstrain/auspice/pull/1352) for more.

## version 2.26.0 - 2021/05/25
* Scatterplot improvements:
* Non-continuous variables can now be used, which allows all colourings (including Genotype, if that's the current colouring) to be scatterplot variables.
* Jittering is applied when the spacing between axis variables is more than 50 pixels.
* See [PR 1346](https://github.com/nextstrain/auspice/pull/1346) for more.
* Normalized frequency values now tend to zero in the absence of data.
See [PR 1325](https://github.com/nextstrain/auspice/pull/1325) for more.
* Colour scale improvements:
* Continuous colourings can provide a scale, which we interpolate between to get the colour scheme
* Custom legend data can be provided, including display text and, for continuous variables, bounds to map legend entries to values in the data.
* Displayed legend entires may be restricted by specifying them in the dataset JSON.
* See [PR 1340](https://github.com/nextstrain/auspice/pull/1340) for more.
* Filtering via the sidebar UI now returns options which match each of the space-separated queries, rather than requiring an exact match of the query.
See [PR 1344](https://github.com/nextstrain/auspice/pull/1344) for more.
* Legend text now takes the maximum available space.
See [PR 1328](https://github.com/nextstrain/auspice/pull/1328) for more.

## version 2.25.1 - 2021/04/07
* Bugfix for cases where certain interactions with scatterplot variables would cause auspice to crash.
See [PR 1332](https://github.com/nextstrain/auspice/pull/1332) for more.

Expand Down Expand Up @@ -145,7 +174,7 @@ This version reverts the change to URL parsing introduced in 2.18.2 which broke
* Improve parsing of auspice URLs with colon characters in the pathname. See [PR 1210](https://github.com/nextstrain/auspice/pull/1210).

## version 2.18.1 - 2020/08/07
* Add between-paragraph padding for text rendering in (non-mobile) narratives.
* Add between-paragraph padding for text rendering in (non-mobile) narratives.

## version 2.18.0 - 2020/08/03
* Parse narratives client side.
Expand Down Expand Up @@ -208,7 +237,7 @@ See [PR 1166](https://github.com/nextstrain/auspice/pull/1166).
[See PR 1126](https://github.com/nextstrain/auspice/pull/1126) for more.
* Add a toggle for whether or not to show transmission lines on the map.
[See PR 1147](https://github.com/nextstrain/auspice/pull/1147) and [PR 1103](https://github.com/nextstrain/auspice/pull/1147) for more.
* Dynamically adjust deme circle size on the map when filtering.
* Dynamically adjust deme circle size on the map when filtering.
[See PR 1135](https://github.com/nextstrain/auspice/pull/1135) for more.
* Allow the genomic diversity data (the data behind the entropy panel) to be downloaded as a TSV.
[See PR 1144](https://github.com/nextstrain/auspice/pull/1144) for more.
Expand All @@ -219,7 +248,7 @@ See [PR 1166](https://github.com/nextstrain/auspice/pull/1166).
#### Other
* Temporarily disable integration tests from the GitHub CI. [See PR 1148](https://github.com/nextstrain/auspice/pull/1148) for more.
* Add a CC-BY license for the downloaded SVG (screenshots) . [See PR 1140](https://github.com/nextstrain/auspice/pull/1140) for more.
* Improvement in code which decides which footers to show.
* Improvement in code which decides which footers to show.
[See PR 1118](https://github.com/nextstrain/auspice/pull/1118) for more.
* Documentation improvements -- see [PR 1127](https://github.com/nextstrain/auspice/pull/1127) for more.
* Fix an error in map positioning in some narrative slides. [See PR 958](https://github.com/nextstrain/auspice/pull/958) for more.
Expand Down Expand Up @@ -273,7 +302,7 @@ This will allow narratives to render slides with the CI displayed.
[See PR 1046](https://github.com/nextstrain/auspice/pull/1046)
* Add the ability to export per-strain metadata of only those strains currently being displayed.
[See PR 1067](https://github.com/nextstrain/auspice/pull/1067)
* Move to using `react-icons` which allows the removal of the font-awesome CSS.
* Move to using `react-icons` which allows the removal of the font-awesome CSS.
This improves ease-of-use and reduces the bundle size.
[See PR 1065](https://github.com/nextstrain/auspice/pull/1065), [PR 1041](https://github.com/nextstrain/auspice/pull/1041)
& [PR 1073](https://github.com/nextstrain/auspice/pull/1073)
Expand Down Expand Up @@ -353,8 +382,8 @@ This improves ease-of-use and reduces the bundle size.
* Update Spanish locale data (still in a partially complete state).
See [commit f9c8ad2](https://github.com/nextstrain/auspice/commit/f9c8ad209a1e5d304fc6f15ec708f3d0be3dec43)
* Reorganisation and general improvements to documentation around contributing to auspice development.
[See PR 978](https://github.com/nextstrain/auspice/pull/978),
[commit 707f563](https://github.com/nextstrain/auspice/commit/707f563aab0a62e0504e393af0cd23da3e4504e0) and
[See PR 978](https://github.com/nextstrain/auspice/pull/978),
[commit 707f563](https://github.com/nextstrain/auspice/commit/707f563aab0a62e0504e393af0cd23da3e4504e0) and
[commit 9f002c9](https://github.com/nextstrain/auspice/commit/9f002c96a676e4603b7b9c06ef7df8a26be6d04c)
* Fix a bug where the narrative table styling introduced in 2.9.0 were applied outside the narratives.
* Fix all linting errors and warnings (potentially the first time this has happened!)
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "auspice",
"version": "2.25.1",
"version": "2.28.0",
"description": "Web app for visualizing pathogen evolution",
"author": "James Hadfield, Trevor Bedford and Richard Neher",
"license": "AGPL-3.0-only",
Expand Down
2 changes: 1 addition & 1 deletion src/actions/recomputeReduxState.js
Original file line number Diff line number Diff line change
Expand Up @@ -568,7 +568,7 @@ const checkAndCorrectErrorsInState = (state, metadata, query, tree, viewingNarra
// todo: these should be JSON definable (via display_defaults)
if (state.layout==="scatter" || state.layout==="clock") {
state.scatterVariables = validateScatterVariables(
state.scatterVariables, metadata.colorings, state.distanceMeasure, state.colorBy, state.layout==="clock"
state, metadata, tree, state.layout==="clock"
);
if (query.scatterX && query.scatterX!==state.scatterVariables.x) delete query.scatterX;
if (query.scatterY && query.scatterY!==state.scatterVariables.y) delete query.scatterY;
Expand Down
105 changes: 41 additions & 64 deletions src/components/controls/choose-layout.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
/* eslint-disable react/jsx-no-bind */
/* ^^^ We can get away with this because <ChooseLayout> doesn't rerender frequently, but fixes are welcome */

import React from "react";
import PropTypes from 'prop-types';
import { connect } from "react-redux";
Expand All @@ -9,9 +6,9 @@ import { withTranslation } from 'react-i18next';
import Select from "react-select/lib/Select";
import * as icons from "../framework/svg-icons";
import { controlsWidth } from "../../util/globals";
import { collectAvailableScatterVariables, validateScatterVariables} from "../../util/scatterplotHelpers";
import { CHANGE_LAYOUT } from "../../actions/types";
import { collectAvailableScatterVariables} from "../../util/scatterplotHelpers";
import { SidebarSubtitle, SidebarButton } from "./styles";
import { changeLayout } from "../../actions/layout";
import Toggle from "./toggle";


Expand All @@ -29,9 +26,8 @@ export const RowContainer = styled.div`
return {
layout: state.controls.layout,
scatterVariables: state.controls.scatterVariables,
colorBy: state.controls.colorBy,
distanceMeasure: state.controls.distanceMeasure,
colorings: state.metadata.colorings,
colorBy: state.controls.colorBy,
showTreeToo: state.controls.showTreeToo,
branchLengthsToDisplay: state.controls.branchLengthsToDisplay
};
Expand All @@ -41,19 +37,8 @@ class ChooseLayout extends React.Component {
layout: PropTypes.string.isRequired,
dispatch: PropTypes.func.isRequired
}
constructor(props) {
super(props);
this.updateLayout = (layout, modifiedScatterVariables=undefined) => {
if (window.NEXTSTRAIN && window.NEXTSTRAIN.animationTickReference) return;
const scatterVariables = modifiedScatterVariables ?
{...this.props.scatterVariables, ...modifiedScatterVariables} :
this.props.scatterVariables;
this.props.dispatch({type: CHANGE_LAYOUT, layout, scatterVariables});
};
}

renderScatterplotAxesSelector() {
const options = collectAvailableScatterVariables(this.props.colorings);
const options = collectAvailableScatterVariables(this.props.colorings, this.props.colorBy);
const selectedX = options.filter((o) => o.value===this.props.scatterVariables.x)[0];
const selectedY = options.filter((o) => o.value===this.props.scatterVariables.y)[0];
const miscSelectProps = {options, clearable: false, searchable: false, multi: false, valueKey: "label"};
Expand All @@ -66,7 +51,7 @@ class ChooseLayout extends React.Component {
<Select
{...miscSelectProps}
value={selectedX}
onChange={(value) => this.updateLayout("scatter", {x: value.value, xLabel: value.label})}
onChange={(value) => this.props.dispatch(changeLayout({x: value.value, xLabel: value.label}))}
/>
</ScatterSelectContainer>
</ScatterVariableContainer>
Expand All @@ -77,40 +62,40 @@ class ChooseLayout extends React.Component {
<Select
{...miscSelectProps}
value={selectedY}
onChange={(value) => this.updateLayout("scatter", {y: value.value, yLabel: value.label})}
onChange={(value) => this.props.dispatch(changeLayout({y: value.value, yLabel: value.label}))}
/>
</ScatterSelectContainer>
</ScatterVariableContainer>
</>
);
}

renderScatterplotToggles() {
renderBranchToggle() {
return (
<>
<div style={{paddingTop: "2px"}}/>
<ScatterVariableContainer>
<Toggle
display
on={this.props.scatterVariables.showBranches}
callback={() => this.updateLayout(this.props.layout, {showBranches: !this.props.scatterVariables.showBranches})}
label={"Show branches"}
/>
</ScatterVariableContainer>
<div style={{paddingTop: "2px"}}/>
<ScatterVariableContainer>
<Toggle
display
on={this.props.scatterVariables.showRegression}
callback={() => this.updateLayout(this.props.layout, {showRegression: !this.props.scatterVariables.showRegression})}
label={"Show regression"}
/>
</ScatterVariableContainer>
<div style={{paddingTop: "2px"}}/>
</>
<ScatterVariableContainer padAbove>
<Toggle
display
on={this.props.scatterVariables.showBranches}
callback={() => this.props.dispatch(changeLayout({showBranches: !this.props.scatterVariables.showBranches}))}
label={"Show branches"}
/>
</ScatterVariableContainer>
);
}
renderRegressionToggle() {
if (this.props.layout === "scatter" && !(this.props.scatterVariables.xContinuous && this.props.scatterVariables.yContinuous)) {
return null; // scatterplot regressions only available if _both_ variables are continuous
}
return (
<ScatterVariableContainer padAbove>
<Toggle
display
on={this.props.scatterVariables.showRegression}
callback={() => this.props.dispatch(changeLayout({showRegression: !this.props.scatterVariables.showRegression}))}
label={"Show regression"}
/>
</ScatterVariableContainer>
);
}

render() {
const { t } = this.props;
if (this.props.showTreeToo) return null;
Expand All @@ -124,7 +109,7 @@ class ChooseLayout extends React.Component {
<RectangularTreeIcon width={25} selected={selected === "rect"}/>
<SidebarButton
selected={selected === "rect"}
onClick={() => this.updateLayout("rect")}
onClick={() => this.props.dispatch(changeLayout({layout: "rect"}))}
>
{t("sidebar:rectangular")}
</SidebarButton>
Expand All @@ -133,7 +118,7 @@ class ChooseLayout extends React.Component {
<RadialTreeIcon width={25} selected={selected === "radial"}/>
<SidebarButton
selected={selected === "radial"}
onClick={() => this.updateLayout("radial")}
onClick={() => this.props.dispatch(changeLayout({layout: "radial"}))}
>
{t("sidebar:radial")}
</SidebarButton>
Expand All @@ -142,7 +127,7 @@ class ChooseLayout extends React.Component {
<UnrootedTreeIcon width={25} selected={selected === "unrooted"}/>
<SidebarButton
selected={selected === "unrooted"}
onClick={() => this.updateLayout("unrooted")}
onClick={() => this.props.dispatch(changeLayout({layout: "unrooted"}))}
>
{t("sidebar:unrooted")}
</SidebarButton>
Expand All @@ -155,14 +140,12 @@ class ChooseLayout extends React.Component {
<ClockIcon width={25} selected={selected === "clock"}/>
<SidebarButton
selected={selected === "clock"}
onClick={() => this.updateLayout(
"clock",
validateScatterVariables(this.props.scatterVariables, this.props.colorings, this.props.distanceMeasure, this.props.colorBy, true)
)}
onClick={() => this.props.dispatch(changeLayout({layout: "clock"}))}
>
{t("sidebar:clock")}
</SidebarButton>
{selected==="clock" && this.renderScatterplotToggles()}
{selected==="clock" && this.renderBranchToggle()}
{selected==="clock" && this.renderRegressionToggle()}
</RowContainer>
) :
null
Expand All @@ -172,19 +155,13 @@ class ChooseLayout extends React.Component {
<ScatterIcon width={25} selected={selected === "scatter"}/>
<SidebarButton
selected={selected === "scatter"}
onClick={() => this.updateLayout(
"scatter",
validateScatterVariables(this.props.scatterVariables, this.props.colorings, this.props.distanceMeasure, this.props.colorBy, false)
)}
onClick={() => this.props.dispatch(changeLayout({layout: "scatter"}))}
>
{t("sidebar:scatter")}
</SidebarButton>
{selected==="scatter" && (
<>
{this.renderScatterplotAxesSelector()}
{this.renderScatterplotToggles()}
</>
)}
{selected==="scatter" && this.renderScatterplotAxesSelector()}
{selected==="scatter" && this.renderBranchToggle()}
{selected==="scatter" && this.renderRegressionToggle()}
</RowContainer>
</div>
);
Expand All @@ -206,7 +183,7 @@ const ScatterVariableContainer = styled.div`
flex-shrink: 1;
flex-basis: auto;
align-self: auto;
padding: 0px 0px 2px 15px;
padding: ${(props) => props.padAbove?"2":"0"}px 0px 2px 15px;
`;

const ScatterAxisName = styled.div`
Expand Down
11 changes: 11 additions & 0 deletions src/components/tree/phyloTree/grid.js
Original file line number Diff line number Diff line change
Expand Up @@ -245,6 +245,13 @@ export const addGrid = function addGrid() {
(this.layout!=="scatter" && this.distance==="num_date")
) {
xGridPoints = computeTemporalGridPoints(xmin, xmax, xAxisPixels, "x");
} else if (this.layout==="scatter" && !this.scatterVariables.xContinuous) {
xGridPoints = {
majorGridPoints: this.xScale.domain().map((name) => ({
name, visibility: "visible", axis: "x", position: name
})),
minorGridPoints: []
};
} else {
xGridPoints = computeNumericGridPoints(xmin, xmax, layout, this.params.minorTicks, "x");
}
Expand Down Expand Up @@ -319,6 +326,10 @@ export const addGrid = function addGrid() {
const yAxisPixels = this.yScale.range()[1] - this.yScale.range()[0];
const temporalGrid = computeTemporalGridPoints(ymin, ymax, yAxisPixels, "y");
majorGridPoints.push(...temporalGrid.majorGridPoints);
} else if (this.layout==="scatter" && !this.scatterVariables.yContinuous) {
majorGridPoints.push(...this.yScale.domain().map((name) => ({
name, visibility: "visible", axis: "y", position: name
})));
} else {
const numericGrid = computeNumericGridPoints(ymin, ymax, layout, 1, "y");
majorGridPoints.push(...numericGrid.majorGridPoints);
Expand Down
Loading

0 comments on commit 6b3eb64

Please sign in to comment.