this fixes (papers over) issue #547.

[rneher]

The issue was missing coordinates which result in missing transmissions from the index, which later results in 'type undefined' errors. This PR just sets lat/long to 0.0 instead. This will result in silly locations, but at least it doesn't break. This is a more general issue as to what should happen with incomplete geo-info.

[jameshadfield]

Hey @rneher -- could you run these JSONs through augur validate <JSONs> -- i'm interested if we pick up that they are "invalid".

I'm happy enough with a temporary solution to stop things crashing, but this is a deeper seated issue in the design of the map component which assumes that every value has a corresponding lat-long. I guess we should (a) not show links to and from "unknown" demes (unknown in the sense that their lat/long is unknown) and (b) display an error to the user.

[rneher]

fails, but for the wrong reason:

ran13:/home/richard/Projects/nextstrain/auspice/data
\:> augur validate --json pater_tree.json pater_meta.json 
Validating pater_tree.json using the tree schema (version nexflu-like)... SUCCESS
Validating pater_meta.json using the meta schema (version nexflu-like)... FAILED
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/27088323' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/28232448' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/28450510' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/28538723' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/28538734' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
	ERROR: 'https_//www.ncbi.nlm.nih.gov/pubmed/28538739' is not valid under any of the given schemas. Trace: ... - patternProperties - ^.+$ - properties - paper_url - oneOf
Validating that pater_meta.json and pater_tree.json are internally consistent... 
ran13:/home/richard/Projects/nextstrain/auspice/data

[jameshadfield]

Thanks... i'll fix export ASAP as this should definitely be an error. The issue of how Auspice should deal with "invalid" JSONs is a thorny one...

[rneher]

note that these were the ancient jsons that caused this issue back in May. Don't even know where they came from, you posted them on slack. My main motivation was to shorten the long issue list of auspice.

[trvrb]

I'm in favor of merging this. Having lots of redundancy in auspice seems like the way to go. Even if these JSONs won't validate, still better to have auspice not crash.

[jameshadfield]

This PR just sets lat/long to 0.0 instead. This will result in silly locations, but at least it doesn't break

I'm confused, on current master this dataset doesn't break (i'm assuming we're using the pater dataset from slack)
This PR adds in missing transmissions using a location of (0,0), whereas master just doesn't display those demes/transmission lines.

I'm very much in favor of (re-)displaying the error message, but not displaying transmissions/demes seems nicer than showing ones that have a fake location.

[trvrb]

I see. I'm agnostic in this case.

[rneher]

I tend to agree that not displaying them is better than drawing to some place in the Atlantic. But it is difficult for the user to know that smth is amiss if only a few lines are missing. My main reason to go back to this was to figure out where exactly this goes wrong for what reason. We had previously just caught it right before drawing. The root cause is info in the meta_json and that previously resulted in a corrupted the transmission data structure.

[rneher]

My take is that the transmission should exist in the data structure since it is in the json, but it should have coordinates unknown and a field valid=false or similar that we can use to avoid drawing it. current master catches corrupted transmission indices when updating, but this data structure is used in many other places...

[jameshadfield]

Augur PR nextstrain/augur#192 modifies augur validate to show the following when trying to validate these JSONs:

Validating that /Users/james/nextstrain/auspice/data/pater_meta.json and /Users/james/nextstrain/auspice/data/pater_tree.json are internally consistent...
	ERROR:  "vietnam", a value of the geographic resolution "country", appears in the tree but not in the metadata.
	ERROR:  	This will cause transmissions & demes involving this location not to be displayed in Auspice
	WARNING:  Color option "country", which contains a color_map, is missing "vietnam"

[jameshadfield]

Here's a possible fix (feel free to trash a1c66ac if it's not desired):

Demes/transmissions involving undefined lat/longs are not rendered & this error notification is shown on page load:

[jameshadfield]

@rneher @trvrb shall we merge this into master as currently stands?

[rneher]

My main reason to get back to this was to understand why this error happened and to reduce the number of high priority bug tags in the issue list. I think this is good to merge.

[jameshadfield]

Great!
Nice work on working out the root cause of this issue, and it's now well communicated in both auspice (via error notification box) and augur (via augur validate)