Research app selection messaging

[Carifio24]

This PR adds a new message type, SelectionStateMessage, to research-app-messages, as well as functionality in the research app to broadcast these messages at appropriate times. Currently, the items that are broadcast on a change are the list of selected HiPS catalogs, the list of selected sources, and the most recent selected source.

I'm not sure if this is exactly how we want to handle the most recent selected source (which would be useful for e.g. glue-wwt). As things are now, every time a new source is selected, messages are broadcast for both the list of sources and the most recent source, which feels redundant. The alternative, I think, would be to only broadcast the source list message, and leave it on the user to identify whether the last source in the list is new or not.

[pkgw]

Cool. I have two main questions after looking this over:

1. Do you have a use case in mind for the ability to update clients on the list of active HiPS catalogs? It certainly seems like something we'll want at some point, but it seems like a slightly different topic than source selection. It's a bit redundant with the hipsCatalogNames field in the ApplicationStateMessage and I tend to think that further details should be thought of as belonging there, rather than as a kind of "selection".

2. Right now, the source and catalog items in the messages are expressed as strings, which seem to be JSON serializations of the app-side data structures. The messaging protocol needs to define these items a lot more rigorously, and we shouldn't be sending JSON-serialized data structures inside a larger JSON message except in very special circumstances. Can we rethink how to approach this? Ideally, someone should be able to write code that can interact with the app based on the docs of the research-app-messages alone — right now you'd have to go into the app source code to understand what to do with the "source" strings, and by not making their structure explicit we make it hard to establish and honor compatibility guarantees.

When a source is selected out of a catalog, the user is probably going to be interested in the various value-add columns provided by the catalog. So I think that the message delivered by the app ultimately ought to include the data from all of the source catalog's columns, potentially in a dictionary or in a two-line TSV format like the one used to convey bulk data — a client is already going to have to be able to parse the latter, so reusing the format here isn't ridiculous, I think. We could further give the clients hints about which columns represent RA, Dec, etc., but even that might not be needed.

[pkgw]

The whitespace here has gotten wonky somehow.

[pkgw]

There are some problems here — there's no threadId in the message and some of the types here should be array types. But overall, since this is an outgoing message, we don't have TypeScript code that needs to do rigorous checking of the message structure, so I think we should be able to skip this function altogether.

[Carifio24]

Yeah, various issues aside, you're right - we don't need this.

[Carifio24]

I didn't have a particular use case in mind for updating the users on the active HiPS catalogs. I was thinking of 'selection' in the broader sense of 'things the user has chosen to display', so it felt natural to include there, but I'm fine with sticking just to sources.

For the source format, I think your suggestions of either using either the TSV-string or dictionary formats are probably the best bet. Either method will require the user to parse things like numerical values themselves. Like you said, the user should already be able to parse TSV output, so maybe we should go with that?

For columns, each source contains ra and dec columns which contain the respective value in radians (as well as catalogName, a string containing the name of the catalog) that are used for the source selection in the app, so I think we can get away without column hints, since those are guaranteed to be present.

[pkgw]

@Carifio24 For the source data, it can just come down to whatever's more convenient to generate in the frontend. With the way that the source data ultimately come from the catalog data export, I think that would be TSV?

[Carifio24]

@pkgw Actually, the app stores each source as an object (whose values are all strings, with the exception of RA and Dec), so it's easier to just put the Source object(s) directly into the message.

From a pywwt point of view, this has the benefit that the received object is already a dictionary (I had expected that it would be a string representation of a dictionary, and so the user would need to parse it themselves, but that doesn't seem to be necessary).

[pkgw]

OK, a couple of fiddly bits about the interface definition and docs to tidy up, I think, but I think we've got the fundamentals down here.

[pkgw]

Er, I think this docstring needs to be relocated?

[Carifio24]

Yep - this got left behind when I was moving things into selections.ts.

[pkgw]

Hm, this docstring should probably just refer the reader to the description of ViewStateMessage.sessionId for its semantics. I see that I just copy-pasted the view-state doc into ApplicationStateMessage.sessionId as well, but it would be better to have a single source of truth for this kind of thing.

[Carifio24]

That makes sense. I'll modify this description to reference ViewStateMessage.sessionId.

[pkgw]

Here I think we need to declare a TypeScript indexer to indicate that there will be additional fields for the catalog columns, and mention that in the docstring.

It might be good to namespace the arbitrary catalog data more strongly (e.g., you can worry about potential issues about well-definedness if the catalog provides a column named ra, etc.) but we're mirroring the internal app data structures here, and I'd prefer not to spend too much time fiddling with those, and I think what we've got here will be fine.

[Carifio24]

This potential issue of well-definedness of name is also present in the app itself. I agree that we don't want to spend too much time messing with the data structure in the app, but I'm thinking we could go with something along the lines of:

export interface Source {
    ra: number;
    dec: number;
    name: string;
    catalogName: string;
    info: {
        [field: string]: string | undefined;
    }
}

where the info object contains all of the catalog data.

[pkgw]

Yeah, that would be the most sensible way to encapsulate, I think. Maybe name the field catalogData or something. If you think it will be straightforward to modify the app's Source object this way too, I do think it would be good for future-proofiness.

[Carifio24]

So I modified the Source interface in the format we had discussed in both the messages and the research app itself.

While I was modifying the app's Source to use this format, I noticed an issue with the way that the Source objects were being constructed - basically, they had initially been created without a name, but TypeScript hadn't noticed because they were partially created from an unpacked object. To fix this, I added a RawSourceInfo interface, which is now how the app stores the closest source when the mouse is moving, before the user clicks on it. This then gets converted into a Source and added to the database when the user makes the selection. This change should also make closestInView ever-so-slightly more efficient.

[pkgw]

OK, I think that's everything!