Move ARAX to TRAPI 1.1

[amykglen]

realized we don't seem to have an issue for this yet - creating one so we have a place to document related work

some links from the mini-hackathon we had about this a little ways back:
https://github.com/NCATSTranslator/ReasonerAPI/tree/v1.1.x
https://github.com/NCATSTranslator/ReasonerAPI/blob/v1.1.x/TranslatorReasonerAPI.yaml
https://github.com/NCATSTranslator/ReasonerAPI/blob/v1.1.x/ChangeLog.md

(although it seems these links are now broken?)

[edeutsch]

TRAPI 1.1 schemas have been merged into master in the ReasonerAPI repo, so URLs are now:
https://github.com/NCATSTranslator/ReasonerAPI/
https://github.com/NCATSTranslator/ReasonerAPI/blob/master/TranslatorReasonerAPI.yaml
https://github.com/NCATSTranslator/ReasonerAPI/blob/master/ChangeLog.md

[edeutsch]

TRAPI 1.1 stuff is being set up in the NewFmt branch.
It is progressing nicely, but there is still more work to be done before I'll welcome everyone into the pool.
I'll let y'all know when it's safe to get in.

[edeutsch]

okay everyone, the pool is open, and let me tell you, changing half your id to ids (ideally the right half) is super fun!
The branch is NewFmt

The changes that need to be made are mostly just:
QNode.id pluralized to ids and changed from CURIE or array of CURIEs to just array of CURIEs with array length > 0
QNode.category pluralized to categories and changed from BiolinkEntity or array of BiolinkEntitys to just array of BiolinkEntitys with array length > 0
QEdge.predicate pluralized to predicates and changed from BiolinkPredicate or array of BiolinkPredicates to just array of BiolinkPredicates with array length > 0

• One amusing gotcha is that list of n>=1 seems to be enforced by the class setter, so you can't do qnode.ids = [] and append later

The following are done:
/ ARAX_database_manager.py
/ query_graph_info.py
/ ARAX_query.py
/ ARAX_messenger.py
/ ARAX_query_graph_interpreter.py
/ ARAX_response.py

and it was a gas!

The following need to be done:
ARAX_decorator.py
ARAX_expander.py
ARAX_filter.py
ARAX_filter_kg.py
ARAX_filter_results.py
ARAX_overlay.py
ARAX_ranker.py
ARAX_resultify.py
and then also any dependent files in subdirs
It would be great if you can take the ones you know best and join in the fun.
It would be great if we can get this done fairly quickly. the endpoints are configured and up, so as soon as we can get all tests passing, it should be pretty quick to getting things up and running.

Note that this will be parallel to our main endpoints for a while. The GUIs are:
https://arax.ncats.io/NewFmt/
https://arax.ncats.io/kg2NewFmt/

and the corresponding API endpoints are:
https://arax.ncats.io/api/arax/v1.1/ui/
https://arax.ncats.io/api/rtxkg2/v1.1/ui/
(note the v1.1 instead of v1.0)

The GUI will also need to be updated to speak TRAPI 1.1. It is not yet.
Just to be clear, don't expect these locations to work properly yet. Much code needs to be fixed and deployed before they will work properly. But they exist.

Thanks!

[dkoslicki]

@finnagin can you take care of the filter_* and you and @chunyuma can work on overlay?

[amykglen]

The latest code from NewFmt is rolled out to the kg2 1.1 endpoint, so you could test to see if that works.

awesome - ok, one issue (unearthed by all of today's work): resultify is complaining about a qnode that has a single curie but has is_set=True. the qnode starts out with is_set=None/False, but add_qnode() does this:

2021-04-26T17:44:12.384100 INFO: [] Adding a QueryNode to Message with input parameters {'key': 'n00', 'ids': ['MONDO:0015766'], 'name': None, 'categories': None, 'is_set': None, 'option_group_id': None}
2021-04-26T17:44:12.384370 WARNING: [] Specified ids '['MONDO:0015766']' is a list, but is_set was not set to true. It must be true in this context, so automatically setting to true. Avoid this warning by explictly setting to true.

should add_qnode() be adjusted so that is_set=True is forced only if ids is a list that has length greater than one?

[finnagin]

All the slow tests pass now!

[edeutsch]

should add_qnode() be adjusted so that is_set=True is forced only if ids is a list that has length greater than one?

yes, you're right! I have just checked in a fix I think.

[edeutsch]

I have just re-rolled out NewFmt to the two v1.1 deployments. @isbluis is working on the GUI so hopefully soon we'll be able to test with the GUI.

Thanks everyone!

[amykglen]

awesome! I just tested out having Expand use the v1.1 KG2 endpoint, and all tests (including slow) passed, so I just pushed that to NewFmt.

[edeutsch]

excellent! I just rolled out the latest NewFmt to both v1.1 endpoints.

[amykglen]

ok, I created the MetaKnowledgeGraph - it's on arax.ncats.io in the KG2.5.2 directory, called kg2c_meta_kg.json. it's about 6 MB, with 56 meta nodes and 53,161 meta edges. let me know if there seem to be any issues with the format! (I think I got all the required pieces..)

[edeutsch]

outstanding, thanks! Looks great and deployed!

Instead of using a browser, y'all may want to test with:

curl https://arax.ncats.io/api/rtxkg2/v1.1/meta_knowledge_graph > zz1.json

since it's a tad big.

[edeutsch]

TRAPI 1.1-supporting GUI and back-end is deployed to NewFmt (except for Build-A-Graph part)

https://arax.ncats.io/NewFmt/

Please test and report.

[amykglen]

when I go to https://arax.ncats.io/kg2NewFmt/, go to the JSON tab, click 'Example', and try to run that query, it gives an error:

An error was encountered while contacting the server (SyntaxError: Unexpected token < in JSON at position 0)

should that be working at this point?

[edeutsch]

Yes, it should certainly be working. I repro'ed and have now fixed. Please try again. you might need to CTRL+REFRESH to make sure you have the latest JavaScript. I suspect this was a casualty of the ingest of master.

[amykglen]

nice, looks good now!

[edeutsch]

call it done!