-
Notifications
You must be signed in to change notification settings - Fork 20
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Programmatically use SmartAPI registry in Expand to find KP APIs #1466
Comments
While additions and alterations to smartapi.py may continue to be made as we identify a more clear use case, at the moment I have made some changes which return more information about trapi endpoints for more flexibility and convenience. Specifically, smartapi.py can be used to return all endpoints, all KPs (a subset of all endpoints), all dev KPs or all prod KPs (which are subsets of all KPs). These could be used for distinguishing endpoints for various purposes elsewhere in ARAX. Additionally, the functions which return these endpoints return more information, including the list of servers of that endpoint and the infores name. I will push these changes to a branch, Issue1466 shortly. |
After receiving a formal standard set by Translator for maturity levels in the smartAPI registry I continued to implement new functionality in smartapi.py on branch issue1466. Because not every TRAPI endpoint is standard in their smartapi registry entries, this code is only imperfectly able to return a list of endpoints and its relevant data. For instance, some KPs do not have infores names listed, some KPs have multiple registry entries, and some KPs have maturity levels not compliant with the four maturity levels specified by Translator. However, this module does well enough at the moment, especially for manual usage. It has been implemented with a CLI for this purpose. Users may get a list of all trapi endpoints, just endpoints which support workflows, or just endpoints which are labeled as KPs. Additionally, these lists can be filtered via the use of a whitelist and blacklist. KPs can be further filtered by specifying a required maturity level with strict or non-strict settings. We may implement automatic usage of this in some capacity in expand, such as a updating a cached list of KPs with certain parameters daily, but as of yet we haven't decided. |
…ing of TRAPI endpoints and KPs usage is explained in the file
Some example CLI commands:
|
There should be sufficient usage documentation in the smartapi.py module, but as an extra measure I can describe the flags here. I believe @edeutsch and @isbluis intend to take a look at the module and provide feedback.
|
awesome, thanks @rcpeene! this view is great:
so |
Yes, n_entries is the number of separate smartapi registry entries that exist with that infores name. This is only used when the -p argument is passed. Otherwise, the output just contains every individual smartapi registry entry without any such collating. |
I showed off this system (with Luis's front end) to everyone at the Architecture call and it was well received (some socks came clean off!) One suggestion was made: suggestion -- can links to the SmartAPI registry entry be added? e.g., https://smart-api.info/registry?q=acca268be3645a24556b81cc15ce0e9a @rcpeene are those URLs available in the query result from SmartAPI? Can you add those to your JSON output? (don't need it in the -p output) thanks! |
It took a bit of investigation, but I was able to figure out how to fetch the smartapi registry urls for endpoints. Now a new field is returned with the endpoint dict named "smartapi_url". |
brill! Looks great: |
Forgot to tag the commit that includes this latest info in the UI: 9ad6f41 |
dynamically selecting KPs from SmartAPI is high priority per 8/3 AHM - and TRAPI 1.3 ARAX should only use TRAPI 1.3 KPs we need to add to |
so we don't forget: when we do dynamic KP selection we will also need to address:
|
note to @rcpeene / myself: remember to regenerate |
…martapi module now. Made small changes to trapi_querier.py and ARAX_expander.py to accomodate this. issue #1466
…cally generated list of kps in expand and refactored kp parameter checking. Issue #1466
nice, thanks! that looks good to me! |
so arax.ci.transltr.io is now able to select KG2 as a KP successfully, but strangely, this error happens always at this same point in processing, without any message in the log: what's very odd is that no such error happens when running the same query locally, even when forcing a maturity of 'staging', like CI has. no idea what this could be... wondering if @edeutsch knows anything? |
I'm guessing that the live stream of JSON objects (that provides the logging and traffic light data is corrupted somehow. If you can provide an exact repro, maybe @isbluis can easily tell us what the offending JSON object is and we can trace the source? |
ah, ok. I see one query plan update message that it might possibly have been unhappy about - just pushed a change to that. I'll give it a few minutes to see if that happens to fix it, otherwise post back here with a repro. thanks! |
huh, ok, it's still broken. the way I'm seeing the error is by going to https://arax.ci.transltr.io/ (which is running the current code in |
So the stream and processing just appears to end there... |
|
If you run that, maybe need to fix dialect, you should see this:
i.e. right after is says it wants to use rtx-kg2, then connection suddenly drops without another word. weird. |
The UI is expected a bunch of status objects like that ultimately followed by the final TRAPI message. But there's not TRAPI message and so it act weird. Maybe @isbluis can take the opportunity to make it a little more robust in the face of this, but ultimately it is unexpected output. |
ah, that's very useful! I think I might've spotted the problem. looks like something is being added to the query plan log with a key that is None, instead of the KP's infores curie |
yep, that seems to have fixed it! arax.ci.transltr.io has finished rebuilding after my commit and now appears to be working. |
quite the list of KPs in the query summary table now! https://arax.ci.transltr.io/?r=57135 |
I see |
this also seems odd - ARAX apparently tries to query itself? though it says there isn't a TRAPI 1.3 endpoint for ARAX, which isn't true. wonder why? (@rcpeene) wonder if maybe the logic that creates the 'excluded_by_version' list is off? (and is somehow including ARAs in that list) |
Hi @amykglen . Regarding I ignore it, but maybe that's what is causing the issue? Maybe we should inform the owners of that server registration, and add more checks on our end? |
yeah, good idea, @isbluis! so I think we're in a working state except cicd.rtx.ai is still unhappy; some tests are failing, apparently because it's having trouble saving responses? not sure if @edeutsch knows anything about this. you can see output for its latest run of pytests here, by expanding the 'Run tests with pytest' section |
actually, I'm thinking that might resolve itself on cicd.rtx.ai's run for my latest commit (that fixed the |
yep, looks like it resolved itself. awesome. so the main issue remaining I'm aware of is ARAs appearing as KPs (for when @rcpeene has a chance). |
This bug with considering ARA's in |
looks great, thanks! (on https://arax.ci.transltr.io/) |
are we good to close this issue, @rcpeene? everything seems to be working great as far as I can tell! |
Yup! |
start using @edeutsch's new ARAXQuery/Expand/smartapi.py!
The text was updated successfully, but these errors were encountered: