Check prefix before complete loading when checking yaml and cwl tools

[bernt-matthias]

I was wondering why planemo ci_find_repos tools-iuc/ (and ci_find_tools) need so much time: approx (3min). Turns out that all json files (some of them seem to be quite big .. which is a problem on its own) in there are parsed and checked if they are tools. Essentially almost the complete run time is caused by json files in tools/hamronization and tools/hyphy.

With this change the runtime is approx 10sec.

Support for json has been added here 0da66ee (before this json was explicitly forbidden) and likely is desired:

• Semantically it makes sense since json is a subset of yaml
• I know that tools can be yaml (or json .. unsure) .. but I never have seen one in the wild or documentation on how to write yaml/json tools.

I'm pretty sure that the PR is unacceptable as-is .. but should serve as a starting point for discussion:

• would it be sufficient to parse the header of the json / yaml files for the quick check if a file might be a tool?
• do we really want json?

How to test the changes?

(Select all options that apply)

• I've included appropriate automated tests.
• This is a refactoring of components with existing test coverage.
• Instructions for manual testing are as follows:
  1. [add testing steps and prerequisites here if you didn't write automated tests covering all your changes]

License

• I agree to license these contributions under Galaxy's current license.
• I agree to allow the Galaxy committers to license these and all my past contributions to the core galaxy codebase under the MIT license. If this condition is an issue, uncheck and just let us know why with an e-mail to galaxy-committers@lists.galaxyproject.org.

[jmchilton]

Is this coming through planemo/runnable.py::for_path on the slow walk through the code? I think it would be better to restrict the types of runnables you load in Planemo.

[jmchilton]

Does adding .json to CWL_EXTENSIONS in the next line to make up for this help anything or does that revert the performance. I would be fine with that change but I'm not sure it helps. I've seen CWL json files I think.

[bernt-matthias]

This does not have any effect on the performance since: looks_like_a_tool also rund looks_like_a_tool_cwl (because it is in BETA_TOOL_CHECKERS) which considers the CWL_EXTENSIONS:

galaxy/lib/galaxy/tool_util/loader_directory.py

Line 150 in cdf9253

for tool_checker in BETA_TOOL_CHECKERS.values():

[bernt-matthias]

Is this coming through planemo/runnable.py::for_path on the slow walk through the code? I think it would be better to restrict the types of runnables you load in Planemo.

No, this function seems not to be called at all during ci_find_repos or ci_find_tools

The trace is:

• planemo/tools.py::yield_tool_sources_on_paths
• planemo/tools.py::yield_tool_sources
• planemo/tools.py::load_tool_sources_from_path
• galaxy/tool_util/loader_directory.py::load_tool_sources_from_path
• galaxy/tool_util/loader_directory.py::_load_tools_from_path
• galaxy/tool_util/loader_directory.py::find_possible_tools_from_path
• galaxy/tool_util/loader_directory.py::looks_like_a_tool
• galaxy/tool_util/loader_directory.py::looks_like_a_tool_yaml
• galaxy/tool_util/loader_directory.py::is_a_yaml_with_class

[bernt-matthias]

From the committers meeting: This should be implemented in a different way: e.g. by just checking header lines of potential tools.

[bernt-matthias]

As a side node the tool parser factory only considers yml:

galaxy/lib/galaxy/tool_util/parser/factory.py

Line 71 in 8f72ec5

if config_file.endswith(".yml"):

[bernt-matthias]

Also this logger output is pretty confusing:

Loading tool from YAML - this is experimental - tool will not function in future.

So, we are sure that YAML tools will never work?

[bernt-matthias]

Implemented a prefix check for yaml and cwl tools.

For the IUC repo

planemo ci_find_repos before 125.35s after 5.01s
planemo ci_find_tool before 182.35s after 6.55s

Also checked that the output is equal.