Duration filters

[trwhitcomb]

Add 2 new built-in Jinja2 filters that are able to take ISO8601 durations and convert them (using the builtin isodatetime module) into either decimal seconds or decimal hours. These allow ISO8601 durations to be specified in the suite, then used for tasks that aren't able to process ISO8601 syntax themselves.

By default, they return floating-point numbers in case the duration cannot be expressed directly as either seconds (unlikely) or hours (definitely possible). If an integer is desired, chain the Jinja2 int filter.

[trwhitcomb]

I may need some assistance from the more LateX-savvy developers. Getting the underscore in the paragraph labels (to be consistent with the other filter documentation) works for PDF, but resulted (at least for me) in some weird superscript characters or something in the HTML.

[hjoliver]

The LaTeX to HTML conversion is generally awful (bring on Sphinx...) but I'll take a quick look soon...

[hjoliver]

(@trwhitcomb - sorry for the delay on this, we have a lot on at the moment, will get to it soon...)

[hjoliver]

Looks good to me, and I can't see any spurious characters in the HTML user guide. Can you just extend tests/jinja2/09-custom-jinja2-filters.t to cover these new filters though?

[oliver-sanders]

This looks fine to me. My only comment is it would be nice to combine these as I can foresee the requirement for duration_to_days etc.

>>> duration('PT1H', '%S')
3600
>>> duration('PT1H', '%H')
1

{{ 'PT1H' | duration('%S') }}
# 3600
{{ 'PT1H' | duration('%H') }}
# 1

[hjoliver]

I second @oliver-sanders' idea.

[trwhitcomb]

@hjoliver @oliver-sanders This is definitely where I wanted to go, and I agree that it might be nice to combine them, and I originally started down that road. However, I wrapped myself around the axle for what the particular formats should be and, more importantly, what they should mean.

When talking about time point formats, it's clear that something like "%s" is the seconds component of the date. When we're talking about durations, it's ambiguous - you could be asking for "%s" to mean the seconds component of a duration (so for 'PT1H' it is 0) or you could mean the total seconds contained within a duration (i.e. 'PT1H' would be 3600.0).

I made the decision to keep these separate so reading the template would be more explicit, but also to highlight that it's the total time of the duration that happens to be expressed in those units (also why it's not an integer).

let me know what you think, but that's the direction that I was going.

[oliver-sanders]

Agreed the '%H' format I posted above is a bad idea, it implies you could also do '%H%S' etc, it was more for a gauge of the idea. How about an explicit filter performing the functionality of both, some options:

{{ 'PT1H' | duration('s') }}
{{ 'PT1H' | duration('seconds') }}
{{ 'PT1H' | duration(as='seconds') }}
{{ 'PT1H' | duration_as('seconds') }}

[hjoliver]

I agree with @oliver-sanders in that a single multi-purpose filter could be extended in the future rather than introducing more filters. (Climate suites might want duration as months or years...). And yeah, normal template notation %H etc. isn't good, as it suggests other options that aren't supported.

[TomekTrzeciak]

I like the keyword syntax the most: {{ 'PT1H' | duration(units='seconds') }}, it allows to add other options in the future.

A side note, with #2733 in master, you can do that without any filter now 😁

{% from '__python__.isodatetime.parsers' import DurationParser %}
{{ DurationParser().parse('PT1H').get_seconds() }}

[hjoliver]

@TomekTrzeciak raises a good point - post #2733 do we still need this PR? We could just document direct use of the isodatetime Python library...

[oliver-sanders]

Possibly, though the interface of the isodatetime library may change in the future, resource permitting.

[trwhitcomb]

@hjoliver @TomekTrzeciak #2733 looks really nice (I was just bemoaning the lack of "zip"!), but I think this should remain separate - rather than having the import in every single routine it's nice to have a few "builtin" convenience routines, especially once I get the keyword syntax implemented in there.

Sort of similar to how the update in #2733 would obviate the existing strftime filter, but I think a better comparison is the pad filter - that's easy enough to implement yourself in a suite, but it's so handy that I think it makes sense to include it in a centralized way.

[hjoliver]

@trwhitcomb - agreed, let's keep this PR. We will await your change to the interface as suggested above... (note also there's a minor conflict in the User Guide now).

[hjoliver]

Ping @trwhitcomb - I'm bumping this back to "soon" as we need to get next release out by end of month. (We can restore it to next-release if it turns out you are able to finish it off in time).

[trwhitcomb]

@hjoliver thanks for the reminder! Updates attached.

[hjoliver]

@trwhitcomb - you have a very minor conflict in the user guide. And one codacy-prompted improvement.

[trwhitcomb]

@hjoliver thanks - I think I got them both.

[oliver-sanders]

Looks good!

pycodestyle would like some whitespace changed (https://travis-ci.org/cylc/cylc/jobs/446433022#L1224), otherwise good to go.

[oliver-sanders]

OK, we are good here.

The Cylc Contributor Licence Agreement has changed since your last commit, before we can merge you need to agree to the new CLA by removing the parenthesis around your name in CONTRIBUTING.md.

See https://github.com/cylc/cylc/blob/master/CONTRIBUTING.md#code-contributors for details.

[hjoliver]

Thanks @trwhitcomb!