Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duration filters #2682

Merged
merged 11 commits into from
Oct 29, 2018
42 changes: 41 additions & 1 deletion doc/src/cylc-user-guide/cug.tex
Original file line number Diff line number Diff line change
Expand Up @@ -5138,7 +5138,7 @@ \subsubsection{Custom Jinja2 Filters, Tests and Globals}

In the argument list of filter or test function, the first argument is
the variable value to be ``filtered'' or ``tested'', respectively, and
subsequent arguments can be whatever else is needed. Currently there are two
subsequent arguments can be whatever else is needed. Currently there are three
custom filters:


Expand Down Expand Up @@ -5186,6 +5186,46 @@ \subsubsection{Custom Jinja2 Filters, Tests and Globals}
\item \lstinline={{'1066/10/14 08:00:00' | strftime('%Y%m%dT%H', '%Y/%m/%d %H:%M:%S')}}= - 10661014T08
\end{myitemize}

\paragraph{duration\_as}

The ``duration\_as'' filter can be used to format ISO8601 duration
strings as a floating-point number of several different units. Units
for the conversion can be specified in a case-insensitive short or long
form:
\begin{myitemize}
\item Seconds - ``s'' or ``seconds''
\item Minutes - ``m'' or ``minutes''
\item Hours - ``h`` or ``hours''
\item Days - ``d'' or ``days''
\item Weeks - ``w'' or ``weeks''
\end{myitemize}

Within the suite, this becomes

\lstset{language=suiterc}
\begin{lstlisting}
{% set CYCLE_INTERVAL = 'PT1D' %}
{{ CYCLE_INTERVAL | duration_as('h') }} # 24.0
{% set CYCLE_SUBINTERVAL = 'PT30M' %}
{{ CYCLE_SUBINTERVAL | duration_as('hours') }} # 0.5
{% set CYCLE_INTERVAL = 'PT1D' %}
{{ CYCLE_INTERVAL | duration_as('s') }} # 86400.0
{% set CYCLE_SUBINTERVAL = 'PT30M' %}
{{ CYCLE_SUBINTERVAL | duration_as('seconds') }} # 1800.0
\end{lstlisting}

While the filtered value is a floating-point number, it is often required to
supply an integer to suite entities (e.g.\ environment variables) that require
it. This is accomplished by chaining filters:

\begin{myitemize}
\item \lstinline={{CYCLE_INTERVAL | duration_as('h') | int}}= - 24
\item \lstinline={{CYCLE_SUBINTERVAL | duration_as('h') | int}}= - 0
\item \lstinline={{CYCLE_INTERVAL | duration_as('s') | int}}= - 86400
\item \lstinline={{CYCLE_SUBINTERVAL | duration_as('s') | int}}= - 1800
\end{myitemize}


\subsubsection{Associative Arrays In Jinja2}

Associative arrays ({\em dicts} in Python) can be very useful.
Expand Down
79 changes: 79 additions & 0 deletions lib/Jinja2Filters/duration_as.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# THIS FILE IS PART OF THE CYLC SUITE ENGINE.
# Copyright (C) 2008-2018 NIWA
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program. If not, see <http://www.gnu.org/licenses/>.
"""Provides a Jinja2 filter for formatting ISO8601 duration strings."""

from isodatetime.parsers import DurationParser

SECONDS_PER_MINUTE = 60.0
MINUTES_PER_HOUR = 60.0
HOURS_PER_DAY = 24.0
DAYS_PER_WEEK = 7.0

SECONDS_PER_HOUR = SECONDS_PER_MINUTE * MINUTES_PER_HOUR
SECONDS_PER_DAY = SECONDS_PER_HOUR * HOURS_PER_DAY
SECONDS_PER_WEEK = SECONDS_PER_DAY * DAYS_PER_WEEK

CONVERSIONS = {
('s', 'seconds'): float,
('m', 'minutes'): lambda s: float(s) / SECONDS_PER_MINUTE,
('h', 'hours'): lambda s: float(s) / SECONDS_PER_HOUR,
('d', 'days'): lambda s: float(s) / SECONDS_PER_DAY,
('w', 'weeks'): lambda s: float(s) / SECONDS_PER_WEEK,
}

def duration_as(iso8601_duration, units):
"""Format an iso8601 duration string as the specified units.

Args:
iso8601_duration (str): Any valid ISO8601 duration as a string.
units (str): Destination unit for the duration conversion

Return:
The total number of the specified unit contained in the specified
duration as a floating-point number.

Raises:
ISO8601SyntaxError: In the event of an invalid datetime string.

Examples:
>>> # Basic usage.
>>> duration_as('PT1M', 's')
60.0
>>> duration_as('PT1H', 'seconds')
3600.0

>>> # Exceptions.
>>> try:
... duration_as('invalid', 's') # Invalid duration
... except Exception as exc:
... print type(exc)
<class 'isodatetime.parsers.ISO8601SyntaxError'>
"""
for converter_names in CONVERSIONS:
if units.lower() in converter_names:
converter = CONVERSIONS[converter_names]
break
else:
raise ValueError('No matching units found for %s' % units)
return converter(DurationParser().parse(iso8601_duration).get_seconds())


if __name__ == "__main__":
for duration in ['PT1H', 'P1D', 'P7D']:
for short_name, _ in CONVERSIONS:
print short_name, duration_as(duration, short_name)
print '\n'

11 changes: 6 additions & 5 deletions tests/jinja2/09-custom-jinja2-filters/reference.log
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
2017-02-16T16:16:48Z INFO - Initial point: 01231212T1212Z
2017-02-16T16:16:48Z INFO - Final point: None
2017-02-16T16:16:48Z INFO - [strftime.01231212T1212Z] -triggered off []
2017-02-16T16:16:51Z INFO - [pad.01231212T1212Z] -triggered off ['strftime.01231212T1212Z']
2017-02-16T16:16:54Z INFO - [end.01231212T1212Z] -triggered off ['pad.01231212T1212Z']
2018-10-22T15:43:24Z INFO - Initial point: 01231212T1212Z
2018-10-22T15:43:24Z INFO - Final point: None
2018-10-22T15:43:24Z INFO - [strftime.01231212T1212Z] -triggered off []
2018-10-22T15:43:28Z INFO - [pad.01231212T1212Z] -triggered off ['strftime.01231212T1212Z']
2018-10-22T15:43:32Z INFO - [duration_as.01231212T1212Z] -triggered off ['pad.01231212T1212Z']
2018-10-22T15:43:36Z INFO - [end.01231212T1212Z] -triggered off ['duration_as.01231212T1212Z']
14 changes: 13 additions & 1 deletion tests/jinja2/09-custom-jinja2-filters/suite.rc
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
initial cycle point = 01231212T1212
[[dependencies]]
[[[R1]]]
graph = strftime => pad => end
graph = strftime => pad => duration_as => end
[runtime]
[[strftime]]
script = test $TEST_1 == '00'; test $TEST_2 == '30'
Expand All @@ -16,5 +16,17 @@
TEST_2 = {{ '12-30-2000' | strftime('%d', '%m-%d-%Y') }}
[[pad]]
script = test {{ 42 | pad(3, 0) }} == '042'
[[duration_as]]
script = """
test $TEST_1 == '3600'
test $TEST_2 == '1'
test $TEST_3 == '1'
test $TEST_4 == '1'
"""
[[[environment]]]
TEST_1 = {{ 'PT1H' | duration_as('seconds') | int }}
TEST_2 = {{ 'P7D' | duration_as('w') | int }}
TEST_3 = {{ 'PT60M' | duration_as('hours') | int }}
TEST_4 = {{ 'PT60S' | duration_as('minutes') | int }}
[[end]]
script = true