:expire trigger broken

[hjoliver]

By broken I mean, it has no effect.

When implementing the spawn-on-demand scheduler, I think I must have forgotten about the :expired trigger, and hence removed expired tasks before triggering on that "output" (because an expired task does not have any other outputs).

[hjoliver]

Per #5361 - we also need to consider required outputs of expired tasks.

[hjoliver]

Oops:

1712 edf61d368c cylc/flow/task_pool.py (Hilary Oliver    2020-04-24 13:29:54 +1200 1712)   
          # TODO [...] expired states are useless due to immediate
          # removal under all circumstances [...]

[MetRonnie]

Have reproduced with this

[scheduling]
    initial cycle point = previous(T00)
    [[special tasks]]
        clock-expire = foo(PT0S)
    [[graph]]
        R1 = """
            slow => foo
            foo:expire => bar
        """
[runtime]
    [[slow]]
        script = sleep 1
    [[foo]]
        script = false
    [[bar]]

In Cylc 7, foo would expire immediately and slow and bar would run. In Cylc 8, slow runs first, then foo expires and bar does not run, the workflow shuts down. This happens in back-compat mode too.

[hjoliver]

@MetRonnie - have you started working on this? If not, I can post a fix pretty quickly.

[hjoliver]

I'll take over this one, it appears @MetRonnie is not working on it...

[oliver-sanders]

Have had an issue reported where a task expired DURING its second try leaving behind a job in the submitted state which subsequently could not be killed (because the task isn't in the pool). Possibly related?

[hjoliver]

Not related to the "expire trigger" as such, but a good point to note. We should not allow an already-active task to expire.