Add cancel/kill all invocations to a service or service instance

[slinkydeveloper]

Fix #827

[slinkydeveloper]

!11344 ➜ cargo run -- service cancel Counter --kill                                                                                     ~/projects/work/restate (issues/827) ✗
    Finished dev [unoptimized + debuginfo] target(s) in 0.23s
     Running `target/debug/restate service cancel Counter --kill`
 ID                                      TARGET                    STATUS             
 inv_1cw1Hbl2xwPi1Bapu1dHgUZKXdWstr3uzn  Counter/my-counter-3/add  running     
 inv_16ZFO31Wksqh7sz5A6BE1AHhp7uBCTqG53  Counter/my-counter-2/add  running     
 inv_1l8n0Bsg2WP35PIjesZk9jpqDMDJApOuCl  Counter/my-counter/get    backing-off 

✔ Are you sure you want to kill these invocations? · yes

✅ Request was sent successfully

[AhmedSoliman]

A) Should we put an upper bound on how many we display on the screen? (maybe a few examples and a message that says how many more?)
B) Do you think it'd cause problems to have this be an unbounded loop of cancellations (in situations with potentially large number of invocations)?

[slinkydeveloper]

Should we put an upper bound on how many we display on the screen?

We could introduce an upper bound, but then this means either that we don't show all the invocations we kill, or we need to shave off some invocations, meaning that during dev the user might have to run many times the kill command to kill all the invocations...

Do you think it'd cause problems to have this be an unbounded loop of cancellations (in situations with potentially large number of invocations)?

Only way to find out is testing it 😄 In principle here we just append commands to the log, so many commands probably will slow down the system a bit while all of those are processed.

[AhmedSoliman]

My 2c:

• I don't think we should have two cancel commands, cancelling a "service" doesn't read right to me. Alternatively. You might want to merge this logic into inv cancel?
• Should we make it more explicit that the intention is to cancel all invocations by adding --all?

[slinkydeveloper]

I don't think we should have two cancel commands, cancelling a "service" doesn't read right to me. Alternatively. You might want to merge this logic into inv cancel?

Sounds good to me, i can disambiguate the "query" by checking if the string starts with inv_ or not.

Should we make it more explicit that the intention is to cancel all invocations by adding --all?

I don't see the need for it TBH, it's quite explicit already and asks for confirmation too.

[AhmedSoliman]

LGTM. Let's run this as an experiment (maybe make it clear that this is a [BETA] command on help?) to see how it behaves and performs before we commit to this interface/behaviour.

[slinkydeveloper]

I would like to avoid adding BETA, as we're trying as much as we can to show that restate is stable.

[AhmedSoliman]

I would like to avoid adding BETA, as we're trying as much as we can to show that restate is stable.

A beta CLI command doesn't mean that restate is not stable! it's a signal that the command interface is experimental and we are likely to change the behavior.

[slinkydeveloper]

Yes but it can give the wrong impression that the cancel/kill feature is not stable, and not just the command interface. I also think that the whole CLI is probably experimental and we'll do some changes to it over time, perhaps breaking old commands too...