Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tasks: general system for recognizing and executing service work #1343

Merged
merged 200 commits into from
Dec 8, 2023

Conversation

sam0x17
Copy link
Contributor

@sam0x17 sam0x17 commented Aug 31, 2023

polkadot-sdk version of original tasks PR located here: paritytech/substrate#14329

Fixes #206

Status

  • Generic Task trait
  • RuntimeTask aggregated enum, compatible with construct_runtime!
  • Casting between Task and RuntimeTask without needing dyn or Box
  • Tasks Example pallet
  • Runtime tests for Tasks example pallet
  • Parsing for task-related macros
  • Retrofit parsing to make macros optional
  • Expansion for task-related macros
  • Adds support for args in tasks
  • Retrofit tasks example pallet to use macros instead of manual syntax
  • Weights
  • Cleanup
  • UI tests
  • Docs

Target Syntax

Adapted from #206 (comment)

// NOTE: this enum is optional and is auto-generated by the other macros if not present
#[pallet::task]
pub enum Task<T: Config> {
    AddNumberIntoTotal {
        i: u32,
    }
}

/// Some running total.
#[pallet::storage]
pub(super) type Total<T: Config<I>, I: 'static = ()> =
StorageValue<_, (u32, u32), ValueQuery>;

/// Numbers to be added into the total.
#[pallet::storage]
pub(super) type Numbers<T: Config<I>, I: 'static = ()> =
StorageMap<_, Twox64Concat, u32, u32, OptionQuery>;

#[pallet::tasks_experimental]
impl<T: Config<I>, I: 'static> Pallet<T, I> {
	/// Add a pair of numbers into the totals and remove them.
	#[pallet::task_list(Numbers::<T, I>::iter_keys())]
	#[pallet::task_condition(|i| Numbers::<T, I>::contains_key(i))]
	#[pallet::task_index(0)]
	pub fn add_number_into_total(i: u32) -> DispatchResult {
		let v = Numbers::<T, I>::take(i).ok_or(Error::<T, I>::NotFound)?;
		Total::<T, I>::mutate(|(total_keys, total_values)| {
			*total_keys += i;
			*total_values += v;
		});
		Ok(())
	}
}

@sam0x17 sam0x17 self-assigned this Aug 31, 2023
@sam0x17 sam0x17 added the T1-FRAME This PR/Issue is related to core FRAME, the framework. label Aug 31, 2023
@sam0x17
Copy link
Contributor Author

sam0x17 commented Sep 6, 2023

btw this branch is going to get force-pushed over later today -- my version that actually has the correct rebase with polkadot-sdk is local right now. This one had some issues.

@sam0x17
Copy link
Contributor Author

sam0x17 commented Sep 6, 2023

remaining compile issue:

the trait bound `RuntimeEvent: From<tasks_example::Event<Runtime>>` is not satisfied

is this because I'm not generating a From impl that I should be generating in my RuntimeTask macro expansion, or do I have something else wrong somewhere else?

I just find it strange that it suddenly needs this because I didn't need this impl at all on the substrate version of this branch.

@KiChjang
Copy link
Contributor

KiChjang commented Sep 6, 2023

Did you forget to include the tasks_example pallet in your construct_runtime?

@sam0x17
Copy link
Contributor Author

sam0x17 commented Sep 6, 2023

Did you forget to include the tasks_example pallet in your construct_runtime?

lmao thank you yes I think this was one of the lines the patch file misses

@sam0x17
Copy link
Contributor Author

sam0x17 commented Sep 6, 2023

Did you forget to include the tasks_example pallet in your construct_runtime?

oh yeah and then the weird follow-up to that that I remember fixing for substrate but now forget how I fixed:

after adding TasksExample: tasks_example to construct_runtime, I then get this error on the construct_runtime!:

  error: expected one of: `Pallet`, `Call`, `Storage`, `Event`, `Error`, `Config`, `Origin`, `Inherent`, `ValidateUnsigned`, `FreezeReason`, `HoldReason`, `Task`, `LockId`, `SlashReason`

🤔

(now pushed up on this branch)

@gupnik
Copy link
Contributor

gupnik commented Sep 7, 2023

Did you forget to include the tasks_example pallet in your construct_runtime?

oh yeah and then the weird follow-up to that that I remember fixing for substrate but now forget how I fixed:

after adding TasksExample: tasks_example to construct_runtime, I then get this error on the construct_runtime!:

  error: expected one of: `Pallet`, `Call`, `Storage`, `Event`, `Error`, `Config`, `Origin`, `Inherent`, `ValidateUnsigned`, `FreezeReason`, `HoldReason`, `Task`, `LockId`, `SlashReason`

🤔

(now pushed up on this branch)

@sam0x17 It seems that task_part becomes None if a pallet doesn't have any tasks. In that case, we probably want to set it to quote::quote!() inside tt_default_parts. Pushed the fix.

@sam0x17
Copy link
Contributor Author

sam0x17 commented Sep 11, 2023

new one @KiChjang / @gupnik:

edit: resolved

On current version of the branch, when compiling substrate/bin/node/runtime, I currently get:

     Compiling kitchensink-runtime v3.0.0-dev (/home/sam/workspace/polkadot-sdk/substrate/bin/node/runtime)
  error[E0532]: expected tuple struct or tuple variant, found type alias `TasksExample`
      --> /home/sam/workspace/polkadot-sdk/substrate/bin/node/runtime/src/lib.rs:2096:3
       |
  2096 |         TasksExample: tasks_example,
       |         ^^^^^^^^^^^^
       |

All I've changed is I've added proper impls for the RuntimeTask methods including enumerate(), and previously this line was fine. Is there something obvious I'm missing?

@sam0x17
Copy link
Contributor Author

sam0x17 commented Dec 7, 2023

🎉

Copy link
Contributor

@louismerlin louismerlin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving for bot, SRLabs will review this in the future, PR is in pipeline

@gupnik gupnik merged commit ac3f14d into master Dec 8, 2023
117 checks passed
@gupnik gupnik deleted the sam-tasks branch December 8, 2023 05:40
alexggh added a commit that referenced this pull request Dec 10, 2023
The test frame ui started failing consistently on latest master [1]. I
assume it was because of a race between
#1343 which introduced
this warning and a PR that updated our tooling version, hence the
warnings don't match perfectly, so regenerated them with
`TRYBUILD=overwrite` as the test suggests.

[1] https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4666766

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
bkchr pushed a commit that referenced this pull request Dec 10, 2023
The test frame ui started failing consistently on latest master [1]. I
assume it was because of a race between
#1343 which introduced
this warning and a PR that updated our tooling version, hence the
warnings don't match perfectly, so regenerated them with
`TRYBUILD=overwrite` as the test suggests.

[1] https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4666766

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
gupnik pushed a commit that referenced this pull request Dec 15, 2023
#1343 introduced Tasks
API. This one moves `do_task` call in frame_system under the
experimental flag, till the previous one is audited.

---------

Co-authored-by: command-bot <>
bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 26, 2024
@Polkadot-Forum
Copy link

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-release-analysis-v1-6-0/5855/1

bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 29, 2024
bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 29, 2024
bkontur added a commit to bkontur/runtimes that referenced this pull request Jan 30, 2024
bkontur added a commit to bkontur/runtimes that referenced this pull request Feb 3, 2024
bkontur added a commit to bkontur/runtimes that referenced this pull request Feb 3, 2024
bkontur added a commit to bkontur/runtimes that referenced this pull request Feb 6, 2024
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
…itytech#1343)

`polkadot-sdk` version of original tasks PR located here:
paritytech/substrate#14329

Fixes paritytech#206

## Status
- [x] Generic `Task` trait
- [x] `RuntimeTask` aggregated enum, compatible with
`construct_runtime!`
- [x] Casting between `Task` and `RuntimeTask` without needing `dyn` or
`Box`
- [x] Tasks Example pallet
- [x] Runtime tests for Tasks example pallet
- [x] Parsing for task-related macros
- [x] Retrofit parsing to make macros optional
- [x] Expansion for task-related macros
- [x] Adds support for args in tasks
- [x] Retrofit tasks example pallet to use macros instead of manual
syntax
- [x] Weights
- [x] Cleanup
- [x] UI tests
- [x] Docs

## Target Syntax
Adapted from
paritytech#206 (comment)

```rust
// NOTE: this enum is optional and is auto-generated by the other macros if not present
#[pallet::task]
pub enum Task<T: Config> {
    AddNumberIntoTotal {
        i: u32,
    }
}

/// Some running total.
#[pallet::storage]
pub(super) type Total<T: Config<I>, I: 'static = ()> =
StorageValue<_, (u32, u32), ValueQuery>;

/// Numbers to be added into the total.
#[pallet::storage]
pub(super) type Numbers<T: Config<I>, I: 'static = ()> =
StorageMap<_, Twox64Concat, u32, u32, OptionQuery>;

#[pallet::tasks_experimental]
impl<T: Config<I>, I: 'static> Pallet<T, I> {
	/// Add a pair of numbers into the totals and remove them.
	#[pallet::task_list(Numbers::<T, I>::iter_keys())]
	#[pallet::task_condition(|i| Numbers::<T, I>::contains_key(i))]
	#[pallet::task_index(0)]
	pub fn add_number_into_total(i: u32) -> DispatchResult {
		let v = Numbers::<T, I>::take(i).ok_or(Error::<T, I>::NotFound)?;
		Total::<T, I>::mutate(|(total_keys, total_values)| {
			*total_keys += i;
			*total_values += v;
		});
		Ok(())
	}
}
```

---------

Co-authored-by: Nikhil Gupta <17176722+gupnik@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Nikhil Gupta <>
Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io>
Co-authored-by: gupnik <nikhilgupta.iitk@gmail.com>
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
The test frame ui started failing consistently on latest master [1]. I
assume it was because of a race between
paritytech#1343 which introduced
this warning and a PR that updated our tooling version, hence the
warnings don't match perfectly, so regenerated them with
`TRYBUILD=overwrite` as the test suggests.

[1] https://gitlab.parity.io/parity/mirrors/polkadot-sdk/-/jobs/4666766

---------

Signed-off-by: Alexandru Gheorghe <alexandru.gheorghe@parity.io>
bgallois pushed a commit to duniter/duniter-polkadot-sdk that referenced this pull request Mar 25, 2024
paritytech#1343 introduced Tasks
API. This one moves `do_task` call in frame_system under the
experimental flag, till the previous one is audited.

---------

Co-authored-by: command-bot <>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T1-FRAME This PR/Issue is related to core FRAME, the framework.
Projects
Status: Done
Status: Backlog
Development

Successfully merging this pull request may close these issues.

[FRAME Core] General system for recognising and executing service work
10 participants