From eaf1bc5633ebbacce97e4f167ebe1d0d268c4b24 Mon Sep 17 00:00:00 2001 From: Kian Paimani <5588131+kianenigma@users.noreply.github.com> Date: Thu, 30 Nov 2023 14:15:46 +0300 Subject: [PATCH] Introduce Polkadot-Sdk `developer_hub` (#2102) This PR introduces the new crate `developer_hub` into the polkadot-sdk repo. The vision for the developer-hub crate is detailed in [this document](https://docs.google.com/document/d/1XLLkFNE8v8HLvZpI2rzsa8N2IN1FcKntc8q-Sc4xBAk/edit?usp=sharing). Screenshot 2023-11-02 at 10 45 48 Other than adding the new crate, it also does the following: * Remove the `substrate` crate, as there is now a unique umbrella crate for multiple things in `developer_hub::polkadot_sdk`. * (backport candidate) A minor change to `frame-support` macros that allows `T::RuntimeOrigin` to also be acceptable as the origin type. * (backport candidate) A minor change to `frame-system` that allows us to deposit events at genesis because now the real genesis config is generated via wasm, and we can safely assume `cfg!(feature = "std")` means only testing. related to #62. * (backport candidate) Introduces a small `read_events_for_pallet` to `frame_system` for easier event reading in tests. * From https://github.com/paritytech/polkadot-sdk-docs/issues/31, it takes action on improving the `pallet::call` docs. * From https://github.com/paritytech/polkadot-sdk-docs/issues/31, it takes action on improving the `UncheckedExtrinsic` docs. ## Way Forward First, a version of this is deployed temporarily [here](https://blog.kianenigma.nl/polkadot-sdk/developer_hub/index.html). I will keep it up to date on a daily basis. ### This Pull Request I see two ways forward: 1. We acknowledge that everything in `developer-hub` is going to be WIP, and merge this asap. We should not yet use links to this crate anywhere. 2. We make this be the feature branch, make PRs against this, and either gradually backport it, or only merge to master once it is done. I am personally in favor of option 1. If we stick to option 2, we need a better way to deploy a staging version of this to gh-pages. ### Issue Tracking The main issues related to the future of `developer_hub` are: - https://github.com/paritytech/polkadot-sdk-docs/issues/31 - https://github.com/paritytech/polkadot-sdk-docs/issues/4 - https://github.com/paritytech/polkadot-sdk-docs/issues/26 - https://github.com/paritytech/polkadot-sdk-docs/issues/32 - https://github.com/paritytech/polkadot-sdk-docs/issues/36 ### After This Pull Request - [ ] create a redirect for https://paritytech.github.io/polkadot-sdk/master/substrate/ - [x] analytics - [ ] link checker - [ ] the matter of publishing, and how all of these relative links for when we do, that is still an open question. There is section on this in the landing page. - [ ] updated https://paritytech.github.io/ --------- Co-authored-by: Liam Aharon Co-authored-by: Juan Girini Co-authored-by: bader y Co-authored-by: Sebastian Kunert Co-authored-by: James Wilson Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com> --- .gitlab/pipeline/build.yml | 2 +- Cargo.lock | 76 +- Cargo.toml | 5 +- cumulus/parachain-template/runtime/src/lib.rs | 3 +- developer-hub/Cargo.toml | 66 ++ developer-hub/headers/toc.html | 54 ++ .../src/guides/changing_consensus.rs | 1 + .../src/guides/cumulus_enabled_parachain.rs | 1 + developer-hub/src/guides/mod.rs | 25 + .../src/guides/xcm_enabled_parachain.rs | 1 + developer-hub/src/guides/your_first_node.rs | 1 + .../src/guides/your_first_pallet/mod.rs | 738 ++++++++++++++++++ .../guides/your_first_pallet/with_event.rs | 101 +++ .../src/guides/your_first_runtime.rs | 1 + developer-hub/src/lib.rs | 43 + developer-hub/src/meta_contributing.rs | 146 ++++ developer-hub/src/polkadot_sdk/cumulus.rs | 130 +++ .../src/polkadot_sdk/frame_runtime.rs | 179 +++++ developer-hub/src/polkadot_sdk/mod.rs | 134 ++++ developer-hub/src/polkadot_sdk/polkadot.rs | 87 +++ .../src/polkadot_sdk/smart_contracts.rs | 9 + developer-hub/src/polkadot_sdk/substrate.rs | 151 ++++ developer-hub/src/polkadot_sdk/templates.rs | 45 ++ developer-hub/src/polkadot_sdk/xcm.rs | 5 + .../reference_docs/blockchain_scalibility.rs | 0 .../blockchain_state_machines.rs | 29 + .../src/reference_docs/chain_spec_genesis.rs | 4 + developer-hub/src/reference_docs/cli.rs | 7 + .../src/reference_docs/consensus_swapping.rs | 6 + .../src/reference_docs/extrinsic_encoding.rs | 277 +++++++ .../src/reference_docs/fee_less_runtime.rs | 12 + .../frame_benchmarking_weight.rs | 23 + .../reference_docs/frame_composite_enums.rs | 1 + .../src/reference_docs/frame_currency.rs | 8 + .../src/reference_docs/frame_origin.rs | 14 + .../reference_docs/frame_runtime_migration.rs | 9 + .../reference_docs/frame_system_accounts.rs | 8 + developer-hub/src/reference_docs/glossary.rs | 62 ++ .../src/reference_docs/light_nodes.rs | 7 + developer-hub/src/reference_docs/metadata.rs | 1 + developer-hub/src/reference_docs/mod.rs | 99 +++ .../runtime_vs_smart_contract.rs | 6 + .../safe_defensive_programming.rs | 1 + .../src/reference_docs/signed_extensions.rs | 79 ++ .../reference_docs/trait_based_programming.rs | 229 ++++++ .../src/reference_docs/wasm_memory.rs | 7 + .../src/reference_docs/wasm_meta_protocol.rs | 113 +++ docs/mermaid/IA.mmd | 14 + docs/mermaid/extrinsics.mmd | 5 + docs/mermaid/polkadot_sdk_parachain.mmd | 11 + docs/mermaid/polkadot_sdk_polkadot.mmd | 10 + docs/mermaid/polkadot_sdk_substrate.mmd | 8 + docs/mermaid/state.mmd | 16 + docs/mermaid/stf.mmd | 21 + docs/mermaid/stf_simple.mmd | 4 + docs/mermaid/substrate_client_runtime.mmd | 8 +- docs/mermaid/substrate_dev.mmd | 2 + docs/mermaid/substrate_simple.mmd | 2 +- docs/mermaid/substrate_with_frame.mmd | 6 +- .../protocol/src/request_response/mod.rs | 4 +- polkadot/node/primitives/src/lib.rs | 2 +- substrate/Cargo.toml | 30 - substrate/client/allocator/src/lib.rs | 2 +- substrate/client/executor/src/lib.rs | 2 +- substrate/frame/balances/src/lib.rs | 48 +- substrate/frame/bounties/src/migrations/v4.rs | 4 +- .../frame/collective/src/migrations/v4.rs | 4 +- .../election-provider-multi-phase/src/mock.rs | 6 +- .../elections-phragmen/src/migrations/v4.rs | 4 +- .../elections-phragmen/src/migrations/v5.rs | 2 +- .../frame/examples/kitchensink/src/lib.rs | 5 +- substrate/frame/fast-unstake/src/types.rs | 1 + substrate/frame/grandpa/src/migrations/v4.rs | 4 +- .../frame/membership/src/migrations/v4.rs | 4 +- .../frame/session/src/historical/offchain.rs | 10 +- substrate/frame/src/lib.rs | 36 +- substrate/frame/support/procedural/src/lib.rs | 46 +- .../procedural/src/pallet/parse/call.rs | 34 +- substrate/frame/support/src/dispatch.rs | 3 +- substrate/frame/support/src/lib.rs | 152 +++- substrate/frame/support/src/migrations.rs | 3 +- substrate/frame/support/src/storage/child.rs | 6 +- substrate/frame/support/src/storage/mod.rs | 6 +- .../deprecated_where_block.stderr | 8 +- .../pallet_ui/call_invalid_origin_type.stderr | 2 +- substrate/frame/system/Cargo.toml | 2 +- substrate/frame/system/src/lib.rs | 24 +- substrate/frame/tips/src/migrations/v4.rs | 4 +- substrate/primitives/api/src/lib.rs | 16 +- .../primitives/npos-elections/src/lib.rs | 5 +- .../runtime-interface/src/pass_by.rs | 2 +- substrate/primitives/runtime/Cargo.toml | 3 + .../runtime/src/generic/checked_extrinsic.rs | 8 +- .../src/generic/unchecked_extrinsic.rs | 32 +- .../runtime/src/offchain/storage_lock.rs | 2 +- substrate/src/lib.rs | 242 ------ 96 files changed, 3410 insertions(+), 471 deletions(-) create mode 100644 developer-hub/Cargo.toml create mode 100644 developer-hub/headers/toc.html create mode 100644 developer-hub/src/guides/changing_consensus.rs create mode 100644 developer-hub/src/guides/cumulus_enabled_parachain.rs create mode 100644 developer-hub/src/guides/mod.rs create mode 100644 developer-hub/src/guides/xcm_enabled_parachain.rs create mode 100644 developer-hub/src/guides/your_first_node.rs create mode 100644 developer-hub/src/guides/your_first_pallet/mod.rs create mode 100644 developer-hub/src/guides/your_first_pallet/with_event.rs create mode 100644 developer-hub/src/guides/your_first_runtime.rs create mode 100644 developer-hub/src/lib.rs create mode 100644 developer-hub/src/meta_contributing.rs create mode 100644 developer-hub/src/polkadot_sdk/cumulus.rs create mode 100644 developer-hub/src/polkadot_sdk/frame_runtime.rs create mode 100644 developer-hub/src/polkadot_sdk/mod.rs create mode 100644 developer-hub/src/polkadot_sdk/polkadot.rs create mode 100644 developer-hub/src/polkadot_sdk/smart_contracts.rs create mode 100644 developer-hub/src/polkadot_sdk/substrate.rs create mode 100644 developer-hub/src/polkadot_sdk/templates.rs create mode 100644 developer-hub/src/polkadot_sdk/xcm.rs create mode 100644 developer-hub/src/reference_docs/blockchain_scalibility.rs create mode 100644 developer-hub/src/reference_docs/blockchain_state_machines.rs create mode 100644 developer-hub/src/reference_docs/chain_spec_genesis.rs create mode 100644 developer-hub/src/reference_docs/cli.rs create mode 100644 developer-hub/src/reference_docs/consensus_swapping.rs create mode 100644 developer-hub/src/reference_docs/extrinsic_encoding.rs create mode 100644 developer-hub/src/reference_docs/fee_less_runtime.rs create mode 100644 developer-hub/src/reference_docs/frame_benchmarking_weight.rs create mode 100644 developer-hub/src/reference_docs/frame_composite_enums.rs create mode 100644 developer-hub/src/reference_docs/frame_currency.rs create mode 100644 developer-hub/src/reference_docs/frame_origin.rs create mode 100644 developer-hub/src/reference_docs/frame_runtime_migration.rs create mode 100644 developer-hub/src/reference_docs/frame_system_accounts.rs create mode 100644 developer-hub/src/reference_docs/glossary.rs create mode 100644 developer-hub/src/reference_docs/light_nodes.rs create mode 100644 developer-hub/src/reference_docs/metadata.rs create mode 100644 developer-hub/src/reference_docs/mod.rs create mode 100644 developer-hub/src/reference_docs/runtime_vs_smart_contract.rs create mode 100644 developer-hub/src/reference_docs/safe_defensive_programming.rs create mode 100644 developer-hub/src/reference_docs/signed_extensions.rs create mode 100644 developer-hub/src/reference_docs/trait_based_programming.rs create mode 100644 developer-hub/src/reference_docs/wasm_memory.rs create mode 100644 developer-hub/src/reference_docs/wasm_meta_protocol.rs create mode 100644 docs/mermaid/IA.mmd create mode 100644 docs/mermaid/extrinsics.mmd create mode 100644 docs/mermaid/polkadot_sdk_parachain.mmd create mode 100644 docs/mermaid/polkadot_sdk_polkadot.mmd create mode 100644 docs/mermaid/polkadot_sdk_substrate.mmd create mode 100644 docs/mermaid/state.mmd create mode 100644 docs/mermaid/stf.mmd create mode 100644 docs/mermaid/stf_simple.mmd create mode 100644 docs/mermaid/substrate_dev.mmd delete mode 100644 substrate/Cargo.toml delete mode 100644 substrate/src/lib.rs diff --git a/.gitlab/pipeline/build.yml b/.gitlab/pipeline/build.yml index 5c13045706c4..d6918173d493 100644 --- a/.gitlab/pipeline/build.yml +++ b/.gitlab/pipeline/build.yml @@ -125,7 +125,7 @@ build-rustdoc: find "$path" -name '*.html' | xargs -I {} -P "$(nproc)" bash -c 'process_file "$@"' _ {} } inject_simple_analytics "./crate-docs" - - echo "" > ./crate-docs/index.html + - echo "" > ./crate-docs/index.html build-implementers-guide: stage: build diff --git a/Cargo.lock b/Cargo.lock index 346ce2f933f1..de4471783abf 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -4366,6 +4366,45 @@ dependencies = [ "syn 1.0.109", ] +[[package]] +name = "developer-hub" +version = "0.0.1" +dependencies = [ + "cumulus-pallet-aura-ext", + "cumulus-pallet-parachain-system", + "docify", + "frame", + "kitchensink-runtime", + "pallet-aura", + "pallet-default-config-example", + "pallet-examples", + "pallet-timestamp", + "parity-scale-codec", + "sc-cli", + "sc-client-db", + "sc-consensus-aura", + "sc-consensus-babe", + "sc-consensus-beefy", + "sc-consensus-grandpa", + "sc-consensus-manual-seal", + "sc-consensus-pow", + "sc-network", + "sc-rpc", + "sc-rpc-api", + "scale-info", + "simple-mermaid 0.1.0 (git+https://github.com/kianenigma/simple-mermaid.git?branch=main)", + "sp-api", + "sp-core", + "sp-io", + "sp-keyring", + "sp-runtime", + "staging-chain-spec-builder", + "staging-node-cli", + "staging-parachain-info", + "subkey", + "substrate-wasm-builder", +] + [[package]] name = "diff" version = "0.1.13" @@ -5201,7 +5240,7 @@ dependencies = [ "pallet-examples", "parity-scale-codec", "scale-info", - "simple-mermaid", + "simple-mermaid 0.1.0 (git+https://github.com/kianenigma/simple-mermaid.git?rev=e48b187bcfd5cc75111acd9d241f1bd36604344b)", "sp-api", "sp-arithmetic", "sp-block-builder", @@ -11175,9 +11214,9 @@ dependencies = [ [[package]] name = "parity-scale-codec" -version = "3.6.4" +version = "3.6.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "dd8e946cc0cc711189c0b0249fb8b599cbeeab9784d83c415719368bb8d4ac64" +checksum = "0dec8a8073036902368c2cdc0387e85ff9a37054d7e7c98e592145e0c92cd4fb" dependencies = [ "arrayvec 0.7.4", "bitvec", @@ -11190,9 +11229,9 @@ dependencies = [ [[package]] name = "parity-scale-codec-derive" -version = "3.6.4" +version = "3.6.5" source = "registry+https://github.com/rust-lang/crates.io-index" -checksum = "2a296c3079b5fefbc499e1de58dc26c09b1b9a5952d26694ee89f04a43ebbb3e" +checksum = "312270ee71e1cd70289dacf597cab7b207aa107d2f28191c2ae45b2ece18a260" dependencies = [ "proc-macro-crate", "proc-macro2", @@ -16644,6 +16683,11 @@ dependencies = [ "wide", ] +[[package]] +name = "simple-mermaid" +version = "0.1.0" +source = "git+https://github.com/kianenigma/simple-mermaid.git?branch=main#e48b187bcfd5cc75111acd9d241f1bd36604344b" + [[package]] name = "simple-mermaid" version = "0.1.0" @@ -17497,6 +17541,7 @@ dependencies = [ name = "sp-runtime" version = "24.0.0" dependencies = [ + "docify", "either", "hash256-std-hasher", "impl-trait-for-tuples", @@ -17507,6 +17552,7 @@ dependencies = [ "scale-info", "serde", "serde_json", + "simple-mermaid 0.1.0 (git+https://github.com/kianenigma/simple-mermaid.git?branch=main)", "sp-api", "sp-application-crypto", "sp-arithmetic", @@ -18293,26 +18339,6 @@ dependencies = [ "sc-cli", ] -[[package]] -name = "substrate" -version = "1.0.0" -dependencies = [ - "frame-support", - "sc-chain-spec", - "sc-cli", - "sc-consensus-aura", - "sc-consensus-babe", - "sc-consensus-beefy", - "sc-consensus-grandpa", - "sc-consensus-manual-seal", - "sc-consensus-pow", - "sc-service", - "simple-mermaid", - "sp-runtime", - "staging-chain-spec-builder", - "subkey", -] - [[package]] name = "substrate-bip39" version = "0.4.4" diff --git a/Cargo.toml b/Cargo.toml index ac474ae56869..a295aca819cc 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -8,6 +8,7 @@ license = "GPL-3.0-only" resolver = "2" members = [ + "developer-hub", "bridges/bin/runtime-common", "bridges/modules/grandpa", "bridges/modules/messages", @@ -186,7 +187,6 @@ members = [ "polkadot/xcm/xcm-simulator", "polkadot/xcm/xcm-simulator/example", "polkadot/xcm/xcm-simulator/fuzzer", - "substrate", "substrate/bin/minimal/node", "substrate/bin/minimal/runtime", "substrate/bin/node-template/node", @@ -476,8 +476,7 @@ default-members = [ "polkadot", "substrate/bin/node/cli" ] panic = "unwind" opt-level = 3 -# make sure dev builds with backtrace do -# not slow us down +# make sure dev builds with backtrace do not slow us down [profile.dev.package.backtrace] inherits = "release" diff --git a/cumulus/parachain-template/runtime/src/lib.rs b/cumulus/parachain-template/runtime/src/lib.rs index 7a064e227d4c..be76855c05b1 100644 --- a/cumulus/parachain-template/runtime/src/lib.rs +++ b/cumulus/parachain-template/runtime/src/lib.rs @@ -513,8 +513,7 @@ impl pallet_parachain_template::Config for Runtime { // Create the runtime by composing the FRAME pallets that were previously configured. construct_runtime!( - pub enum Runtime - { + pub struct Runtime { // System support stuff. System: frame_system = 0, ParachainSystem: cumulus_pallet_parachain_system = 1, diff --git a/developer-hub/Cargo.toml b/developer-hub/Cargo.toml new file mode 100644 index 000000000000..c19d5480ec47 --- /dev/null +++ b/developer-hub/Cargo.toml @@ -0,0 +1,66 @@ +[package] +name = "developer-hub" +description = "The one stop shop for developers of the polakdot-sdk" +license = "GPL-3.0-or-later WITH Classpath-exception-2.0" +homepage = "paritytech.github.io" +repository.workspace = true +authors.workspace = true +edition.workspace = true +# This crate is not publish-able to crates.io for now because of docify. +publish = false +version = "0.0.1" + +[dependencies] +# Needed for all FRAME-based code +parity-scale-codec = { version = "3.0.0", default-features = false } +scale-info = { version = "2.6.0", default-features = false } +frame = { path = "../substrate/frame", features = ["runtime", "experimental"] } +pallet-examples = { path = "../substrate/frame/examples" } +pallet-default-config-example = { path = "../substrate/frame/examples/default-config" } + +# How we build docs in rust-docs +simple-mermaid = { git = "https://github.com/kianenigma/simple-mermaid.git", branch = "main" } +docify = "0.2.6" + +# Polkadot SDK deps, typically all should only be scope such that we can link to their doc item. +node-cli = { package = "staging-node-cli", path = "../substrate/bin/node/cli" } +kitchensink-runtime = { path = "../substrate/bin/node/runtime" } +chain-spec-builder = { package = "staging-chain-spec-builder", path = "../substrate/bin/utils/chain-spec-builder" } +subkey = { path = "../substrate/bin/utils/subkey" } + +# Substrate +sc-network = { path = "../substrate/client/network" } +sc-rpc-api = { path = "../substrate/client/rpc-api" } +sc-rpc = { path = "../substrate/client/rpc" } +sc-client-db = { path = "../substrate/client/db" } +sc-cli = { path = "../substrate/client/cli" } +sc-consensus-aura = { path = "../substrate/client/consensus/aura" } +sc-consensus-babe = { path = "../substrate/client/consensus/babe" } +sc-consensus-grandpa = { path = "../substrate/client/consensus/grandpa" } +sc-consensus-beefy = { path = "../substrate/client/consensus/beefy" } +sc-consensus-manual-seal = { path = "../substrate/client/consensus/manual-seal" } +sc-consensus-pow = { path = "../substrate/client/consensus/pow" } +substrate-wasm-builder = { path = "../substrate/utils/wasm-builder" } + +# Cumulus +cumulus-pallet-aura-ext = { path = "../cumulus/pallets/aura-ext" } +cumulus-pallet-parachain-system = { path = "../cumulus/pallets/parachain-system", features = [ + "parameterized-consensus-hook", +] } +parachain-info = { package = "staging-parachain-info", path = "../cumulus/parachains/pallets/parachain-info" } +pallet-aura = { path = "../substrate/frame/aura", default-features = false } +pallet-timestamp = { path = "../substrate/frame/timestamp" } + +# Primitives +sp-io = { path = "../substrate/primitives/io" } +sp-api = { path = "../substrate/primitives/api" } +sp-core = { path = "../substrate/primitives/core" } +sp-keyring = { path = "../substrate/primitives/keyring" } +sp-runtime = { path = "../substrate/primitives/runtime" } + +[dev-dependencies] +parity-scale-codec = "3.6.5" +scale-info = "2.9.0" + +[features] +experimental = [ "pallet-aura/experimental" ] diff --git a/developer-hub/headers/toc.html b/developer-hub/headers/toc.html new file mode 100644 index 000000000000..d4d017eb207a --- /dev/null +++ b/developer-hub/headers/toc.html @@ -0,0 +1,54 @@ + + diff --git a/developer-hub/src/guides/changing_consensus.rs b/developer-hub/src/guides/changing_consensus.rs new file mode 100644 index 000000000000..7ba742f10723 --- /dev/null +++ b/developer-hub/src/guides/changing_consensus.rs @@ -0,0 +1 @@ +//! # Changing Consensus diff --git a/developer-hub/src/guides/cumulus_enabled_parachain.rs b/developer-hub/src/guides/cumulus_enabled_parachain.rs new file mode 100644 index 000000000000..fafd97feb829 --- /dev/null +++ b/developer-hub/src/guides/cumulus_enabled_parachain.rs @@ -0,0 +1 @@ +//! # Cumulus Enabled Parachain diff --git a/developer-hub/src/guides/mod.rs b/developer-hub/src/guides/mod.rs new file mode 100644 index 000000000000..21de220a3b8a --- /dev/null +++ b/developer-hub/src/guides/mod.rs @@ -0,0 +1,25 @@ +//! # Polkadot Developer Hub Guides +//! +//! This crate contains a collection of guides that are foundational to the developers of +//! Polkadot SDK. They common user-journeys that are traversed in the Polkadot ecosystem. + +/// Write your first simple pallet, learning the most most basic features of FRAME along the way. +pub mod your_first_pallet; + +/// Writing your first real [runtime](`crate::reference_docs::wasm_meta_protocol`), and successfully +/// compiling it to [WASM](crate::polkadot_sdk::substrate#wasm-build). +pub mod your_first_runtime; + +/// Running the given runtime with a node. No specific consensus mechanism is used at this stage. +pub mod your_first_node; + +/// How to change the consensus engine of both the node and the runtime. +pub mod changing_consensus; + +/// How to enhance a given runtime and node to be cumulus-enabled, run it as a parachain and connect +/// it to a relay-chain. +pub mod cumulus_enabled_parachain; + +/// How to make a given runtime XCM-enabled, capable of sending messages (`Transact`) between itself +/// and the relay chain to which it is connected. +pub mod xcm_enabled_parachain; diff --git a/developer-hub/src/guides/xcm_enabled_parachain.rs b/developer-hub/src/guides/xcm_enabled_parachain.rs new file mode 100644 index 000000000000..4518cab93421 --- /dev/null +++ b/developer-hub/src/guides/xcm_enabled_parachain.rs @@ -0,0 +1 @@ +//! # XCM Enabled Parachain diff --git a/developer-hub/src/guides/your_first_node.rs b/developer-hub/src/guides/your_first_node.rs new file mode 100644 index 000000000000..d12349c99063 --- /dev/null +++ b/developer-hub/src/guides/your_first_node.rs @@ -0,0 +1 @@ +//! # Your first Node diff --git a/developer-hub/src/guides/your_first_pallet/mod.rs b/developer-hub/src/guides/your_first_pallet/mod.rs new file mode 100644 index 000000000000..0d2cc26d7576 --- /dev/null +++ b/developer-hub/src/guides/your_first_pallet/mod.rs @@ -0,0 +1,738 @@ +//! # Currency Pallet +//! +//! By the end of this guide, you will write a small FRAME pallet (see +//! [`crate::polkadot_sdk::frame_runtime`]) that is capable of handling a simple crypto-currency. +//! This pallet will: +//! +//! 1. Allow anyone to mint new tokens into accounts (which is obviously not a great idea for a real +//! system). +//! 2. Allow any user that owns tokens to transfer them to others. +//! 3. Track the total issuance of all tokens at all times. +//! +//! > This guide will build a currency pallet from scratch using only the lowest primitives of +//! > FRAME, and is mainly intended for education, not *applicability*. For example, almost all +//! > FRAME-based runtimes use various techniques to re-use a currency pallet instead of writing +//! > one. Further advance FRAME related topics are discussed in [`crate::reference_docs`]. +//! +//! ## Topics Covered +//! +//! The following FRAME topics are covered in this guide. See the documentation of the +//! associated items to know more. +//! +//! - [Storage](frame::pallet_macros::storage) +//! - [Call](frame::pallet_macros::call) +//! - [Event](frame::pallet_macros::event) +//! - [Error](frame::pallet_macros::error) +//! - Basics of testing a pallet +//! - [Constructing a runtime](frame::runtime::prelude::construct_runtime) +//! +//! ## Writing Your First Pallet +//! +//! You should have studied the following modules as a prelude to this guide: +//! +//! - [`crate::reference_docs::blockchain_state_machines`] +//! - [`crate::reference_docs::trait_based_programming`] +//! - [`crate::polkadot_sdk::frame_runtime`] +//! +//! ### Shell Pallet +//! +//! Consider the following as a "shell pallet". We continue building the rest of this pallet based +//! on this template. +//! +//! [`pallet::config`](frame::pallet_macros::config) and +//! [`pallet::pallet`](frame::pallet_macros::pallet) are both mandatory parts of any pallet. Refer +//! to the documentation of each to get an overview of what they do. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", shell_pallet)] +//! +//! ### Storage +//! +//! First, we will need to create two onchain storage declarations. +//! +//! One should be a mapping from account-ids to a balance type, and one value that is the total +//! issuance. +// +// For the rest of this guide, we will opt for a balance type of u128. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", Balance)] +//! +//! The definition of these two storage items, based on [`frame::pallet_macros::storage`] details, +//! is as follows: +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", TotalIssuance)] +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", Balances)] +//! +//! ### Dispatchables +//! +//! Next, we will define the dispatchable functions. As per [`frame::pallet_macros::call`], these +//! will be defined as normal `fn`s attached to `struct Pallet`. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", impl_pallet)] +//! +//! The logic of the functions is self-explanatory. Instead, we will focus on the FRAME-related +//! details: +//! +//! - Where do `T::AccountId` and `T::RuntimeOrigin` come from? These are both defined in +//! [`frame::prelude::frame_system::Config`], therefore we can access them in `T`. +//! - What is `ensure_signed`, and what does it do with the aforementioned `T::RuntimeOrigin`? This +//! is outside the scope of this guide, and you can learn more about it in the origin reference +//! document ([`crate::reference_docs::frame_origin`]). For now, you should only know the +//! signature of the function: it takes a generic `T::RuntimeOrigin` and returns a +//! `Result`. So by the end of this function call, we know that this dispatchable +//! was signed by `who`. +#![doc = docify::embed!("../substrate/frame/system/src/lib.rs", ensure_signed)] +//! +//! +//! - Where does `mutate`, `get` and `insert` and other storage APIs come from? All of them are +//! explained in the corresponding `type`, for example, for `Balances::::insert`, you can look +//! into [`frame::prelude::StorageMap::insert`]. +//! +//! - The return type of all dispatchable functions is [`frame::prelude::DispatchResult`]: +#![doc = docify::embed!("../substrate/frame/support/src/dispatch.rs", DispatchResult)] +//! +//! Which is more or less a normal Rust `Result`, with a custom [`frame::prelude::DispatchError`] as +//! the `Err` variant. We won't cover this error in detail here, but importantly you should know +//! that there is an `impl From<&'static string> for DispatchError` provided (see +//! [here](`frame::prelude::DispatchError#impl-From<%26'static+str>-for-DispatchError`)). Therefore, +//! we can use basic string literals as our error type and `.into()` them into `DispatchError`. +//! +//! - Why are all `get` and `mutate` functions returning an `Option`? This is the default behavior +//! of FRAME storage APIs. You can learn more about how to override this by looking into +//! [`frame::pallet_macros::storage`], and +//! [`frame::prelude::ValueQuery`]/[`frame::prelude::OptionQuery`] +//! +//! ### Improving Errors +//! +//! How we handle error in the above snippets is fairly rudimentary. Let's look at how this can be +//! improved. First, we can use [`frame::prelude::ensure`] to express the error slightly better. +//! This macro will call `.into()` under the hood. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", transfer_better)] +//! +//! Moreover, you will learn in the [Safe Defensive Programming +//! section](crate::reference_docs::safe_defensive_programming) that it is always recommended to use +//! safe arithmetic operations in your runtime. By using [`frame::traits::CheckedSub`], we can not +//! only take a step in that direction, but also improve the error handing and make it slightly more +//! ergonomic. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", transfer_better_checked)] +//! +//! This is more or less all the logic that there is this basic currency pallet! +//! +//! ### Your First (Test) Runtime +//! +//! Next, we create a "test runtime" in order to test our pallet. Recall from +//! [`crate::polkadot_sdk::frame_runtime`] that a runtime is a collection of pallets, expressed +//! through [`frame::runtime::prelude::construct_runtime`]. All runtimes also have to include +//! [`frame::prelude::frame_system`]. So we expect to see a runtime with two pallet, `frame_system` +//! and the one we just wrote. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", runtime)] +//! +//! > [`frame::pallet_macros::derive_impl`] is a FRAME feature that enables developers to have +//! > defaults for associated types. +//! +//! Recall that within our pallet, (almost) all blocks of code are generic over ``. And, +//! because `trait Config: frame_system::Config`, we can get access to all items in `Config` (or +//! `frame_system::Config`) using `T::NameOfItem`. This is all within the boundaries of how Rust +//! traits and generics work. If unfamiliar with this pattern, read +//! [`crate::reference_docs::trait_based_programming`] before going further. +//! +//! Crucially, a typical FRAME runtime contains a `struct Runtime`. The main role of this `struct` +//! is to implement the `trait Config` of all pallets. That is, anywhere within your pallet code +//! where you see `` (read: *"some type `T` that implements `Config`"*), in the runtime, +//! it can be replaced with ``, because `Runtime` implements `Config` of all pallets, as we +//! see above. +//! +//! Another way to think about this is that within a pallet, a lot of types are "unknown" and, we +//! only know that they will be provided at some later point. For example, when you write +//! `T::AccountId` (which is short for `::AccountId`) in your pallet, +//! you are in fact saying "*Some type `AccountId` that will be known later*". That "later" is in +//! fact when you specify these types when you implement all `Config` traits for `Runtime`. +//! +//! As you see above, `frame_system::Config` is setting the `AccountId` to `u64`. Of course, a real +//! runtime will not use this type, and instead reside to a proper type like a 32-byte standard +//! public key. This is a HUGE benefit that FRAME developers can tap into: through the framework +//! being so generic, different types can always be customized to simple things when needed. +//! +//! > Imagine how hard it would have been if all tests had to use a real 32-byte account id, as +//! > opposed to just a u64 number 🙈. +//! +//! ### Your First Test +//! +//! The above is all you need to execute the dispatchables of your pallet. The last thing you need +//! to learn is that all of your pallet testing code should be wrapped in +//! [`frame::testing_prelude::TestState`]. This is a type that provides access to an in-memory state +//! to be used in our tests. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", first_test)] +//! +//! In the first test, we simply assert that there is no total issuance, and no balance associated +//! with account `1`. Then, we mint some balance into `1`, and re-check. +//! +//! As noted above, the `T::AccountId` is now `u64`. Moreover, `Runtime` is replacing ``. +//! This is why for example you see `Balances::::get(..)`. Finally, notice that the +//! dispatchables are simply functions that can be called on top of the `Pallet` struct. +//! +//! TODO: hard to explain exactly `RuntimeOrigin::signed(1)` at this point. +//! +//! Congratulations! You have written your first pallet and tested it! Next, we learn a few optional +//! steps to improve our pallet. +//! +//! ## Improving the Currency Pallet +//! +//! ### Better Test Setup +//! +//! Idiomatic FRAME pallets often use Builder pattern to define their initial state. +//! +//! > The Polkadot Blockchain Academy's Rust entrance exam has a +//! > [section](https://github.com/Polkadot-Blockchain-Academy/pba-qualifier-exam/blob/main/src/m_builder.rs) +//! > on this that you can use to learn the Builder Pattern. +//! +//! Let's see how we can implement a better test setup using this pattern. First, we define a +//! `struct StateBuilder`. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", StateBuilder)] +//! +//! This struct is meant to contain the same list of accounts and balances that we want to have at +//! the beginning of each block. We hardcoded this to `let accounts = vec![(1, 100), (2, 100)];` so +//! far. Then, if desired, we attach a default value for this struct. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", default_state_builder)] +//! +//! Like any other builder pattern, we attach functions to the type to mutate its internal +//! properties. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", impl_state_builder_add)] +//! +//! Finally --the useful part-- we write our own custom `build_and_execute` function on +//! this type. This function will do multiple things: +//! +//! 1. It would consume `self` to produce our `TestState` based on the properties that we attached +//! to `self`. +//! 2. It would execute any test function that we pass in as closure. +//! 3. A nifty trick, this allows our test setup to have some code that is executed both before and +//! after each test. For example, in this test, we do some additional checking about the +//! correctness of the `TotalIssuance`. We leave it up to you as an exercise to learn why the +//! assertion should always hold, and how it is checked. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", impl_state_builder_build)] +//! +//! We can write tests that specifically check the initial state, and making sure our `StateBuilder` +//! is working exactly as intended. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", state_builder_works)] +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", state_builder_add_balance)] +//! +//! ### More Tests +//! +//! Now that we have a more ergonomic test setup, let's see how a well written test for transfer and +//! mint would look like. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", transfer_works)] +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", mint_works)] +//! +//! It is always a good idea to build a mental model where you write *at least* one test for each +//! "success path" of a dispatchable, and one test for each "failure path", such as: +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", transfer_from_non_existent_fails)] +//! +//! We leave it up to you to write a test that triggers to `InsufficientBalance` error. +//! +//! ### Event and Error +//! +//! Our pallet is mainly missing two parts that are common in most FRAME pallets: Events, and +//! Errors. First, let's understand what each is. +//! +//! - **Error**: The static string-based error scheme we used so far is good for readability, but it +//! has a few drawbacks. These string literals will bloat the final wasm blob, and are relatively +//! heavy to transmit and encode/decode. Moreover, it is easy to mistype them by one character. +//! FRAME errors are exactly a solution to maintain readability, whilst fixing the drawbacks +//! mentioned. In short, we use an enum to represent different variants of our error. These +//! variants are then mapped in an efficient way (using only `u8` indices) to +//! [`sp_runtime::DispatchError::Module`] Read more about this in [`frame::pallet_macros::error`]. +//! +//! - **Event**: Events are akin to the return type of dispatchables. They should represent what +//! happened at the end of a dispatch operation. Therefore, the convention is to use passive tense +//! for event names (eg. `SomethingHappened`). This allows other sub-systems or external parties +//! (eg. a light-node, a DApp) to listen to particular events happening, without needing to +//! re-execute the whole state transition function. +//! +//! TODO: both need to be improved a lot at the pallet-macro rust-doc level. Also my explanation +//! of event is probably not the best. +//! +//! With the explanation out of the way, let's see how these components can be added. Both follow a +//! fairly familiar syntax: normal Rust enums, with an extra `#[frame::event/error]` attribute +//! attached. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", Event)] +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", Error)] +//! +//! One slightly custom part of this is the `#[pallet::generate_deposit(pub(super) fn +//! deposit_event)]` part. Without going into too much detail, in order for a pallet to emit events +//! to the rest of the system, it needs to do two things: +//! +//! 1. Declare a type in its `Config` that refers to the overarching event type of the runtime. In +//! short, by doing this, the pallet is expressing an important bound: `type RuntimeEvent: +//! From>`. Read: a `RuntimeEvent` exists, and it can be created from the local `enum +//! Event` of this pallet. This enables the pallet to convert its `Event` into `RuntimeEvent`, and +//! store it where needed. +//! +//! 2. But, doing this conversion and storing is too much to expect each pallet to define. FRAME +//! provides a default way of storing events, and this is what `pallet::generate_deposit` is doing. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", config_v2)] +//! +//! > These `Runtime*` types are better explained in +//! > [`crate::reference_docs::frame_composite_enums`]. +//! +//! Then, we can rewrite the `transfer` dispatchable as such: +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", transfer_v2)] +//! +//! Then, notice how now we would need to provide this `type RuntimeEvent` in our test runtime +//! setup. +#![doc = docify::embed!("./src/guides/your_first_pallet/mod.rs", runtime_v2)] +//! +//! In this snippet, the actual `RuntimeEvent` type (right hand side of `type RuntimeEvent = +//! RuntimeEvent`) is generated by `construct_runtime`. An interesting way to inspect this type is +//! to see its definition in rust-docs: +//! [`crate::guides::your_first_pallet::pallet_v2::tests::runtime_v2::RuntimeEvent`]. +//! +//! +//! +//! ## What Next? +//! +//! The following topics where used in this guide, but not covered in depth. It is suggested to +//! study them subsequently: +//! +//! - [`crate::reference_docs::safe_defensive_programming`]. +//! - [`crate::reference_docs::frame_origin`]. +//! - [`crate::reference_docs::frame_composite_enums`]. +//! - The pallet we wrote in this guide was using `dev_mode`, learn more in +//! [`frame::pallet_macros::config`]. +//! - Learn more about the individual pallet items/macros, such as event and errors and call, in +//! [`frame::pallet_macros`]. + +#[docify::export] +#[frame::pallet(dev_mode)] +pub mod shell_pallet { + use frame::prelude::*; + + #[pallet::config] + pub trait Config: frame_system::Config {} + + #[pallet::pallet] + pub struct Pallet(_); +} + +#[frame::pallet(dev_mode)] +pub mod pallet { + use frame::prelude::*; + + #[docify::export] + pub type Balance = u128; + + #[pallet::config] + pub trait Config: frame_system::Config {} + + #[pallet::pallet] + pub struct Pallet(_); + + #[docify::export] + /// Single storage item, of type `Balance`. + #[pallet::storage] + pub type TotalIssuance = StorageValue<_, Balance>; + + #[docify::export] + /// A mapping from `T::AccountId` to `Balance` + #[pallet::storage] + pub type Balances = StorageMap<_, _, T::AccountId, Balance>; + + #[docify::export(impl_pallet)] + #[pallet::call] + impl Pallet { + /// An unsafe mint that can be called by anyone. Not a great idea. + pub fn mint_unsafe( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + // ensure that this is a signed account, but we don't really check `_anyone`. + let _anyone = ensure_signed(origin)?; + + // update the balances map. Notice how all `` remains as ``. + Balances::::mutate(dest, |b| *b = Some(b.unwrap_or(0) + amount)); + // update total issuance. + TotalIssuance::::mutate(|t| *t = Some(t.unwrap_or(0) + amount)); + + Ok(()) + } + + /// Transfer `amount` from `origin` to `dest`. + pub fn transfer( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + let sender = ensure_signed(origin)?; + + // ensure sender has enough balance, and if so, calculate what is left after `amount`. + let sender_balance = Balances::::get(&sender).ok_or("NonExistentAccount")?; + if sender_balance < amount { + return Err("InsufficientBalance".into()) + } + let reminder = sender_balance - amount; + + // update sender and dest balances. + Balances::::mutate(dest, |b| *b = Some(b.unwrap_or(0) + amount)); + Balances::::insert(&sender, reminder); + + Ok(()) + } + } + + #[allow(unused)] + impl Pallet { + #[docify::export] + pub fn transfer_better( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + let sender = ensure_signed(origin)?; + + let sender_balance = Balances::::get(&sender).ok_or("NonExistentAccount")?; + ensure!(sender_balance >= amount, "InsufficientBalance"); + let reminder = sender_balance - amount; + + // .. snip + Ok(()) + } + + #[docify::export] + /// Transfer `amount` from `origin` to `dest`. + pub fn transfer_better_checked( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + let sender = ensure_signed(origin)?; + + let sender_balance = Balances::::get(&sender).ok_or("NonExistentAccount")?; + let reminder = sender_balance.checked_sub(amount).ok_or("InsufficientBalance")?; + + // .. snip + Ok(()) + } + } + + #[cfg(any(test, doc))] + pub(crate) mod tests { + use crate::guides::your_first_pallet::pallet::*; + use frame::testing_prelude::*; + + #[docify::export] + mod runtime { + use super::*; + // we need to reference our `mod pallet` as an identifier to pass to + // `construct_runtime`. + use crate::guides::your_first_pallet::pallet as pallet_currency; + + construct_runtime!( + pub struct Runtime { + // ---^^^^^^ This is where `struct Runtime` is defined. + System: frame_system, + Currency: pallet_currency, + } + ); + + #[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)] + impl frame_system::Config for Runtime { + type Block = MockBlock; + // within pallet we just said `::AccountId`, now we + // finally specified it. + type AccountId = u64; + } + + // our simple pallet has nothing to be configured. + impl pallet_currency::Config for Runtime {} + } + + pub(crate) use runtime::*; + + #[allow(unused)] + #[docify::export] + fn new_test_state_basic() -> TestState { + let mut state = TestState::new_empty(); + let accounts = vec![(1, 100), (2, 100)]; + state.execute_with(|| { + for (who, amount) in &accounts { + Balances::::insert(who, amount); + TotalIssuance::::mutate(|b| *b = Some(b.unwrap_or(0) + amount)); + } + }); + + state + } + + #[docify::export] + pub(crate) struct StateBuilder { + balances: Vec<(::AccountId, Balance)>, + } + + #[docify::export(default_state_builder)] + impl Default for StateBuilder { + fn default() -> Self { + Self { balances: vec![(1, 100), (2, 100)] } + } + } + + #[docify::export(impl_state_builder_add)] + impl StateBuilder { + fn add_balance( + mut self, + who: ::AccountId, + amount: Balance, + ) -> Self { + self.balances.push((who, amount)); + self + } + } + + #[docify::export(impl_state_builder_build)] + impl StateBuilder { + pub(crate) fn build_and_execute(self, test: impl FnOnce() -> ()) { + let mut ext = TestState::new_empty(); + ext.execute_with(|| { + for (who, amount) in &self.balances { + Balances::::insert(who, amount); + TotalIssuance::::mutate(|b| *b = Some(b.unwrap_or(0) + amount)); + } + }); + + ext.execute_with(test); + + // assertions that must always hold + ext.execute_with(|| { + assert_eq!( + Balances::::iter().map(|(_, x)| x).sum::(), + TotalIssuance::::get().unwrap_or_default() + ); + }) + } + } + + #[docify::export] + #[test] + fn first_test() { + TestState::new_empty().execute_with(|| { + // We expect account 1 to have no funds. + assert_eq!(Balances::::get(&1), None); + assert_eq!(TotalIssuance::::get(), None); + + // mint some funds into 1 + assert_ok!(Pallet::::mint_unsafe(RuntimeOrigin::signed(1), 1, 100)); + + // re-check the above + assert_eq!(Balances::::get(&1), Some(100)); + assert_eq!(TotalIssuance::::get(), Some(100)); + }) + } + + #[docify::export] + #[test] + fn state_builder_works() { + StateBuilder::default().build_and_execute(|| { + assert_eq!(Balances::::get(&1), Some(100)); + assert_eq!(Balances::::get(&2), Some(100)); + assert_eq!(Balances::::get(&3), None); + assert_eq!(TotalIssuance::::get(), Some(200)); + }); + } + + #[docify::export] + #[test] + fn state_builder_add_balance() { + StateBuilder::default().add_balance(3, 42).build_and_execute(|| { + assert_eq!(Balances::::get(&3), Some(42)); + assert_eq!(TotalIssuance::::get(), Some(242)); + }) + } + + #[test] + #[should_panic] + fn state_builder_duplicate_genesis_fails() { + StateBuilder::default() + .add_balance(3, 42) + .add_balance(3, 43) + .build_and_execute(|| { + assert_eq!(Balances::::get(&3), None); + assert_eq!(TotalIssuance::::get(), Some(242)); + }) + } + + #[docify::export] + #[test] + fn mint_works() { + StateBuilder::default().build_and_execute(|| { + // given the initial state, when: + assert_ok!(Pallet::::mint_unsafe(RuntimeOrigin::signed(1), 2, 100)); + + // then: + assert_eq!(Balances::::get(&2), Some(200)); + assert_eq!(TotalIssuance::::get(), Some(300)); + + // given: + assert_ok!(Pallet::::mint_unsafe(RuntimeOrigin::signed(1), 3, 100)); + + // then: + assert_eq!(Balances::::get(&3), Some(100)); + assert_eq!(TotalIssuance::::get(), Some(400)); + }); + } + + #[docify::export] + #[test] + fn transfer_works() { + StateBuilder::default().build_and_execute(|| { + // given the the initial state, when: + assert_ok!(Pallet::::transfer(RuntimeOrigin::signed(1), 2, 50)); + + // then: + assert_eq!(Balances::::get(&1), Some(50)); + assert_eq!(Balances::::get(&2), Some(150)); + assert_eq!(TotalIssuance::::get(), Some(200)); + + // when: + assert_ok!(Pallet::::transfer(RuntimeOrigin::signed(2), 1, 50)); + + // then: + assert_eq!(Balances::::get(&1), Some(100)); + assert_eq!(Balances::::get(&2), Some(100)); + assert_eq!(TotalIssuance::::get(), Some(200)); + }); + } + + #[docify::export] + #[test] + fn transfer_from_non_existent_fails() { + StateBuilder::default().build_and_execute(|| { + // given the the initial state, when: + assert_err!( + Pallet::::transfer(RuntimeOrigin::signed(3), 1, 10), + "NonExistentAccount" + ); + + // then nothing has changed. + assert_eq!(Balances::::get(&1), Some(100)); + assert_eq!(Balances::::get(&2), Some(100)); + assert_eq!(Balances::::get(&3), None); + assert_eq!(TotalIssuance::::get(), Some(200)); + }); + } + } +} + +#[frame::pallet(dev_mode)] +pub mod pallet_v2 { + use super::pallet::Balance; + use frame::prelude::*; + + #[docify::export(config_v2)] + #[pallet::config] + pub trait Config: frame_system::Config { + /// The overarching event type of the runtime. + type RuntimeEvent: From> + + IsType<::RuntimeEvent> + + TryInto>; + } + + #[pallet::pallet] + pub struct Pallet(_); + + #[pallet::storage] + pub type Balances = StorageMap<_, _, T::AccountId, Balance>; + + #[pallet::storage] + pub type TotalIssuance = StorageValue<_, Balance>; + + #[docify::export] + #[pallet::error] + pub enum Error { + /// Account does not exist. + NonExistentAccount, + /// Account does not have enough balance. + InsufficientBalance, + } + + #[docify::export] + #[pallet::event] + #[pallet::generate_deposit(pub(super) fn deposit_event)] + pub enum Event { + /// A transfer succeeded. + Transferred { from: T::AccountId, to: T::AccountId, amount: Balance }, + } + + #[pallet::call] + impl Pallet { + #[docify::export(transfer_v2)] + pub fn transfer( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + let sender = ensure_signed(origin)?; + + // ensure sender has enough balance, and if so, calculate what is left after `amount`. + let sender_balance = + Balances::::get(&sender).ok_or(Error::::NonExistentAccount)?; + let reminder = + sender_balance.checked_sub(amount).ok_or(Error::::InsufficientBalance)?; + + Balances::::mutate(&dest, |b| *b = Some(b.unwrap_or(0) + amount)); + Balances::::insert(&sender, reminder); + + Self::deposit_event(Event::::Transferred { from: sender, to: dest, amount }); + + Ok(()) + } + } + + #[cfg(any(test, doc))] + pub mod tests { + use super::{super::pallet::tests::StateBuilder, *}; + use frame::testing_prelude::*; + + #[docify::export] + pub mod runtime_v2 { + use super::*; + use crate::guides::your_first_pallet::pallet_v2 as pallet_currency; + + construct_runtime!( + pub struct Runtime { + System: frame_system, + Currency: pallet_currency, + } + ); + + #[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)] + impl frame_system::Config for Runtime { + type Block = MockBlock; + type AccountId = u64; + } + + impl pallet_currency::Config for Runtime { + type RuntimeEvent = RuntimeEvent; + } + } + + pub(crate) use runtime_v2::*; + + #[docify::export(transfer_works_v2)] + #[test] + fn transfer_works() { + StateBuilder::default().build_and_execute(|| { + // skip the genesis block, as events are not deposited there and we need them for + // the final assertion. + System::set_block_number(1); + + // given the the initial state, when: + assert_ok!(Pallet::::transfer(RuntimeOrigin::signed(1), 2, 50)); + + // then: + assert_eq!(Balances::::get(&1), Some(50)); + assert_eq!(Balances::::get(&2), Some(150)); + assert_eq!(TotalIssuance::::get(), Some(200)); + + // now we can also check that an event has been deposited: + assert_eq!( + System::read_events_for_pallet::>(), + vec![Event::Transferred { from: 1, to: 2, amount: 50 }] + ); + }); + } + } +} diff --git a/developer-hub/src/guides/your_first_pallet/with_event.rs b/developer-hub/src/guides/your_first_pallet/with_event.rs new file mode 100644 index 000000000000..a65aac324f07 --- /dev/null +++ b/developer-hub/src/guides/your_first_pallet/with_event.rs @@ -0,0 +1,101 @@ +#[frame::pallet(dev_mode)] +pub mod pallet { + use frame::prelude::*; + + #[docify::export] + pub type Balance = u128; + + #[pallet::config] + pub trait Config: frame_system::Config {} + + #[pallet::pallet] + pub struct Pallet(_); + + #[docify::export] + /// Single storage item, of type `Balance`. + #[pallet::storage] + pub type TotalIssuance = StorageValue<_, Balance>; + + #[docify::export] + /// A mapping from `T::AccountId` to `Balance` + #[pallet::storage] + pub type Balances = StorageMap<_, _, T::AccountId, Balance>; + + #[docify::export(impl_pallet)] + #[pallet::call] + impl Pallet { + /// An unsafe mint that can be called by anyone. Not a great idea. + pub fn mint_unsafe( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + // ensure that this is a signed account, but we don't really check `_anyone`. + let _anyone = ensure_signed(origin)?; + + // update the balances map. Notice how all `` remains as ``. + Balances::::mutate(dest, |b| *b = Some(b.unwrap_or(0) + amount)); + // update total issuance. + TotalIssuance::::mutate(|t| *t = Some(t.unwrap_or(0) + amount)); + + Ok(()) + } + + /// Transfer `amount` from `origin` to `dest`. + pub fn transfer( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + let sender = ensure_signed(origin)?; + + // ensure sender has enough balance, and if so, calculate what is left after `amount`. + let sender_balance = Balances::::get(&sender).ok_or("NonExistentAccount")?; + if sender_balance < amount { + return Err("NotEnoughBalance".into()) + } + let reminder = sender_balance - amount; + + // update sender and dest balances. + Balances::::mutate(dest, |b| *b = Some(b.unwrap_or(0) + amount)); + Balances::::insert(&sender, reminder); + + Ok(()) + } + } + + #[allow(unused)] + impl Pallet { + #[docify::export] + pub fn transfer_better( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + let sender = ensure_signed(origin)?; + + let sender_balance = Balances::::get(&sender).ok_or("NonExistentAccount")?; + ensure!(sender_balance >= amount, "NotEnoughBalance"); + let reminder = sender_balance - amount; + + // .. snip + Ok(()) + } + + #[docify::export] + /// Transfer `amount` from `origin` to `dest`. + pub fn transfer_better_checked( + origin: T::RuntimeOrigin, + dest: T::AccountId, + amount: Balance, + ) -> DispatchResult { + let sender = ensure_signed(origin)?; + + let sender_balance = Balances::::get(&sender).ok_or("NonExistentAccount")?; + let reminder = sender_balance.checked_sub(amount).ok_or("NotEnoughBalance")?; + + // .. snip + Ok(()) + } + } +} diff --git a/developer-hub/src/guides/your_first_runtime.rs b/developer-hub/src/guides/your_first_runtime.rs new file mode 100644 index 000000000000..3e02ef1b1b28 --- /dev/null +++ b/developer-hub/src/guides/your_first_runtime.rs @@ -0,0 +1 @@ +//! # Your first Runtime diff --git a/developer-hub/src/lib.rs b/developer-hub/src/lib.rs new file mode 100644 index 000000000000..eb71865ca440 --- /dev/null +++ b/developer-hub/src/lib.rs @@ -0,0 +1,43 @@ +//! # Developer Hub +//! +//! The Polkadot SDK Developer Hub. +//! +//! This crate is a *minimal*, but *always-accurate* source of information for those wishing to +//! build on the Polkadot SDK. +//! +//! > **Work in Progress**: This crate is under heavy development. Expect content to be moved and +//! > changed. Do not use links to this crate yet. See [`meta_contributing`] for more information. +//! +//! ## Getting Started +//! +//! We suggest the following reading sequence: +//! +//! - Start by learning about the the [`polkadot_sdk`], its structure and context. +//! - Then, head over the [`guides`]. This modules contains in-depth guides about the most important +//! user-journeys of the Polkadot SDK. +//! - Whilst reading the guides, you might find back-links to [`crate::reference_docs`]. +//! - Finally, is the parent website of this crate that contains the +//! list of further tools related to the Polkadot SDK. +//! +//! ## Information Architecture +//! +//! This section paints a picture over the high-level information architecture of this crate. +#![doc = simple_mermaid::mermaid!("../../docs/mermaid/IA.mmd")] +#![allow(rustdoc::invalid_html_tags)] // TODO: remove later. https://github.com/paritytech/polkadot-sdk-docs/issues/65 +#![allow(rustdoc::bare_urls)] // TODO: remove later. https://github.com/paritytech/polkadot-sdk-docs/issues/65 +#![warn(rustdoc::broken_intra_doc_links)] +#![warn(rustdoc::private_intra_doc_links)] + +/// Meta information about this crate, how it is built, what principles dictates its evolution and +/// how one can contribute to it. +pub mod meta_contributing; + +/// In-depth guides about the most common components of the Polkadot SDK. They are slightly more +/// high level and broad than reference docs. +pub mod guides; +/// An introduction to the Polkadot SDK. Read this module to learn about the structure of the SDK, +/// the tools that are provided as a part of it, and to gain a high level understanding of each. +pub mod polkadot_sdk; +/// Reference documents covering in-depth topics across the Polkadot SDK. It is suggested to read +/// these on-demand, while you are going through the [`guides`] or other content. +pub mod reference_docs; diff --git a/developer-hub/src/meta_contributing.rs b/developer-hub/src/meta_contributing.rs new file mode 100644 index 000000000000..70a23f825835 --- /dev/null +++ b/developer-hub/src/meta_contributing.rs @@ -0,0 +1,146 @@ +//! # Contribution +//! +//! The following sections cover more detailed information about this crate and how it should be +//! maintained. +//! +//! ## Why Rust Docs? +//! +//! We acknowledge that blockchain based systems, particularly a cutting-edge one like Polkadot SDK +//! is a software artifact that is complex, and rapidly evolving. This makes the task of documenting +//! it externally extremely difficult, especially with regards to making sure it is up-to-date. +//! +//! Consequently, we argue that the best hedge against this is to move as much of the documentation +//! near the source code as possible. This would further incentivize developers to keep the +//! documentation up-to-date, as the overhead is reduced by making sure everything is in one +//! repository, and everything being in `.rs` files. +//! +//! > This is not say that a more visually appealing version of this crate (for example as an +//! > `md-book`) cannot exist, but it would be outside the scope of this crate. +//! +//! Moreover, we acknowledge that a major pain point has been not only outdated *concepts*, but also +//! *outdated code*. For this, we commit to making sure no code-snippet in this crate is left as +//! `///ignore` or `///no_compile`, making sure all code snippets are self-contained, compile-able, +//! and correct at every single revision of the entire repository. +//! +//! > This also allows us to have a clear versioning on the entire content of this crate. For every +//! commit of the Polkadot SDK, there would be one version of this crate that is guaranteed to be +//! correct. +//! +//! > To achieve this, we often use [`docify`](https://github.com/sam0x17/docify), a nifty invention +//! > of `@sam0x17`. +//! +//! Also see: . +//! +//! ## Scope +//! +//! The above would NOT be attainable if we don't acknowledge that the scope of this crate MUST be +//! limited, or else its maintenance burden would be infeasible or not worthwhile. In short, future +//! maintainers should always strive to keep the content of this repository as minimal as possible. +//! Some of the following principles are specifically there to be the guidance for this. +//! +//! ## Principles +//! +//! The following guidelines are meant to be the guiding torch of those who contribute to this +//! crate. +//! +//! 1. 🔺 Ground Up: Information should be layed out in the most ground-up fashion. The lowest level +//! (i.e. "ground") is Rust-docs. The highest level (i.e. "up") is "outside of this crate". In +//! between lies [`reference_docs`] and [`guides`], from low to high. The point of this principle +//! is to document as much of the information as possible in the lower level media, as it is +//! easier to maintain and more reachable. Then, use excessive linking to back-link when writing +//! in a more high level. +//! +//! > A prime example of this, the details of the FRAME storage APIs should NOT be explained in a +//! > high level tutorial. They should be explained in the rust-doc of the corresponding type or +//! > macro. +//! +//! 2. 🧘 Less is More: For reasons mentioned [above](#crate::why-rust-docs), the more concise this +//! crate is, the better. +//! 3. √ Don’t Repeat Yourself – DRY: A summary of the above two points. Authors should always +//! strive to avoid any duplicate information. Every concept should ideally be documented in +//! *ONE* place and one place only. This makes the task of maintaining topics significantly +//! easier. +//! +//! > A prime example of this, the list of CLI arguments of a particular binary should not be +//! > documented in multiple places across this crate. It should be only be documented in the +//! > corresponding crate (e.g. `sc_cli`). +//! +//! > Moreover, this means that as a contributor, **it is your responsibility to have a grasp over +//! > what topics are already covered in this crate, and how you can build on top of the information +//! > that they already pose, rather than repeating yourself**. +//! +//! For more details about documenting guidelines, see: +//! +//! +//! #### Example: Explaining `#[pallet::call]` +//! +//!
+//! +//! Let's consider the seemingly simple example of explaining to someone dead-simple code of a FRAME +//! call and see how we can use the above principles. +//! +//! +//! +//! ``` +//! #[frame::pallet(dev_mode)] +//! pub mod pallet { +//! # use frame::prelude::*; +//! # #[pallet::config] +//! # pub trait Config: frame_system::Config {} +//! # #[pallet::pallet] +//! # pub struct Pallet(_); +//! #[pallet::call] +//! impl Pallet { +//! pub fn a_simple_call(origin: OriginFor, data: u32) -> DispatchResult { +//! ensure!(data > 10, "SomeStaticString"); +//! todo!(); +//! } +//! } +//! } +//! ``` +//! +//! * Before even getting started, what is with all of this ``? We link to +//! [`crate::reference_docs::trait_based_programming`]. +//! * First, the name. Why is this called `pallet::call`? This goes back to `enum Call`, which is +//! explained in [`crate::reference_docs::frame_composite_enums`]. Build on top of this! +//! * Then, what is `origin`? Just an account id? [`crate::reference_docs::frame_origin`]. +//! * Then, what is `DispatchResult`? Why is this called *dispatch*? Probably something that can be +//! explained in the documentation of [`frame::prelude::DispatchResult`]. +//! * Why is `"SomeStaticString"` a valid error? Because there is implementation for it that you can +//! see [here](frame::prelude::DispatchError#impl-From<%26'static+str>-for-DispatchError). +//! +//! +//! All of these are examples of underlying information that a contributor should: +//! +//! 1. Try and create and they are going along. +//! 2. Back-link to if they already exist. +//! +//! Of course, all of this is not set in stone as a either/or rule. Sometimes, it is necessary to +//! rephrase a concept in a new context. +//! +//!
+//! +//! ## `docs.substrate.io` +//! +//! This crate is meant to gradually replace `docs.substrate.io`. As any content is added here, the +//! corresponding counter-part should be marked as deprecated, as described +//! [here](https://github.com/paritytech/polkadot-sdk-docs/issues/26). +//! +//! ## `crates.io` and Publishing +//! +//! As it stands now, this crate cannot be published to crates.io because of its use of +//! [workspace-level `docify`](https://github.com/sam0x17/docify/issues/22). For now, we accept this +//! compromise, but in the long term, we should work towards finding a way to maintain different +//! revisions of this crate. +//! +//! ## How to Build +//! +//! To build this crate properly, with with right HTML headers injected, run: +//! +//! ```no_compile +//! RUSTDOCFLAGS="--html-in-header $(pwd)/developer-hub/headers/toc.html" cargo doc -p developer-hub +//! ``` +//! +//! adding `--no-deps` would speed up the process while development. If even faster build time for +//! docs is needed, you can temporarily remove most of the substrate/cumulus dependencies that are +//! only used for linking purposes. diff --git a/developer-hub/src/polkadot_sdk/cumulus.rs b/developer-hub/src/polkadot_sdk/cumulus.rs new file mode 100644 index 000000000000..07a48c92d807 --- /dev/null +++ b/developer-hub/src/polkadot_sdk/cumulus.rs @@ -0,0 +1,130 @@ +//! # Cumulus +//! +//! Substrate provides a framework ([FRAME]) through which a blockchain node and runtime can easily +//! be created. Cumulus aims to extend the same approach to creation of Polkadot parachains. +//! +//! > Cumulus clouds are shaped sort of like dots; together they form a system that is intricate, +//! > beautiful and functional. +//! +//! ## Example: Runtime +//! +//! A Cumulus-based runtime is fairly similar to other [FRAME]-based runtimes. Most notably, the +//! following changes are applied to a normal FRAME-based runtime to make it a Cumulus-based +//! runtime: +//! +//! #### Cumulus Pallets +//! +//! A parachain runtime should use a number of pallets that are provided by Cumulus and Substrate. +//! Notably: +//! +//! - [`frame-system`](frame::prelude::frame_system), like all FRAME-based runtimes. +//! - [`cumulus_pallet_parachain_system`] +//! - [`parachain_info`] +#![doc = docify::embed!("./src/polkadot_sdk/cumulus.rs", system_pallets)] +//! +//! Given that all Cumulus-based runtimes use a simple Aura-based consensus mechanism, the following +//! pallets also need to be added: +//! +//! - [`pallet_timestamp`] +//! - [`pallet_aura`] +//! - [`cumulus_pallet_aura_ext`] +#![doc = docify::embed!("./src/polkadot_sdk/cumulus.rs", consensus_pallets)] +//! +//! +//! Finally, a separate macro, similar to +//! [`impl_runtime_api`](frame::runtime::prelude::impl_runtime_apis), which creates the default set +//! of runtime APIs, will generate the parachain runtime's validation runtime API, also known as +//! parachain validation function (PVF). Without this API, the relay chain is unable to validate +//! blocks produced by our parachain. +#![doc = docify::embed!("./src/polkadot_sdk/cumulus.rs", validate_block)] +//! +//! --- +//! +//! [FRAME]: crate::polkadot_sdk::frame_runtime + +#![deny(rustdoc::broken_intra_doc_links)] +#![deny(rustdoc::private_intra_doc_links)] + +#[cfg(test)] +mod tests { + mod runtime { + pub use frame::{ + deps::sp_consensus_aura::sr25519::AuthorityId as AuraId, prelude::*, + runtime::prelude::*, testing_prelude::*, + }; + + #[docify::export(CR)] + construct_runtime!( + pub struct Runtime { + // system-level pallets. + System: frame_system, + Timestamp: pallet_timestamp, + ParachainSystem: cumulus_pallet_parachain_system, + ParachainInfo: parachain_info, + + // parachain consensus support -- mandatory. + Aura: pallet_aura, + AuraExt: cumulus_pallet_aura_ext, + } + ); + + #[docify::export] + mod system_pallets { + use super::*; + + #[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)] + impl frame_system::Config for Runtime { + type Block = MockBlock; + type OnSetCode = cumulus_pallet_parachain_system::ParachainSetCode; + } + + impl cumulus_pallet_parachain_system::Config for Runtime { + type RuntimeEvent = RuntimeEvent; + type OnSystemEvent = (); + type SelfParaId = parachain_info::Pallet; + type OutboundXcmpMessageSource = (); + type XcmpMessageHandler = (); + type ReservedDmpWeight = (); + type ReservedXcmpWeight = (); + type CheckAssociatedRelayNumber = + cumulus_pallet_parachain_system::RelayNumberMonotonicallyIncreases; + type ConsensusHook = cumulus_pallet_aura_ext::FixedVelocityConsensusHook< + Runtime, + 6000, // relay chain block time + 1, + 1, + >; + type WeightInfo = (); + type DmpQueue = frame::traits::EnqueueWithOrigin<(), sp_core::ConstU8<0>>; + } + + impl parachain_info::Config for Runtime {} + } + + #[docify::export] + mod consensus_pallets { + use super::*; + + impl pallet_aura::Config for Runtime { + type AuthorityId = AuraId; + type DisabledValidators = (); + type MaxAuthorities = ConstU32<100_000>; + type AllowMultipleBlocksPerSlot = ConstBool; + #[cfg(feature = "experimental")] + type SlotDuration = pallet_aura::MinimumPeriodTimesTwo; + } + + #[docify::export(timestamp)] + #[derive_impl(pallet_timestamp::config_preludes::TestDefaultConfig as pallet_timestamp::DefaultConfig)] + impl pallet_timestamp::Config for Runtime {} + + impl cumulus_pallet_aura_ext::Config for Runtime {} + } + + #[docify::export(validate_block)] + cumulus_pallet_parachain_system::register_validate_block! { + Runtime = Runtime, + BlockExecutor = cumulus_pallet_aura_ext::BlockExecutor::, + } + } +} diff --git a/developer-hub/src/polkadot_sdk/frame_runtime.rs b/developer-hub/src/polkadot_sdk/frame_runtime.rs new file mode 100644 index 000000000000..d9cb1fd531bc --- /dev/null +++ b/developer-hub/src/polkadot_sdk/frame_runtime.rs @@ -0,0 +1,179 @@ +//! # FRAME +//! +//! ```no_compile +//! ______ ______ ________ ___ __ __ ______ +//! /_____/\ /_____/\ /_______/\ /__//_//_/\ /_____/\ +//! \::::_\/_\:::_ \ \ \::: _ \ \\::\| \| \ \\::::_\/_ +//! \:\/___/\\:(_) ) )_\::(_) \ \\:. \ \\:\/___/\ +//! \:::._\/ \: __ `\ \\:: __ \ \\:.\-/\ \ \\::___\/_ +//! \:\ \ \ \ `\ \ \\:.\ \ \ \\. \ \ \ \\:\____/\ +//! \_\/ \_\/ \_\/ \__\/\__\/ \__\/ \__\/ \_____\/ +//! ``` +//! +//! > **F**ramework for **R**untime **A**ggregation of **M**odularized **E**ntities: Substrate's +//! > State Transition Function (Runtime) Framework. +//! +//! ## Introduction +//! +//! As described in [`crate::reference_docs::wasm_meta_protocol`], at a high-level Substrate-based +//! blockchains are composed of two parts: +//! +//! 1. A *runtime* which represents the state transition function (i.e. "Business Logic") of a +//! blockchain, and is encoded as a WASM blob. +//! 2. A node whose primary purpose is to execute the given runtime. +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/substrate_simple.mmd")] +//! +//! *FRAME is the Substrate's framework of choice to build a runtime.* +//! +//! FRAME is composed of two major components, **pallets** and a **runtime**. +//! +//! ## Pallets +//! +//! A pallet is a unit of encapsulated logic. It has a clearly defined responsibility and can be +//! linked to other pallets. In order to be reusable, pallets shipped with FRAME strive to only care +//! about its own responsibilities and make as few assumptions about the general runtime as +//! possible. A pallet is analogous to a _module_ in the runtime. +//! +//! A pallet is defined as a `mod pallet` wrapped by the [`frame::pallet`] macro. Within this macro, +//! pallet components/parts can be defined. Most notable of these parts are: +//! +//! - [Config](frame::pallet_macros::config), allowing a pallet to make itself configurable and +//! generic over types, values and such. +//! - [Storage](frame::pallet_macros::storage), allowing a pallet to define onchain storage. +//! - [Dispatchable function](frame::pallet_macros::call), allowing a pallet to define extrinsics +//! that are callable by end users, from the outer world. +//! - [Events](frame::pallet_macros::event), allowing a pallet to emit events. +//! - [Errors](frame::pallet_macros::error), allowing a pallet to emit well-formed errors. +//! +//! Some of these pallet components resemble the building blocks of a smart contract. While both +//! models are programming state transition functions of blockchains, there are crucial differences +//! between the two. See [`crate::reference_docs::runtime_vs_smart_contract`] for more. +//! +//! Most of these components are defined using macros, the full list of which can be found in +//! [`frame::pallet_macros`]. +//! +//! ### Example +//! +//! The following examples showcases a minimal pallet. +#![doc = docify::embed!("src/polkadot_sdk/frame_runtime.rs", pallet)] +//! +//! +//! A runtime is a collection of pallets that are amalgamated together. Each pallet typically has +//! some configurations (exposed as a `trait Config`) that needs to be *specified* in the runtime. +//! This is done with [`frame::runtime::prelude::construct_runtime`]. +//! +//! A (real) runtime that actually wishes to compile to WASM needs to also implement a set of +//! runtime-apis. These implementation can be specified using the +//! [`frame::runtime::prelude::impl_runtime_apis`] macro. +//! +//! ### Example +//! +//! The following example shows a (test) runtime that is composing the pallet demonstrated above, +//! next to the [`frame::prelude::frame_system`] pallet, into a runtime. +#![doc = docify::embed!("src/polkadot_sdk/frame_runtime.rs", runtime)] +//! +//! ## More Examples +//! +//! You can find more FRAME examples that revolve around specific features at [`pallet_examples`]. +//! +//! ## Alternatives 🌈 +//! +//! There is nothing in the Substrate's node side code-base that mandates the use of FRAME. While +//! FRAME makes it very simple to write Substrate-based runtimes, it is by no means intended to be +//! the only one. At the end of the day, any WASM blob that exposes the right set of runtime APIs is +//! a valid Runtime form the point of view of a Substrate client (see +//! [`crate::reference_docs::wasm_meta_protocol`]). Notable examples are: +//! +//! * writing a runtime in pure Rust, as done in [this template](https://github.com/JoshOrndorff/frameless-node-template). +//! * writing a runtime in AssemblyScript,as explored in [this project](https://github.com/LimeChain/subsembly). + +#[cfg(test)] +mod tests { + use frame::prelude::*; + + /// A FRAME based pallet. This `mod` is the entry point for everything else. All + /// `#[pallet::xxx]` macros must be defined in this `mod`. Although, frame also provides an + /// experimental feature to break these parts into different `mod`s. See [`pallet_examples`] for + /// more. + #[docify::export] + #[frame::pallet(dev_mode)] + pub mod pallet { + use super::*; + + /// The configuration trait of a pallet. Mandatory. Allows a pallet to receive types at a + /// later point from the runtime that wishes to contain it. It allows the pallet to be + /// parameterized over both types and values. + #[pallet::config] + pub trait Config: frame_system::Config { + /// A type that is not known now, but the runtime that will contain this pallet will + /// know it later, therefore we define it here as an associated type. + type RuntimeEvent: IsType<::RuntimeEvent> + + From>; + + /// A parameterize-able value that we receive later via the `Get<_>` trait. + type ValueParameter: Get; + + /// Similar to [`Config::ValueParameter`], but using `const`. Both are functionally + /// equal, but offer different tradeoffs. + const ANOTHER_VALUE_PARAMETER: u32; + } + + /// A mandatory struct in each pallet. All functions callable by external users (aka. + /// transactions) must be attached to this type (see [`frame::pallet_macros::call`]). For + /// convenience, internal (private) functions can also be attached to this type. + #[pallet::pallet] + pub struct Pallet(PhantomData); + + /// The events tha this pallet can emit. + #[pallet::event] + pub enum Event {} + + /// A storage item that this pallet contains. This will be part of the state root trie/root + /// of the blockchain. + #[pallet::storage] + pub type Value = StorageValue; + + /// All *dispatchable* call functions (aka. transactions) are attached to `Pallet` in a + /// `impl` block. + #[pallet::call] + impl Pallet { + /// This will be callable by external users, and has two u32s as a parameter. + pub fn some_dispatchable( + _origin: OriginFor, + _param: u32, + _other_para: u32, + ) -> DispatchResult { + Ok(()) + } + } + } + + /// A simple runtime that contains the above pallet and `frame_system`, the mandatory pallet of + /// all runtimes. This runtime is for testing, but it shares a lot of similarities with a *real* + /// runtime. + #[docify::export] + pub mod runtime { + use super::pallet as pallet_example; + use frame::{prelude::*, testing_prelude::*}; + + // The major macro that amalgamates pallets into `struct Runtime` + construct_runtime!( + pub struct Runtime { + System: frame_system, + Example: pallet_example, + } + ); + + // These `impl` blocks specify the parameters of each pallet's `trait Config`. + #[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)] + impl frame_system::Config for Runtime { + type Block = MockBlock; + } + + impl pallet_example::Config for Runtime { + type RuntimeEvent = RuntimeEvent; + type ValueParameter = ConstU32<42>; + const ANOTHER_VALUE_PARAMETER: u32 = 42; + } + } +} diff --git a/developer-hub/src/polkadot_sdk/mod.rs b/developer-hub/src/polkadot_sdk/mod.rs new file mode 100644 index 000000000000..38ce5c404ffa --- /dev/null +++ b/developer-hub/src/polkadot_sdk/mod.rs @@ -0,0 +1,134 @@ +//! # Polkadot SDK +//! +//! [Polkadot SDK](https://github.com/paritytech/polkadot-sdk) provides the main resources needed to +//! start building on the [Polkadot network](https://polkadot.network), a scalable, multi-chain +//! blockchain platform that enables different blockchains to securely interoperate. +//! +//! [![StackExchange](https://img.shields.io/badge/StackExchange-Polkadot%20and%20Substrate-222222?logo=stackexchange)](https://substrate.stackexchange.com/) +//! +//! [![awesomeDot](https://img.shields.io/badge/polkadot-awesome-e6007a?logo=polkadot)](https://github.com/Awsmdot/awesome-dot) +//! [![wiki](https://img.shields.io/badge/polkadot-wiki-e6007a?logo=polkadot)](https://wiki.polkadot.network/) +//! [![forum](https://img.shields.io/badge/polkadot-forum-e6007a?logo=polkadot)](https://forum.polkadot.network/) +//! +//! [![RFCs](https://img.shields.io/badge/fellowship-RFCs-e6007a?logo=polkadot)](https://github.com/polkadot-fellows/rfcs) +//! [![Runtime](https://img.shields.io/badge/fellowship-runtimes-e6007a?logo=polkadot)](https://github.com/polkadot-fellows/runtimes) +//! [![Manifesto](https://img.shields.io/badge/fellowship-manifesto-e6007a?logo=polkadot)](https://github.com/polkadot-fellows/manifesto) +//! +//! ## Getting Started +//! +//! The primary way to get started with the Polkadot SDK is to start writing a FRAME-based runtime. +//! See: +//! +//! * [`polkadot`], to understand what is Polkadot as a development platform. +//! * [`substrate`], for an overview of what Substrate as the main blockchain framework of Polkadot +//! SDK. +//! * [`frame`], to learn about how to write blockchain applications aka. "App Chains". +//! * Continue with the [`developer_hub`'s "getting started"](crate#getting-started). +//! +//! ## Components +//! +//! #### Substrate +//! +//! [![Substrate-license](https://img.shields.io/badge/License-GPL3%2FApache2.0-blue)](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/LICENSE-APACHE2) +//! [![GitHub +//! Repo](https://img.shields.io/badge/github-substrate-2324CC85)](https://github.com/paritytech/polkadot-sdk/blob/master/substrate) +//! +//! [`substrate`] is the base blockchain framework used to power the Polkadot SDK. It is a full +//! toolkit to create sovereign blockchains, including but not limited to those who connect to +//! Polkadot as parachains. +//! +//! #### FRAME +//! +//! [![Substrate-license](https://img.shields.io/badge/License-Apache2.0-blue)](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/LICENSE-APACHE2) +//! [![GitHub +//! Repo](https://img.shields.io/badge/github-frame-2324CC85)](https://github.com/paritytech/polkadot-sdk/blob/master/substrate/frame) +//! +//! [`frame`] is the framework used to create Substrate-based application logic, aka. runtimes. +//! Learn more about the distinction of a runtime and node in +//! [`reference_docs::wasm_meta_protocol`]. +//! +//! #### Cumulus +//! +//! [![Cumulus-license](https://img.shields.io/badge/License-GPL3-blue)](https://github.com/paritytech/polkadot-sdk/blob/master/cumulus/LICENSE) +//! [![GitHub +//! Repo](https://img.shields.io/badge/github-cumulus-white)](https://github.com/paritytech/polkadot-sdk/blob/master/cumulus) +//! +//! [`cumulus`] transforms FRAME-based runtimes into Polkadot-compatible parachain runtimes, and +//! Substrate-based nodes into Polkadot/Parachain-compatible nodes. +//! +//! #### XCM +//! +//! [![XCM-license](https://img.shields.io/badge/License-GPL3-blue)](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/LICENSE) +//! [![GitHub +//! Repo](https://img.shields.io/badge/github-XCM-e6007a?logo=polkadot)](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/xcm) +//! +//! [`xcm`], short for "cross consensus message", is the primary format that is used for +//! communication between parachains, but is intended to be extensible to other use cases as well. +//! +//! #### Polkadot +//! +//! [![Polkadot-license](https://img.shields.io/badge/License-GPL3-blue)](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot/LICENSE) +//! [![GitHub +//! Repo](https://img.shields.io/badge/github-polkadot-e6007a?logo=polkadot)](https://github.com/paritytech/polkadot-sdk/blob/master/polkadot) +//! +//! [`polkadot`] is an implementation of a Polkadot node in Rust, by `@paritytech`. The Polkadot +//! runtimes are located under the +//! [`polkadot-fellows/runtimes`](https://github.com/polkadot-fellows/runtimes) repository. +//! +//! ### Summary +//! +//! The following diagram summarizes how some of the components of Polkadot SDK work together: +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/polkadot_sdk_substrate.mmd")] +//! +//! A Substrate-based chain is a blockchain composed of a runtime and a node. As noted above, the +//! runtime is the application logic of the blockchain, and the node is everything else. +//! See [`crate::reference_docs::wasm_meta_protocol`] for an in-depth explanation of this. The +//! former is built with [`frame`], and the latter is built with rest of Substrate. +//! +//! > You can think of a Substrate-based chain as a white-labeled blockchain. +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/polkadot_sdk_polkadot.mmd")] +//! Polkadot is itself a Substrate-based chain, composed of the exact same two components. It has +//! specialized logic in both the node and the runtime side, but it is not "special" in any way. +//! +//! A parachain is a "special" Substrate-based chain, whereby both the node and the runtime +//! components have became "Polkadot-aware" using Cumulus. +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/polkadot_sdk_parachain.mmd")] +//! +//! ## Notable Upstream Crates +//! +//! - [`parity-scale-codec`](https://github.com/paritytech/parity-scale-codec) +//! - [`parity-db`](https://github.com/paritytech/parity-db) +//! - [`trie`](https://github.com/paritytech/trie) +//! - [`parity-common`](https://github.com/paritytech/parity-common) +//! +//! ## Trophy Section: Notable Downstream Projects +//! +//! A list of projects and tools in the blockchain ecosystem that one way or another parts of the +//! Polkadot SDK: +//! +//! * [Polygon's spin-off, Avail](https://github.com/availproject/avail) +//! * [Cardano Partner Chains](https://iohk.io/en/blog/posts/2023/11/03/partner-chains-are-coming-to-cardano/) +//! * [Starknet's Madara Sequencer](https://github.com/keep-starknet-strange/madara) +//! +//! [`substrate`]: crate::polkadot_sdk::substrate +//! [`frame`]: crate::polkadot_sdk::frame_runtime +//! [`cumulus`]: crate::polkadot_sdk::cumulus +//! [`polkadot`]: crate::polkadot_sdk::polkadot +//! [`xcm`]: crate::polkadot_sdk::xcm + +/// Lean about Cumulus, the framework that transforms [`substrate`]-based chains into +/// [`polkadot`]-enabled parachains. +pub mod cumulus; +/// Learn about FRAME, the framework used to build Substrate runtimes. +pub mod frame_runtime; +/// Learn about Polkadot as a platform. +pub mod polkadot; +/// Learn about different ways through which smart contracts can be utilized on top of Substrate, +/// and in the Polkadot ecosystem. +pub mod smart_contracts; +/// Learn about Substrate, the main blockchain framework used in the Polkadot ecosystem. +pub mod substrate; +/// Index of all the templates that can act as first scaffold for a new project. +pub mod templates; +/// Learn about XCM, the de-facto communication language between different consensus systems. +pub mod xcm; diff --git a/developer-hub/src/polkadot_sdk/polkadot.rs b/developer-hub/src/polkadot_sdk/polkadot.rs new file mode 100644 index 000000000000..d157a660e564 --- /dev/null +++ b/developer-hub/src/polkadot_sdk/polkadot.rs @@ -0,0 +1,87 @@ +//! # Polkadot +//! +//! Implementation of the Polkadot node/host in Rust. +//! +//! ## Learn More and Get Involved +//! +//! - [Polkadot Forum](https://forum.polkadot.network/) +//! - [Polkadot Parachains](https://parachains.info/) +//! - [Polkadot (multi-chain) Explorer](https://subscan.io/) +//! - Polkadot Fellowship +//! - [Manifesto](https://github.com/polkadot-fellows/manifesto) +//! - [Runtimes](https://github.com/polkadot-fellows/runtimes) +//! - [RFCs](https://github.com/polkadot-fellows/rfcs) +//! - [Polkadot Specs](spec.polkadot.network) +//! - [The Polkadot Parachain Host Implementers' Guide](https://paritytech.github.io/polkadot-sdk/book/) +//! - [Whitepaper](https://www.polkadot.network/whitepaper/) +//! +//! ## Alternative Node Implementations 🌈 +//! +//! - [Smoldot](https://crates.io/crates/smoldot-light). Polkadot light node/client. +//! - [KAGOME](https://github.com/qdrvm/kagome). C++ implementation of the Polkadot host. +//! - [Gossamer](https://github.com/ChainSafe/gossamer). Golang implementation of the Polkadot host. +//! +//! ## Platform +//! +//! In this section, we examine what what platform Polkadot exactly provides to developers. +//! +//! ### Polkadot White Paper +//! +//! The original vision of Polkadot (everything in the whitepaper, which was eventually called +//! **Polkadot 1.0**) revolves around the following arguments: +//! +//! * Future is multi-chain, because we need different chains with different specialization to +//! achieve widespread goals. +//! * In other words, no single chain is good enough to achieve all goals. +//! * A multi-chain future will inadvertently suffer from fragmentation of economic security. +//! * This stake fragmentation will make communication over consensus system with varying security +//! levels inherently unsafe. +//! +//! Polkadot's answer to the above is: +//! +//! > The chains of the future must have a way to share their economic security, whilst maintaining +//! > their execution and governance sovereignty. These chains are called "Parachains". +//! +//! * Shared Security: The idea of shared economic security sits at the core of Polkadot. Polkadot +//! enables different parachains* to pool their economic security from Polkadot (i.e. "*Relay +//! Chain*"). +//! * (heterogenous) Sharded Execution: Yet, each parachain is free to have its own execution logic +//! (runtime), which also encompasses governance and sovereignty. Moreover, Polkadot ensures the +//! correct execution of all parachain, without having all of its validators re-execute all +//! parachain blocks. When seen from this perspective, the fact that Polkadot executes different +//! parachains means it is a platform that has fully delivered (the holy grail of) "Full Execution +//! Sharding". TODO: link to approval checking article. https://github.com/paritytech/polkadot-sdk-docs/issues/66 +//! * A framework to build blockchains: In order to materialize the ecosystem of parachains, an easy +//! blockchain framework must exist. This is [Substrate](crate::polkadot_sdk::substrate), +//! [FRAME](crate::polkadot_sdk::frame_runtime) and [Cumulus](crate::polkadot_sdk::cumulus). +//! * A communication language between blockchains: In order for these blockchains to communicate, +//! they need a shared language. [XCM](crate::polkadot_sdk::xcm) is one such language, and the one +//! that is most endorsed in the Polkadot ecosystem. +//! +//! > Note that the interoperability promised by Polkadot is unparalleled in that any two parachains +//! > connected to Polkadot have the same security and can have much better guarantees about the +//! > security of the recipient of any message. TODO: weakest link in bridges systems. https://github.com/paritytech/polkadot-sdk-docs/issues/66 +//! +//! Polkadot delivers the above vision, alongside a flexible means for parachains to schedule +//! themselves with the Relay Chain. To achieve this, Polkadot has been developed with an +//! architecture similar to that of a computer. Polkadot Relay Chain has a number of "cores". Each +//! core is (in simple terms) capable of progressing 1 parachain at a time. For example, a parachain +//! can schedule itself on a single core for 5 relay chain blocks. +//! +//! Within the scope of Polkadot 1.x, two main scheduling ways have been considered: +//! +//! * Long term Parachains, obtained through locking a sum of DOT in an auction system. +//! * on-demand Parachains, purchased through paying DOT to the relay-chain whenever needed. +//! +//! ### The Future +//! +//! After delivering Polkadot 1.x, the future of Polkadot as a protocol and platform is in the hands +//! of the community and the fellowship. This is happening most notable through the RFC process. +//! Some of the RFCs that do alter Polkadot as a platform and have already passed are as follows: +//! +//! - RFC#1: [Agile-coretime](https://github.com/polkadot-fellows/RFCs/blob/main/text/0001-agile-coretime.md): +//! Agile periodic-sale-based model for assigning Coretime on the Polkadot Ubiquitous Computer. +//! - RFC#5: [Coretime-interface](https://github.com/polkadot-fellows/RFCs/blob/main/text/0005-coretime-interface.md): +//! Interface for manipulating the usage of cores on the Polkadot Ubiquitous Computer. +// TODO: add more context and explanations about Polkadot as the Ubiquitous Computer and related +// tech. https://github.com/paritytech/polkadot-sdk-docs/issues/66 diff --git a/developer-hub/src/polkadot_sdk/smart_contracts.rs b/developer-hub/src/polkadot_sdk/smart_contracts.rs new file mode 100644 index 000000000000..a4916f9c9218 --- /dev/null +++ b/developer-hub/src/polkadot_sdk/smart_contracts.rs @@ -0,0 +1,9 @@ +//! # Smart Contracts +//! +//! TODO: @cmichi https://github.com/paritytech/polkadot-sdk-docs/issues/56 +//! +//! - WASM and EVM based, pallet-contracts and pallet-evm. +//! - single-daap-chain, transition from ink! to FRAME. +//! - Link to `use.ink` +//! - Link to [`crate::reference_docs::runtime_vs_smart_contract`]. +//! - https://use.ink/migrate-ink-contracts-to-polkadot-frame-parachain/ diff --git a/developer-hub/src/polkadot_sdk/substrate.rs b/developer-hub/src/polkadot_sdk/substrate.rs new file mode 100644 index 000000000000..399f18d03be9 --- /dev/null +++ b/developer-hub/src/polkadot_sdk/substrate.rs @@ -0,0 +1,151 @@ +//! # Substrate +//! +//! Substrate is a Rust framework for building blockchains in a modular and extensible way. While in +//! itself un-opinionated, it is the main engine behind the Polkadot ecosystem. +//! +//! ## Overview, Philosophy +//! +//! Substrate approaches blockchain development with an acknowledgement of a few self-evident +//! truths: +//! +//! 1. Society and technology evolves. +//! 2. Humans are fallible. +//! +//! This, makes the task of designing a correct, safe and long-lasting blockchain system hard. +//! +//! Nonetheless, in strive towards achieve this goal, Substrate embraces the following: +//! +//! 1. Use of **Rust** as a modern and safe programming language, which limits human error through +//! various means, most notably memory and type safety. +//! 2. Substrate is written from the ground-up with a *generic, modular and extensible* design. This +//! ensures that software components can be easily swapped and upgraded. Examples of this is +//! multiple consensus mechanisms provided by Substrate, as listed below. +//! 3. Lastly, the final blockchain system created with the above properties needs to be +//! upgradeable. In order to achieve this, Substrate is designed as a meta-protocol, whereby the +//! application logic of the blockchain (called "Runtime") is encoded as a WASM blob, and is +//! stored in the state. The rest of the system (called "node") acts as the executor of the WASM +//! blob. +//! +//! In essence, the meta-protocol of all Substrate based chains is the "Runtime as WASM blob" +//! accord. This enables the Runtime to become inherently upgradeable, crucially without forks. The +//! upgrade is merely a matter of the WASM blob being changed in the state, which is, in principle, +//! same as updating an account's balance. Learn more about this in detail in +//! [`crate::reference_docs::wasm_meta_protocol`]. +//! +//! > A great analogy for substrate is the following: Substrate node is a gaming console, and a WASM +//! > runtime, possibly created with FRAME is the game being inserted into the console. +//! +//! [`frame`], Substrate's default runtime development library, takes the above safety practices +//! even further by embracing a declarative programming model whereby correctness is enhanced and +//! the system is highly configurable through parameterization. Learn more about this in +//! [`crate::reference_docs::trait_based_programming`]. +//! +//! ## How to Get Started +//! +//! Substrate offers different options at the spectrum of technical freedom <-> development ease. +//! +//! * The easiest way to use Substrate is to use one of the templates (some of which listed at +//! [`crate::polkadot_sdk::templates`]) and only tweak the parameters of the runtime or node. This +//! allows you to launch a blockchain in minutes, but is limited in technical freedom. +//! * Next, most developers wish to develop their custom runtime modules, for which the de-facto way +//! is [`frame`](crate::polkadot_sdk::frame_runtime). +//! * Finally, Substrate is highly configurable at the node side as well, but this is the most +//! technically demanding. +//! +//! > A notable Substrate-based blockchain that has built both custom FRAME pallets and custom +//! > node-side components is . +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/substrate_dev.mmd")] +//! +//! ## Structure +//! +//! Substrate contains a large number of crates, therefore it is useful to have an overview of what +//! they are, and how they are organized. In broad terms, these crates are divided into three +//! categories: +//! +//! * `sc-*` (short for *Substrate-client*) crates, located under `./client` folder. These are all +//! the crates that lead to the node software. Notable examples [`sc_network`], various consensus +//! crates, RPC ([`sc_rpc_api`]) and database ([`sc_client_db`]), all of which are expected to +//! reside in the node side. +//! * `sp-*` (short for *substrate-primitives*) crates, located under `./primitives` folder. These +//! are crates that facilitate both the node and the runtime, but are not opinionated about what +//! framework is using for building the runtime. Notable examples are [`sp_api`] and [`sp_io`], +//! which form the communication bridge between the node and runtime. +//! * `pallet-*` and `frame-*` crates, located under `./frame` folder. These are the crates related +//! to FRAME. See [`frame`] for more information. +//! +//! ### WASM Build +//! +//! Many of the Substrate crates, such as entire `sp-*`, need to compile to both WASM (when a WASM +//! runtime is being generated) and native (for example, when testing). To achieve this, Substrate +//! follows the convention of the Rust community, and uses a `feature = "std"` to signify that a +//! crate is being built with the standard library, and is built for native. Otherwise, it is built +//! for `no_std`. +//! +//! This can be summarized in `#![cfg_attr(not(feature = "std"), no_std)]`, which you can often find +//! in any Substrate-based runtime. +//! +//! Substrate-based runtimes use [`substrate_wasm_builder`] in their `build.rs` to automatically +//! build their WASM files as a part of normal build command (e.g. `cargo build`). Once built, the +//! wasm file is placed in `./target/{debug|release}/wbuild/{runtime_name}.wasm`. +//! +//! ### Binaries +//! +//! Multiple binaries are shipped with substrate, the most important of which are located in the +//! [`./bin`](https://github.com/paritytech/polkadot-sdk/tree/master/substrate/bin) folder. +//! +//! * [`node_cli`] is an extensive substrate node that contains the superset of all runtime and node +//! side features. The corresponding runtime, called [`kitchensink_runtime`] contains all of the +//! modules that are provided with `FRAME`. This node and runtime is only used for testing and +//! demonstration. +//! * [`chain_spec_builder`]: Utility to build more detailed chain-specs for the aforementioned +//! node. Other projects typically contain a `build-spec` subcommand that does the same. +//! * [`node_template`](https://github.com/paritytech/polkadot-sdk/tree/master/substrate/bin/node-template): +//! a template node that contains a minimal set of features and can act as a starting point of a +//! project. +//! * [`subkey`]: Substrate's key management utility. +//! +//! ### Anatomy of a Binary Crate +//! +//! From the above, [`node_cli`]/[`kitchensink_runtime`] and `node-template` are essentially +//! blueprints of a Substrate-based project, as the name of the latter is implying. Each +//! Substrate-based project typically contains the following: +//! +//! * Under `./runtime`, a `./runtime/src/lib.rs` which is the top level runtime amalgamator file. +//! This file typically contains the [`frame::runtime::prelude::construct_runtime`] and +//! [`frame::runtime::prelude::impl_runtime_apis`] macro calls, which is the final definition of a +//! runtime. +//! +//! * Under `./node`, a `main.rs`, which is the starting point, and a `./service.rs`, which contains +//! all the node side components. Skimming this file yields an overview of the networking, +//! database, consensus and similar node side components. +//! +//! > The above two are conventions, not rules. +//! +//! > See for an update on how the node side +//! > components are being amalgamated. +//! +//! ## Parachain? +//! +//! As noted above, Substrate is the main engine behind the Polkadot ecosystem. One of the ways +//! through which Polkadot can be utilized is by building "parachains", blockchains that are +//! connected to Polkadot's shared security. +//! +//! To build a parachain, one could use [Cumulus](crate::polkadot_sdk::cumulus), the library on +//! top of Substrate, empowering any substrate-based chain to be a Polkadot parachain. +//! +//! ## Where To Go Next? +//! +//! Additional noteworthy crates within substrate: +//! +//! - RPC APIs of a Substrate node: [`sc_rpc_api`]/[`sc_rpc`] +//! - CLI Options of a Substrate node: [`sc_cli`] +//! - All of the consensus related crates provided by Substrate: +//! - [`sc_consensus_aura`] +//! - [`sc_consensus_babe`] +//! - [`sc_consensus_grandpa`] +//! - [`sc_consensus_beefy`] (TODO: @adrian, add some high level docs https://github.com/paritytech/polkadot-sdk-docs/issues/57) +//! - [`sc_consensus_manual_seal`] +//! - [`sc_consensus_pow`] + +#[doc(hidden)] +pub use crate::polkadot_sdk; diff --git a/developer-hub/src/polkadot_sdk/templates.rs b/developer-hub/src/polkadot_sdk/templates.rs new file mode 100644 index 000000000000..f60c75b8f219 --- /dev/null +++ b/developer-hub/src/polkadot_sdk/templates.rs @@ -0,0 +1,45 @@ +//! # Templates +//! +//! ### Internal +//! +//! The following templates are maintained as a part of the `polkadot-sdk` repository: +//! +//! - classic [`substrate-node-template`]: is a white-labeled substrate-based blockchain with a +//! moderate amount of features. It can act as a great starting point for those who want to learn +//! Substrate/FRAME and want to have a template that is already doing something. +//! - [`substrate-minimal-template`]: Same as the above, but it contains the least amount of code in +//! both the node and runtime. It is a great starting point for those who want to deeply learn +//! Substrate and FRAME. +//! - classic [`cumulus-parachain-template`], which is the de-facto parachain template shipped with +//! Cumulus. It is the parachain-enabled version of [`substrate-node-template`]. +//! +//! ### External Templates +//! +//! Noteworthy templates outside of this repository. +//! +//! - [`extended-parachain-template`](https://github.com/paritytech/extended-parachain-template): A +//! parachain template that contains more built-in functionality such as assets and NFTs. +//! - [`frontier-parachain-template`](https://github.com/paritytech/frontier-parachain-template): A +//! parachain template for launching EVM-compatible parachains. +//! +//! [`substrate-node-template`]: https://github.com/paritytech/polkadot-sdk/blob/master/substrate/bin/node-template/ +//! [`substrate-minimal-template`]: https://github.com/paritytech/polkadot-sdk/blob/master/substrate/bin/minimal/ +//! [`cumulus-parachain-template`]: https://github.com/paritytech/polkadot-sdk/blob/master/cumulus/parachain-template/ + +// TODO: in general, we need to make a deliberate choice here of moving a few key templates to this +// repo (nothing stays in `substrate-developer-hub`) and the everything else should be community +// maintained. https://github.com/paritytech/polkadot-sdk-docs/issues/67 + +// TODO: we should rename `substrate-node-template` to `substrate-basic-template`, +// `substrate-blockchain-template`. `node` is confusing in the name. +// `substrate-blockchain-template` and `cumulus-parachain-template` go well together 🤝. https://github.com/paritytech/polkadot-sdk-docs/issues/67 + +// NOTE: a super important detail that I am looking forward to here is +// and +// . Meaning that I would not spend time on +// teaching someone too much detail about the ugly thing we call "node" nowadays. In the future, I +// am sure we will either have a better "node-builder" code that can actually be tested, or an +// "omni-node" that can run (almost) any wasm file. We should already build tutorials in this +// direction IMO. This also affects all the templates. If we have a good neat runtime file, which we +// are moving toward, and a good node-builder, we don't need all of these damn templates. These +// templates are only there because the boilerplate is super horrible atm. diff --git a/developer-hub/src/polkadot_sdk/xcm.rs b/developer-hub/src/polkadot_sdk/xcm.rs new file mode 100644 index 000000000000..0d600f751c8b --- /dev/null +++ b/developer-hub/src/polkadot_sdk/xcm.rs @@ -0,0 +1,5 @@ +//! # XCM +//! +//! @KiChjang @franciscoaguirre +//! TODO: RFCs, xcm-spec, the future of the repo, minimal example perhaps, forward to where actual +//! docs are hosted. https://github.com/paritytech/polkadot-sdk-docs/issues/58 diff --git a/developer-hub/src/reference_docs/blockchain_scalibility.rs b/developer-hub/src/reference_docs/blockchain_scalibility.rs new file mode 100644 index 000000000000..e69de29bb2d1 diff --git a/developer-hub/src/reference_docs/blockchain_state_machines.rs b/developer-hub/src/reference_docs/blockchain_state_machines.rs new file mode 100644 index 000000000000..611ca2b778a9 --- /dev/null +++ b/developer-hub/src/reference_docs/blockchain_state_machines.rs @@ -0,0 +1,29 @@ +//! # State Transition Function +//! +//! This document briefly explains how in the context of Substrate-based blockchains, we view the +//! blockchain as a **decentralized state transition function**. +//! +//! Recall that a blockchain's main purpose is to help a permissionless set of entities to agree on +//! a shared data-set, and how it evolves. This is called the **State**, also referred to as +//! "onchain" data, or *Storage* in the context of FRAME. The state is where the account balance of +//! each user is, for example, stored, and there is a canonical version of it that everyone agrees +//! upon. +//! +//! Then, recall that a typical blockchain system will alter its state through execution of blocks. +//! *The component that dictates how this state alteration can happen is called the state transition +//! function*. +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/stf_simple.mmd")] +//! +//! In Substrate-based blockchains, the state transition function is called the *Runtime*. This is +//! explained further in [`crate::reference_docs::wasm_meta_protocol`]. +//! +//! With this in mind, we can paint a complete picture of a blockchain as a state machine: +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/stf.mmd")] +//! +//! In essence, the state of the blockchain at block N is the outcome of applying the state +//! transition function to the the previous state, and the current block as input. This can be +//! mathematically represented as: +//! +//! ```math +//! STF = F(State_N, Block_N) -> State_{N+1} +//! ``` diff --git a/developer-hub/src/reference_docs/chain_spec_genesis.rs b/developer-hub/src/reference_docs/chain_spec_genesis.rs new file mode 100644 index 000000000000..2ac51a91f2de --- /dev/null +++ b/developer-hub/src/reference_docs/chain_spec_genesis.rs @@ -0,0 +1,4 @@ +//! Chain spec and genesis build. +//! +//! What is chain-spec. +//! What is genesis state and how to build it. diff --git a/developer-hub/src/reference_docs/cli.rs b/developer-hub/src/reference_docs/cli.rs new file mode 100644 index 000000000000..9274e86b04ef --- /dev/null +++ b/developer-hub/src/reference_docs/cli.rs @@ -0,0 +1,7 @@ +//! # Command Line Arguments +//! +//! +//! Notes: +//! +//! - Command line arguments of a typical substrate based chain, and how to find and learn them. +//! - How to extend them with your custom stuff. diff --git a/developer-hub/src/reference_docs/consensus_swapping.rs b/developer-hub/src/reference_docs/consensus_swapping.rs new file mode 100644 index 000000000000..e639761ee97b --- /dev/null +++ b/developer-hub/src/reference_docs/consensus_swapping.rs @@ -0,0 +1,6 @@ +//! Consensus Swapping +//! +//! Notes: +//! +//! - The typical workshop done by Joshy in some places where he swaps out the consensus to be PoW. +//! - This could also be a tutorial rather than a ref doc, depending on the size. diff --git a/developer-hub/src/reference_docs/extrinsic_encoding.rs b/developer-hub/src/reference_docs/extrinsic_encoding.rs new file mode 100644 index 000000000000..b10b100bfbbe --- /dev/null +++ b/developer-hub/src/reference_docs/extrinsic_encoding.rs @@ -0,0 +1,277 @@ +//! # Constructing and Signing Extrinsics +//! +//! Extrinsics are payloads that are stored in blocks which are responsible for altering the state +//! of a blockchain via the [_state transition +//! function_][crate::reference_docs::blockchain_state_machines]. +//! +//! Substrate is configurable enough that extrinsics can take any format. In practice, runtimes +//! tend to use our [`sp_runtime::generic::UncheckedExtrinsic`] type to represent extrinsics, +//! because it's generic enough to cater for most (if not all) use cases. In Polkadot, this is +//! configured [here](https://github.com/polkadot-fellows/runtimes/blob/94b2798b69ba6779764e20a50f056e48db78ebef/relay/polkadot/src/lib.rs#L1478) +//! at the time of writing. +//! +//! What follows is a description of how extrinsics based on this +//! [`sp_runtime::generic::UncheckedExtrinsic`] type are encoded into bytes. Specifically, we are +//! looking at how extrinsics with a format version of 4 are encoded. This version is itself a part +//! of the payload, and if it changes, it indicates that something about the encoding may have +//! changed. +//! +//! # Encoding an Extrinsic +//! +//! At a high level, all extrinsics compatible with [`sp_runtime::generic::UncheckedExtrinsic`] +//! are formed from concatenating some details together, as in the following pseudo-code: +//! +//! ```text +//! extrinsic_bytes = concat( +//! compact_encoded_length, +//! version_and_maybe_signature, +//! call_data +//! ) +//! ``` +//! +//! For clarity, the actual implementation in Substrate looks like this: +#![doc = docify::embed!("../substrate/primitives/runtime/src/generic/unchecked_extrinsic.rs", unchecked_extrinsic_encode_impl)] +//! +//! Let's look at how each of these details is constructed: +//! +//! ## compact_encoded_length +//! +//! This is a [SCALE compact encoded][frame::deps::codec::Compact] integer which is equal to the +//! length, in bytes, of the rest of the extrinsic details. +//! +//! To obtain this value, we must encode and concatenate together the rest of the extrinsic details +//! first, and then obtain the byte length of these. We can then compact encode that length, and +//! prepend it to the rest of the details. +//! +//! ## version_and_maybe_signature +//! +//! If the extrinsic is _unsigned_, then `version_and_maybe_signature` will be just one byte +//! denoting the _transaction protocol version_, which is 4 (or `0b0000_0100`). +//! +//! If the extrinsic is _signed_ (all extrinsics submitted from users must be signed), then +//! `version_and_maybe_signature` is obtained by concatenating some details together, ie: +//! +//! ```text +//! version_and_maybe_signature = concat( +//! version_and_signed, +//! from_address, +//! signature, +//! signed_extensions_extra, +//! ) +//! ``` +//! +//! Each of the details to be concatenated together is explained below: +//! +//! ### version_and_signed +//! +//! This is one byte, equal to `0x84` or `0b1000_0100` (i.e. an upper 1 bit to denote that it is +//! signed, and then the transaction version, 4, in the lower bits). +//! +//! ### from_address +//! +//! This is the [SCALE encoded][frame::deps::codec] address of the sender of the extrinsic. The +//! address is the first generic parameter of [`sp_runtime::generic::UncheckedExtrinsic`], and so +//! can vary from chain to chain. +//! +//! The address type used on the Polkadot relay chain is [`sp_runtime::MultiAddress`], +//! where `AccountId32` is defined [here][`sp_core::crypto::AccountId32`]. When constructing a +//! signed extrinsic to be submitted to a Polkadot node, you'll always use the +//! [`sp_runtime::MultiAddress::Id`] variant to wrap your `AccountId32`. +//! +//! ### signature +//! +//! This is the [SCALE encoded][frame::deps::codec] signature. The signature type is configured via +//! the third generic parameter of [`sp_runtime::generic::UncheckedExtrinsic`], which determines the +//! shape of the signature and signing algorithm that should be used. +//! +//! The signature is obtained by signing the _signed payload_ bytes (see below on how this is +//! constructed) using the private key associated with the address and correct algorithm. +//! +//! The signature type used on the Polkadot relay chain is [`sp_runtime::MultiSignature`]; the +//! variants there are the types of signature that can be provided. +//! +//! ### signed_extensions_extra +//! +//! This is the concatenation of the [SCALE encoded][frame::deps::codec] bytes representing each of +//! the [_signed extensions_][sp_runtime::traits::SignedExtension], and are configured by the +//! fourth generic parameter of [`sp_runtime::generic::UncheckedExtrinsic`]. Learn more about +//! signed extensions [here][crate::reference_docs::signed_extensions]. +//! +//! When it comes to constructing an extrinsic, each signed extension has two things that we are +//! interested in here: +//! +//! - The actual SCALE encoding of the signed extension type itself; this is what will form our +//! `signed_extensions_extra` bytes. +//! - An `AdditionalSigned` type. This is SCALE encoded into the `signed_extensions_additional` data +//! of the _signed payload_ (see below). +//! +//! Either (or both) of these can encode to zero bytes. +//! +//! Each chain configures the set of signed extensions that it uses in its runtime configuration. +//! At the time of writing, Polkadot configures them +//! [here](https://github.com/polkadot-fellows/runtimes/blob/1dc04eb954eadf8aadb5d83990b89662dbb5a074/relay/polkadot/src/lib.rs#L1432C25-L1432C25). +//! Some of the common signed extensions are defined +//! [here][frame::deps::frame_system#signed-extensions]. +//! +//! Information about exactly which signed extensions are present on a chain and in what order is +//! also a part of the metadata for the chain. For V15 metadata, it can be +//! [found here][frame::deps::frame_support::__private::metadata::v15::ExtrinsicMetadata]. +//! +//! ## call_data +//! +//! This is the main payload of the extrinsic, which is used to determine how the chain's state is +//! altered. This is defined by the second generic parameter of +//! [`sp_runtime::generic::UncheckedExtrinsic`]. +//! +//! A call can be anything that implements [`Encode`][frame::deps::codec::Encode]. In FRAME-based +//! runtimes, a call is represented as an enum of enums, where the outer enum represents the FRAME +//! pallet being called, and the inner enum represents the call being made within that pallet, and +//! any arguments to it. Read more about the call enum +//! [here][crate::reference_docs::frame_composite_enums]. +//! +//! FRAME `Call` enums are automatically generated, and end up looking something like this: +#![doc = docify::embed!("./src/reference_docs/extrinsic_encoding.rs", call_data)] +//! +//! In pseudo-code, this `Call` enum encodes equivalently to: +//! +//! ```text +//! call_data = concat( +//! pallet_index, +//! call_index, +//! call_args +//! ) +//! ``` +//! +//! - `pallet_index` is a single byte denoting the index of the pallet that we are calling into, and +//! is what the tag of the outermost enum will encode to. +//! - `call_index` is a single byte denoting the index of the call that we are making the pallet, +//! and is what the tag of the inner enum will encode to. +//! - `call_args` are the SCALE encoded bytes for each of the arguments that the call expects, and +//! are typically provided as values to the inner enum. +//! +//! Information about the pallets that exist for a chain (including their indexes), the calls +//! available in each pallet (including their indexes), and the arguments required for each call +//! can be found in the metadata for the chain. For V15 metadata, this information +//! [is here][frame::deps::frame_support::__private::metadata::v15::PalletMetadata]. +//! +//! # The Signed Payload Format +//! +//! All extrinsics submitted to a node from the outside world (also known as _transactions_) need to +//! be _signed_. The data that needs to be signed for some extrinsic is called the _signed payload_, +//! and its shape is described by the following pseudo-code: +//! +//! ```text +//! signed_payload = concat( +//! call_data, +//! signed_extensions_extra, +//! signed_extensions_additional, +//! ) +//! +//! if length(signed_payload) > 256 { +//! signed_payload = blake2_256(signed_payload) +//! } +//! ``` +//! +//! The bytes representing `call_data` and `signed_extensions_extra` can be obtained as descibed +//! above. `signed_extensions_additional` is constructed by SCALE encoding the +//! ["additional signed" data][sp_runtime::traits::SignedExtension::AdditionalSigned] for each +//! signed extension that the chain is using, in order. +//! +//! Once we've concatenated those together, we hash the result if it's greater than 256 bytes in +//! length using a Blake2 256bit hasher. +//! +//! The [`sp_runtime::generic::SignedPayload`] type takes care of assembling the correct payload +//! for us, given `call_data` and a tuple of signed extensions. +//! +//! # Example Encoding +//! +//! Using [`sp_runtime::generic::UncheckedExtrinsic`], we can construct and encode an extrinsic +//! as follows: +#![doc = docify::embed!("./src/reference_docs/extrinsic_encoding.rs", encoding_example)] + +#[docify::export] +pub mod call_data { + use parity_scale_codec::{Decode, Encode}; + + // The outer enum composes calls within + // different pallets together. We have two + // pallets, "PalletA" and "PalletB". + #[derive(Encode, Decode)] + pub enum Call { + #[codec(index = 0)] + PalletA(PalletACall), + #[codec(index = 7)] + PalletB(PalletBCall), + } + + // An inner enum represents the calls within + // a specific pallet. "PalletA" has one call, + // "Foo". + #[derive(Encode, Decode)] + pub enum PalletACall { + #[codec(index = 0)] + Foo(String), + } + + #[derive(Encode, Decode)] + pub enum PalletBCall { + #[codec(index = 0)] + Bar(String), + } +} + +#[docify::export] +pub mod encoding_example { + use super::call_data::{Call, PalletACall}; + use crate::reference_docs::signed_extensions::signed_extensions_example; + use parity_scale_codec::Encode; + use sp_core::crypto::AccountId32; + use sp_keyring::sr25519::Keyring; + use sp_runtime::{ + generic::{SignedPayload, UncheckedExtrinsic}, + MultiAddress, MultiSignature, + }; + + // Define some signed extensions to use. We'll use a couple of examples + // from the signed extensions reference doc. + type SignedExtensions = + (signed_extensions_example::AddToPayload, signed_extensions_example::AddToSignaturePayload); + + // We'll use `UncheckedExtrinsic` to encode our extrinsic for us. We set + // the address and signature type to those used on Polkadot, use our custom + // `Call` type, and use our custom set of `SignedExtensions`. + type Extrinsic = + UncheckedExtrinsic, Call, MultiSignature, SignedExtensions>; + + pub fn encode_demo_extrinsic() -> Vec { + // The "from" address will be our Alice dev account. + let from_address = MultiAddress::::Id(Keyring::Alice.to_account_id()); + + // We provide some values for our expected signed extensions. + let signed_extensions = ( + signed_extensions_example::AddToPayload(1), + signed_extensions_example::AddToSignaturePayload, + ); + + // Construct our call data: + let call_data = Call::PalletA(PalletACall::Foo("Hello".to_string())); + + // The signed payload. This takes care of encoding the call_data, + // signed_extensions_extra and signed_extensions_additional, and hashing + // the result if it's > 256 bytes: + let signed_payload = SignedPayload::new(&call_data, signed_extensions.clone()); + + // Sign the signed payload with our Alice dev account's private key, + // and wrap the signature into the expected type: + let signature = { + let sig = Keyring::Alice.sign(&signed_payload.encode()); + MultiSignature::Sr25519(sig) + }; + + // Now, we can build and encode our extrinsic: + let ext = Extrinsic::new_signed(call_data, from_address, signature, signed_extensions); + + let encoded_ext = ext.encode(); + encoded_ext + } +} diff --git a/developer-hub/src/reference_docs/fee_less_runtime.rs b/developer-hub/src/reference_docs/fee_less_runtime.rs new file mode 100644 index 000000000000..43a761a6c52c --- /dev/null +++ b/developer-hub/src/reference_docs/fee_less_runtime.rs @@ -0,0 +1,12 @@ +//! # Fee-Less Runtime +//! +//! +//! Notes: +//! +//! - An extension of [`runtime_vs_smart_contract`], showcasing the tools needed to build a safe +//! runtime that is fee-less. +//! - Would need to use unsigned origins, custom validate_unsigned, check the existence of some NFT +//! and some kind of rate limiting (eg. any account gets 5 free tx per day). +//! - The rule of thumb is that as long as the unsigned validate does one storage read, similar to +//! nonce, it is fine. +//! - This could possibly be a good tutorial/template, rather than a reference doc. diff --git a/developer-hub/src/reference_docs/frame_benchmarking_weight.rs b/developer-hub/src/reference_docs/frame_benchmarking_weight.rs new file mode 100644 index 000000000000..f65f4174ec66 --- /dev/null +++ b/developer-hub/src/reference_docs/frame_benchmarking_weight.rs @@ -0,0 +1,23 @@ +//! # FRAME Benchmarking and Weights. +//! +//! Notes: +//! +//! On Weight as a concept. +//! +//! - Why we need it. Super important. People hate this. We need to argue why it is worth it. +//! - Axis of weight: PoV + Time. +//! - pre dispatch weight vs. metering and post dispatch correction. +//! - mention that we will do this for PoV +//! - you can manually refund using `DispatchResultWithPostInfo`. +//! - Technically you can have weights with any benchmarking framework. You just need one number to +//! be computed pre-dispatch. But FRAME gives you a framework for this. +//! - improve documentation of `#[weight = ..]` and `#[pallet::weight(..)]`. All syntax variation +//! should be covered. +//! +//! on FRAME benchmarking machinery: +//! +//! - component analysis, why everything must be linear. +//! - how to write benchmarks, how you must think of worst case. +//! - how to run benchmarks. +//! +//! - https://www.shawntabrizi.com/substrate/substrate-storage-deep-dive/ diff --git a/developer-hub/src/reference_docs/frame_composite_enums.rs b/developer-hub/src/reference_docs/frame_composite_enums.rs new file mode 100644 index 000000000000..6051cd534467 --- /dev/null +++ b/developer-hub/src/reference_docs/frame_composite_enums.rs @@ -0,0 +1 @@ +//! # FRAME Composite Enums diff --git a/developer-hub/src/reference_docs/frame_currency.rs b/developer-hub/src/reference_docs/frame_currency.rs new file mode 100644 index 000000000000..ba181373062f --- /dev/null +++ b/developer-hub/src/reference_docs/frame_currency.rs @@ -0,0 +1,8 @@ +//! FRAME Currency Abstractions and Traits +//! +//! Notes: +//! +//! - History, `Currency` trait. +//! - `Hold` and `Freeze` with diagram. +//! - `HoldReason` and `FreezeReason` +//! - This footgun: https://github.com/paritytech/polkadot-sdk/pull/1900#discussion_r1363783609 diff --git a/developer-hub/src/reference_docs/frame_origin.rs b/developer-hub/src/reference_docs/frame_origin.rs new file mode 100644 index 000000000000..a4078377cd77 --- /dev/null +++ b/developer-hub/src/reference_docs/frame_origin.rs @@ -0,0 +1,14 @@ +//! # FRAME Origin +//! +//! Notes: +//! +//! - Def talk about account abstraction and how it is a solved issue in frame. See Gav's talk in +//! Protocol Berg 2023 +//! - system's raw origin, how it is amalgamated with other origins into one type +//! [`frame_composite_enums`] +//! - signed origin +//! - unsigned origin, link to [`fee_less_runtime`] +//! - Root origin, how no one can obtain it. +//! - Abstract origin: how FRAME allows you to express "origin is 2/3 of the this body or 1/2 of +//! that body or half of the token holders". +//! - `type CustomOrigin: EnsureOrigin<_>` in pallets. diff --git a/developer-hub/src/reference_docs/frame_runtime_migration.rs b/developer-hub/src/reference_docs/frame_runtime_migration.rs new file mode 100644 index 000000000000..0616ccbb6f57 --- /dev/null +++ b/developer-hub/src/reference_docs/frame_runtime_migration.rs @@ -0,0 +1,9 @@ +//! # Runtime Runtime Upgrade and Testing +//! +//! +//! Notes: +//! +//! - Flow of things, when does `on_runtime_upgrade` get called. Link to to `Hooks` and its diagram +//! as source of truth. +//! - Data migration and when it is needed. +//! - Look into the pba-lecture. diff --git a/developer-hub/src/reference_docs/frame_system_accounts.rs b/developer-hub/src/reference_docs/frame_system_accounts.rs new file mode 100644 index 000000000000..ae9d2c9e0cb3 --- /dev/null +++ b/developer-hub/src/reference_docs/frame_system_accounts.rs @@ -0,0 +1,8 @@ +//! # FRAME Accounts +//! +//! How `frame_system` handles accountIds. Nonce. Consumers and Providers, reference counting. + +// - poorly understood topics, needs one great article to rul them all. +// - https://github.com/paritytech/substrate/issues/14425 +// - https://github.com/paritytech/substrate/pull/12951 +// - https://substrate.stackexchange.com/questions/263/what-is-the-meaning-of-the-account-provider-sufficients-and-consumer diff --git a/developer-hub/src/reference_docs/glossary.rs b/developer-hub/src/reference_docs/glossary.rs new file mode 100644 index 000000000000..d0bd6accce6c --- /dev/null +++ b/developer-hub/src/reference_docs/glossary.rs @@ -0,0 +1,62 @@ +//! # Glossary +//! +//! #### State +//! +//! The data around which the blockchain network wishes to come to consensus. Also +//! referred to as "onchain data", "onchain storage" or sometimes just "storage". In UTXO based +//! blockchains, is referred to as "ledger". +//! +//! **Synonyms**: Onchain data, Onchain storage, Storage, Ledger +//! +//! #### State Transition Function +//! +//! The WASM Blob that dictates how the blockchain should transition its state upon encountering new +//! blocks. +//! +//! #### Host +//! +//! The environment that hosts and executes the [state transition function's WASM +//! blob](#state-transition-function). +//! +//! #### Node +//! +//! The full software artifact that contains the [host](#host), but importantly also all the other +//! modules needed to be part of a blockchain network, such as peer-to-peer networking, database and +//! such. +//! +//! **Synonyms**: Client +//! +//! #### Light Node +//! +//! Same as [node](#nodes), but when capable of following the network only through listening to +//! block headers. Usually capable of running in more constrained environments, such as an embedded +//! device, phone, or a web browser. +//! +//! **Synonyms**: Light Client +//! +//! #### Offchain +//! +//! #### Host Function: +//! +//! #### Runtime API: +//! +//! #### Dispatchable: +//! +//! Callable +//! +//! #### Extrinsic +//! +//! +//! #### Pallet +//! +//! #### Full Node +//! +//! #### Archive Node +//! +//! #### Validator +//! +//! #### Collator +//! +//! #### Parachain +//! +//! aka. AppChain. diff --git a/developer-hub/src/reference_docs/light_nodes.rs b/developer-hub/src/reference_docs/light_nodes.rs new file mode 100644 index 000000000000..a6a0a828ef58 --- /dev/null +++ b/developer-hub/src/reference_docs/light_nodes.rs @@ -0,0 +1,7 @@ +//! # Light Clients +//! +//! +//! Notes: should contain only high level information about light clients, then link to how to set +//! it up in PAPI and SubXT +//! https://docs.substrate.io/learn/light-clients-in-substrate-connect/ +//! https://github.com/substrate-developer-hub/substrate-front-end-template/pull/277 diff --git a/developer-hub/src/reference_docs/metadata.rs b/developer-hub/src/reference_docs/metadata.rs new file mode 100644 index 000000000000..702c1c30fd9c --- /dev/null +++ b/developer-hub/src/reference_docs/metadata.rs @@ -0,0 +1 @@ +//! # Metadata diff --git a/developer-hub/src/reference_docs/mod.rs b/developer-hub/src/reference_docs/mod.rs new file mode 100644 index 000000000000..44284394000d --- /dev/null +++ b/developer-hub/src/reference_docs/mod.rs @@ -0,0 +1,99 @@ +//! # Polkadot SDK Reference Docs. +//! +//! This is the entry point for all reference documents that enhance one's learning experience in +//! the Polkadot SDK. +//! +//! Note that this module also contains the [glossary](crate::reference_docs::glossary). +//! +//! ## What is a "reference document"? +//! +//! First, see [why we use rust-docs for everything](crate#why-rust-docs) and our documentation +//! [principles](crate#principles). We acknowledge that as much of the crucial information should be +//! embedded in the low level rust-docs. Then, high level scenarios should be covered in +//! [`crate::guides`]. Finally, we acknowledge that there is a category of information that is: +//! +//! 1. crucial to know. +//! 2. is too high level to be in the rust-doc of any one `type`, `trait` or `fn`. +//! 3. is too low level to be encompassed in a [`crate::guides`]. +//! +//! We call this class of documents "reference documents". Our goal should be to minimize the number +//! of "reference" docs, as they incur maintenance burden. + +/// Learn how Substrate and FRAME use traits and associated types to make modules generic in a +/// type-safe manner. +pub mod trait_based_programming; + +/// Learn about the way Substrate and FRAME view their blockchains as state machines. +pub mod blockchain_state_machines; + +/// The glossary. +pub mod glossary; + +/// Learn about the WASM meta-protocol of all Substrate-based chains. +pub mod wasm_meta_protocol; + +/// Learn about the differences between smart contracts and a FRAME-based runtime. They are both +/// "code stored onchain", but how do they differ? +pub mod runtime_vs_smart_contract; + +/// Learn about how extrinsics are encoded to be transmitted to a node and stored in blocks. +pub mod extrinsic_encoding; + +/// Learn about the signed extensions that form a part of extrinsics. +// TODO: @jsdw https://github.com/paritytech/polkadot-sdk-docs/issues/42 +pub mod signed_extensions; + +/// Learn about *"Origin"* A topic in FRAME that enables complex account abstractions to be built. +// TODO: @shawntabrizi https://github.com/paritytech/polkadot-sdk-docs/issues/43 +pub mod frame_origin; + +/// Learn about how to write safe and defensive code in your FRAME runtime. +// TODO: @CrackTheCode016 https://github.com/paritytech/polkadot-sdk-docs/issues/44 +pub mod safe_defensive_programming; + +/// Learn about composite enums in FRAME-based runtimes, such as "RuntimeEvent" and "RuntimeCall". +pub mod frame_composite_enums; + +/// Learn about how to make a pallet/runtime that is fee-less and instead uses another mechanism to +/// control usage and sybil attacks. +pub mod fee_less_runtime; + +/// Learn about metadata, the main means through which an upgradeable runtime communicates its +/// properties to the outside world. +// TODO: @jsdw https://github.com/paritytech/polkadot-sdk-docs/issues/47 +pub mod metadata; + +/// Learn about how frame-system handles `account-ids`, nonces, consumers and providers. +pub mod frame_system_accounts; + +/// Learn about the currency-related abstractions provided in FRAME. +pub mod frame_currency; + +/// Learn about benchmarking and weight. +// TODO: @shawntabrizi @ggwpez https://github.com/paritytech/polkadot-sdk-docs/issues/50 +pub mod frame_benchmarking_weight; + +/// Learn about chain specification file and the genesis state of the blockchain. +// TODO: @michalkucharczyk https://github.com/paritytech/polkadot-sdk-docs/issues/51 +pub mod chain_spec_genesis; + +/// Learn about all the memory limitations of the WASM runtime when it comes to memory usage. +// TODO: @kianenigma https://github.com/paritytech/polkadot-sdk-docs/issues/52 +pub mod wasm_memory; + +/// Learn about Substrate's CLI, and how it can be extended. +// TODO: @kianenigma https://github.com/paritytech/polkadot-sdk-docs/issues/53 +pub mod cli; + +/// Learn about Substrate's consensus algorithms, and how you can switch between two. +// TODO: @JoshOrndorff @kianenigma https://github.com/paritytech/polkadot-sdk-docs/issues/54 +pub mod consensus_swapping; + +/// Learn about all the advance ways to test your coordinate a rutnime upgrade and data migration. +// TODO: @liamaharon https://github.com/paritytech/polkadot-sdk-docs/issues/55 +pub mod frame_runtime_migration; + +/// Learn about light nodes, how they function, and how Substrate-based chains come +/// light-node-first out of the box. +// TODO: @jsdw @josepot https://github.com/paritytech/polkadot-sdk-docs/issues/68 +pub mod light_nodes; diff --git a/developer-hub/src/reference_docs/runtime_vs_smart_contract.rs b/developer-hub/src/reference_docs/runtime_vs_smart_contract.rs new file mode 100644 index 000000000000..7f96fa1800ae --- /dev/null +++ b/developer-hub/src/reference_docs/runtime_vs_smart_contract.rs @@ -0,0 +1,6 @@ +//! Runtime vs. Smart Contracts +//! +//! Notes: +//! +//! Why one can be weighed, and one MUST be metered. +//! https://forum.polkadot.network/t/where-contracts-fail-and-runtimes-chains-are-needed/4464/3 diff --git a/developer-hub/src/reference_docs/safe_defensive_programming.rs b/developer-hub/src/reference_docs/safe_defensive_programming.rs new file mode 100644 index 000000000000..9d0f028e570d --- /dev/null +++ b/developer-hub/src/reference_docs/safe_defensive_programming.rs @@ -0,0 +1 @@ +//! diff --git a/developer-hub/src/reference_docs/signed_extensions.rs b/developer-hub/src/reference_docs/signed_extensions.rs new file mode 100644 index 000000000000..28b1426536bc --- /dev/null +++ b/developer-hub/src/reference_docs/signed_extensions.rs @@ -0,0 +1,79 @@ +//! Signed extensions are, briefly, a means for different chains to extend the "basic" extrinsic +//! format with custom data that can be checked by the runtime. +//! +//! # Example +//! +//! Defining a couple of very simple signed extensions looks like the following: +#![doc = docify::embed!("./src/reference_docs/signed_extensions.rs", signed_extensions_example)] + +#[docify::export] +pub mod signed_extensions_example { + use parity_scale_codec::{Decode, Encode}; + use scale_info::TypeInfo; + use sp_runtime::traits::SignedExtension; + + // This doesn't actually check anything, but simply allows + // some arbitrary `u32` to be added to the extrinsic payload + #[derive(Debug, Encode, Decode, Clone, Eq, PartialEq, TypeInfo)] + pub struct AddToPayload(pub u32); + + impl SignedExtension for AddToPayload { + const IDENTIFIER: &'static str = "AddToPayload"; + type AccountId = (); + type Call = (); + type AdditionalSigned = (); + type Pre = (); + + fn additional_signed( + &self, + ) -> Result< + Self::AdditionalSigned, + sp_runtime::transaction_validity::TransactionValidityError, + > { + Ok(()) + } + + fn pre_dispatch( + self, + _who: &Self::AccountId, + _call: &Self::Call, + _info: &sp_runtime::traits::DispatchInfoOf, + _len: usize, + ) -> Result { + Ok(()) + } + } + + // This is the opposite; nothing will be added to the extrinsic payload, + // but the AdditionalSigned type (`1234u32`) will be added to the + // payload to be signed. + #[derive(Debug, Encode, Decode, Clone, Eq, PartialEq, TypeInfo)] + pub struct AddToSignaturePayload; + + impl SignedExtension for AddToSignaturePayload { + const IDENTIFIER: &'static str = "AddToSignaturePayload"; + type AccountId = (); + type Call = (); + type AdditionalSigned = u32; + type Pre = (); + + fn additional_signed( + &self, + ) -> Result< + Self::AdditionalSigned, + sp_runtime::transaction_validity::TransactionValidityError, + > { + Ok(1234) + } + + fn pre_dispatch( + self, + _who: &Self::AccountId, + _call: &Self::Call, + _info: &sp_runtime::traits::DispatchInfoOf, + _len: usize, + ) -> Result { + Ok(()) + } + } +} diff --git a/developer-hub/src/reference_docs/trait_based_programming.rs b/developer-hub/src/reference_docs/trait_based_programming.rs new file mode 100644 index 000000000000..249f4bcbce0f --- /dev/null +++ b/developer-hub/src/reference_docs/trait_based_programming.rs @@ -0,0 +1,229 @@ +//! # Trait-based Programming +//! +//! This document walks you over a peculiar way of using Rust's `trait` items. This pattern is +//! abundantly used within [`frame`] and is therefore paramount important for a smooth transition +//! into it. +//! +//! The rest of this document assumes familiarity with the +//! [Rust book's Advanced Traits](https://doc.rust-lang.org/book/ch19-03-advanced-traits.html) +//! section. +//! Moreover, we use the [`frame::traits::Get`]. +//! +//! First, imagine we are writing a FRAME pallet. We represent this pallet with a `struct Pallet`, +//! and this pallet wants to implement the functionalities of that pallet, for example a simple +//! `transfer` function. For the sake of education, we are interested in having a `MinTransfer` +//! amount, expressed as a [`frame::traits::Get`], which will dictate what is the minimum amount +//! that can be transferred. +//! +//! We can foremost write this as simple as the following snippet: +#![doc = docify::embed!("./src/reference_docs/trait_based_programming.rs", basic)] +//! +//! +//! In this example, we use arbitrary choices for `AccountId`, `Balance` and the `MinTransfer` type. +//! This works great for **one team's purposes** but we have to remember that Substrate and FRAME +//! are written as generic frameworks, intended to be highly configurable. +//! +//! In a broad sense, there are two avenues in exposing configurability: +//! +//! 1. For *values* that need to be generic, for example `MinTransfer`, we attach them to the +//! `Pallet` struct as fields: +//! +//! ``` +//! struct Pallet { +//! min_transfer: u128, +//! } +//! ``` +//! +//! 2. For *types* that need to be generic, we would have to use generic or associated types, such +//! as: +//! +//! ``` +//! struct Pallet { +//! min_transfer: u128, +//! _marker: std::marker::PhantomData, +//! } +//! ``` +//! +//! Substrate and FRAME, for various reasons (performance, correctness, type safety) has opted to +//! use *types* to declare both *values* and *types* as generic. This is the essence of why the +//! `Get` trait exists. +//! +//! This would bring us to the second iteration of the pallet, which would look like: +#![doc = docify::embed!("./src/reference_docs/trait_based_programming.rs", generic)] +//! +//! In this example, we managed to make all 3 of our types generic. Taking the example of the +//! `AccountId`, one should read the above as following: +//! +//! > The `Pallet` does not know what type `AccountId` concretely is, but it knows that it is +//! > something that adheres to being `From<[u8; 32]>`. +//! +//! This method would work, but it suffers from two downsides: +//! +//! 1. It is verbose, each `impl` block would have to reiterate all of the trait bounds. +//! 2. It cannot easily share/inherit generic types. Imagine multiple pallets wanting to be generic +//! over a single `AccountId`. There is no easy way to express that in this model. +//! +//! Finally, this brings us to using traits and associated types on traits to express the above. +//! Trait associated types have the benefit of: +//! +//! 1. Being less verbose, as in effect they can *group multiple `type`s together*. +//! 2. Can inherit from one another by declaring +//! [supertraits](https://doc.rust-lang.org/rust-by-example/trait/supertraits.html). +//! +//! > Interestingly, one downside of associated types is that declaring defaults on them is not +//! > stable yet. In the meantime, we have built our own custom mechanics around declaring defaults +//! for associated types, see [`pallet_default_config_example`]. +//! +//! The last iteration of our code would look like this: +#![doc = docify::embed!("./src/reference_docs/trait_based_programming.rs", trait_based)] +//! +//! Notice how instead of having multiple generics, everything is generic over a single ``, and all types are fetched through `T`, for example `T::AccountId`, `T::MinTransfer`. +//! +//! Finally, imagine all pallets wanting to be generic over `AccountId`. This can be achieved by +//! having individual `trait Configs` declare a shared `trait SystemConfig` as their +//! [supertrait](https://doc.rust-lang.org/rust-by-example/trait/supertraits.html). +#![doc = docify::embed!("./src/reference_docs/trait_based_programming.rs", with_system)] +//! In FRAME, this shared supertrait is [`frame::prelude::frame_system`]. +//! +//! Notice how this made no difference in the syntax of the rest of the code. `T::AccountId` is +//! still a valid type, since `T` implements `Config` and `Config` implies `SystemConfig`, which +//! has a `type AccountId`. +//! +//! Note, in some instances one would need to use what is known as the fully-qualified-syntax to +//! access a type to help the Rust compiler disambiguate. +#![doc = docify::embed!("./src/reference_docs/trait_based_programming.rs", fully_qualified)] +//! +//! This syntax can sometimes become more complicated when you are dealing with nested traits. +//! Consider the following example, in which we fetch the `type Balance` from another trait +//! `CurrencyTrait`. +#![doc = docify::embed!("./src/reference_docs/trait_based_programming.rs", fully_qualified_complicated)] +//! +//! Notice the final `type BalanceOf` and how it is defined. Using such aliases to shorten the +//! length of fully qualified syntax is a common pattern in FRAME. +//! +//! The above example is almost identical to the well-known (and somewhat notorious) `type +//! BalanceOf` that is often used in the context of [`frame::traits::fungible`]. +#![doc = docify::embed!("../substrate/frame/fast-unstake/src/types.rs", BalanceOf)] +//! +//! ## Additional Resources +//! +//! - +//! - [Substrate Seminar - Traits and Generic Types](https://www.youtube.com/watch?v=6cp10jVWNl4) +//! - +#![allow(unused)] + +use frame::traits::Get; + +#[docify::export] +mod basic { + struct Pallet; + + type AccountId = frame::deps::sp_runtime::AccountId32; + type Balance = u128; + type MinTransfer = frame::traits::ConstU128<10>; + + impl Pallet { + fn transfer(_from: AccountId, _to: AccountId, _amount: Balance) { + todo!() + } + } +} + +#[docify::export] +mod generic { + use super::*; + + struct Pallet { + _marker: std::marker::PhantomData<(AccountId, Balance, MinTransfer)>, + } + + impl Pallet + where + Balance: frame::traits::AtLeast32BitUnsigned, + MinTransfer: frame::traits::Get, + AccountId: From<[u8; 32]>, + { + fn transfer(_from: AccountId, _to: AccountId, amount: Balance) { + assert!(amount >= MinTransfer::get()); + unimplemented!(); + } + } +} + +#[docify::export] +mod trait_based { + use super::*; + + trait Config { + type AccountId: From<[u8; 32]>; + type Balance: frame::traits::AtLeast32BitUnsigned; + type MinTransfer: frame::traits::Get; + } + + struct Pallet(std::marker::PhantomData); + impl Pallet { + fn transfer(_from: T::AccountId, _to: T::AccountId, amount: T::Balance) { + assert!(amount >= T::MinTransfer::get()); + unimplemented!(); + } + } +} + +#[docify::export] +mod with_system { + use super::*; + + pub trait SystemConfig { + type AccountId: From<[u8; 32]>; + } + + pub trait Config: SystemConfig { + type Balance: frame::traits::AtLeast32BitUnsigned; + type MinTransfer: frame::traits::Get; + } + + pub struct Pallet(std::marker::PhantomData); + impl Pallet { + fn transfer(_from: T::AccountId, _to: T::AccountId, amount: T::Balance) { + assert!(amount >= T::MinTransfer::get()); + unimplemented!(); + } + } +} + +#[docify::export] +mod fully_qualified { + use super::with_system::*; + + // Simple of using fully qualified syntax. + type AccountIdOf = ::AccountId; +} + +#[docify::export] +mod fully_qualified_complicated { + use super::with_system::*; + + trait CurrencyTrait { + type Balance: frame::traits::AtLeast32BitUnsigned; + fn more_stuff() {} + } + + trait Config: SystemConfig { + type Currency: CurrencyTrait; + } + + struct Pallet(std::marker::PhantomData); + impl Pallet { + fn transfer( + _from: T::AccountId, + _to: T::AccountId, + _amount: <::Currency as CurrencyTrait>::Balance, + ) { + unimplemented!(); + } + } + + /// A common pattern in FRAME. + type BalanceOf = <::Currency as CurrencyTrait>::Balance; +} diff --git a/developer-hub/src/reference_docs/wasm_memory.rs b/developer-hub/src/reference_docs/wasm_memory.rs new file mode 100644 index 000000000000..4f4cda31094e --- /dev/null +++ b/developer-hub/src/reference_docs/wasm_memory.rs @@ -0,0 +1,7 @@ +//! # WASM Memory Limitations. +//! +//! Notes: +//! +//! - Stack: Need to use `Box<_>` +//! - Heap: Substrate imposes a limit. PvF execution has its own limits +//! - Heap: There is also a maximum amount that a single allocation can have. diff --git a/developer-hub/src/reference_docs/wasm_meta_protocol.rs b/developer-hub/src/reference_docs/wasm_meta_protocol.rs new file mode 100644 index 000000000000..f45f4c8fa3c7 --- /dev/null +++ b/developer-hub/src/reference_docs/wasm_meta_protocol.rs @@ -0,0 +1,113 @@ +//! # WASM Meta Protocol +//! +//! All Substrate based chains adhere to a unique architectural design novel to the Polkadot +//! ecosystem. We refer to this design as the "WASM Meta Protocol". +//! +//! Consider the fact that a traditional blockchain software is usually a monolithic artifact. +//! Upgrading any part of the system implies upgrading the entire system. This has historically led +//! to cumbersome forkful upgrades to be the status quo in the blockchain ecosystem. +//! +//! Moreover, the idea of "storing code in the state" is explored in the context of smart contracts +//! platforms, but has not been expanded further. +//! +//! Substrate mixes these two ideas together, and takes the novel approach of storing the +//! blockchain's main "state transition function" in the main blockchain state, in the same fashion +//! that a smart contract platform stores the code of individual contracts in its state. As noted in +//! [`crate::reference_docs::blockchain_state_machines`], this state transition function is called +//! the **Runtime**, and WASM is chosen as the bytecode. The Runtime is stored under a special key +//! in the state (see +//! [`sp_core::storage::well_known_keys`](../../../sp_core/index.html)) and can be +//! updated as a part of the state transition function's execution, just like a user's account +//! balance can be updated. +//! +//! > Note that while we drew an analogy between smart contracts and runtimes in the above, there +//! > are fundamental differences between the two, explained in +//! > [`crate::reference_docs::runtime_vs_smart_contract`]. +//! +//! The rest of the system that is NOT the state transition function is called the **node**, and +//! is a normal binary that is compiled from Rust to different hardware targets. +//! +//! This design enables all Substrate-based chains to be fork-less-ly upgradeable, because the +//! Runtime can be updates on the fly, within the execution of a block, and the node is (for the +//! most part) oblivious to the change that is happening. +//! +//! Therefore, the high-level architecture of a any Substrate-based chain can be demonstrated as +//! follows: +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/substrate_simple.mmd")] +//! +//! The node and the runtime need to communicate. This is done through two concepts: +//! +//! 1. **Host functions**: a way for the (WASM) runtime to talk to the node. All host functions are +//! defined in [`sp_io`]. For example, [`sp_io::storage`] are the set of host functions that +//! allow the runtime to read and write data to the on-chain state. +//! 2. **Runtime APIs**: a way for the node to talk to the WASM runtime. Runtime APIs are defined +//! using macros and utilities in [`sp_api`]. For example, [`sp_api::Core`] is the most +//! fundamental runtime API that any blockchain must implement in order to be able to (re) +//! execute blocks. +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/substrate_client_runtime.mmd")] +//! +//! A runtime must have a set of runtime APIs in order to have any meaningful blockchain +//! functionality, but it can also expose more APIs. See TODO as an example of how to add custom +//! runtime APIs to your FRAME-based runtime. +//! +//! Similarly, for a runtime to be "compatible" with a node, the node must implement the full set of +//! host functions that the runtime at any point in time requires. Given the fact that a runtime can +//! evolve in time, and a blockchain node (typically) wishes to be capable of re-executing all the +//! previous blocks, this means that a node must always maintain support for the old host functions. +//! This also implies that adding a new host function is a big commitment and should be done with +//! care. This is why, for example, adding a new host function to Polkadot always requires an RFC. +//! +//! ## Node vs. Runtime +//! +//! A common question is: which components of the system end up being part of the node, and which +//! ones of the runtime? +//! +//! Recall from [`crate::reference_docs::blockchain_state_machines`] that the runtime is the state +//! transition function. Anything that needs to influence how your blockchain's state is updated, +//! should be a part of the runtime. For example, the logic around currency, governance, identity or +//! any other application-specific logic that has to do with the state is part of the runtime. +//! +//! Anything that does not have to do with the state-transition function and will only +//! facilitate/enable it is part of the node. For example, the database, networking, and even +//! consensus algorithm are all node-side components. +//! +//! > The consensus is to your runtime what HTTP is to a web-application. It is the underlying +//! > engine that enables trustless execution of the runtime in a distributed manner whilst +//! > maintaining a canonical outcome of that execution. +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/substrate_with_frame.mmd")] +//! +//! ## State +//! +//! From the previous sections, we know that the a database component is part of the node, not the +//! runtime. We also hinted that a set of host functions ([`sp_io::storage`]) are how the runtime +//! issues commands to the node to read/write to the state. Let's dive deeper into this. +//! +//! The state of the blockchain, what we seek to come to consensus about, is indeed *kept* in the +//! node side. Nonetheless, the runtime is the only component that: +//! +//! 1. Can update the state. +//! 2. Can fully interpret the state. +//! +//! In fact, [`sp_core::storage::well_known_keys`] are the only state keys that the node side is +//! aware of. The rest of the state, including what logic the runtime has, what balance each user +//! has and such are all only comprehensible to the runtime. +#![doc = simple_mermaid::mermaid!("../../../docs/mermaid/state.mmd")] +//! +//! In the above diagram, all of the state keys and values are opaque bytes to the node. The node +//! does not know what they mean, and it does not now what is the type of the corresponding value +//! (e.g. if it is a number of a vector). Contrary, the runtime knows both the meaning of their +//! keys, and the type of the values. +//! +//! This opaque-ness is the fundamental reason why Substrate-based chains can fork-less-ly upgrade: +//! because the node side code is kept oblivious to all of the details of the state transition +//! function. Therefore, the state transition function can freely upgrade without the node needing +//! to know. +//! +//! ## Native Runtime +//! +//! TODO +//! +//! +//! ## Example: Block Execution. +//! +//! TODO diff --git a/docs/mermaid/IA.mmd b/docs/mermaid/IA.mmd new file mode 100644 index 000000000000..8fcb74fa0a91 --- /dev/null +++ b/docs/mermaid/IA.mmd @@ -0,0 +1,14 @@ +flowchart + parity[paritytech.github.io] --> devhub[developer_hub] + + devhub --> polkadot_sdk + devhub --> reference_docs + devhub --> tutorial + + polkadot_sdk --> substrate + polkadot_sdk --> frame + polkadot_sdk --> cumulus + polkadot_sdk --> polkadot + polkadot_sdk --> xcm + + diff --git a/docs/mermaid/extrinsics.mmd b/docs/mermaid/extrinsics.mmd new file mode 100644 index 000000000000..4afd4ab8f755 --- /dev/null +++ b/docs/mermaid/extrinsics.mmd @@ -0,0 +1,5 @@ +flowchart TD + E(Extrinsic) ---> I(Inherent); + E --> T(Transaction) + T --> ST("Signed (aka. Transaction)") + T --> UT(Unsigned) diff --git a/docs/mermaid/polkadot_sdk_parachain.mmd b/docs/mermaid/polkadot_sdk_parachain.mmd new file mode 100644 index 000000000000..3f38fce046c2 --- /dev/null +++ b/docs/mermaid/polkadot_sdk_parachain.mmd @@ -0,0 +1,11 @@ +flowchart LR + subgraph Parachain[A Polkadot Parachain] + ParachainNode[Parachain Node] + ParachainRuntime[Parachain Runtime] + end + + FRAME -.-> ParachainRuntime + Substrate[Substrate Node Libraries] -.-> ParachainNoe + + CumulusC[Cumulus Node Libraries] -.-> ParachainNode + CumulusR[Cumulus Runtime Libraries] -.-> ParachainRuntime diff --git a/docs/mermaid/polkadot_sdk_polkadot.mmd b/docs/mermaid/polkadot_sdk_polkadot.mmd new file mode 100644 index 000000000000..3326cc593839 --- /dev/null +++ b/docs/mermaid/polkadot_sdk_polkadot.mmd @@ -0,0 +1,10 @@ +flowchart LR + + subgraph Polkadot[The Polkadot Relay Chain] + PolkadotNode[Polkadot Node] + PolkadotRuntime[Polkadot Runtime] + end + + FRAME -.-> PolkadotRuntime + Substrate[Substrate Node Libraries] -.-> PolkadotNode + diff --git a/docs/mermaid/polkadot_sdk_substrate.mmd b/docs/mermaid/polkadot_sdk_substrate.mmd new file mode 100644 index 000000000000..dfaf20d241f8 --- /dev/null +++ b/docs/mermaid/polkadot_sdk_substrate.mmd @@ -0,0 +1,8 @@ +flowchart LR + subgraph SubstrateChain[A Substrate-based blockchain] + Node + Runtime + end + + FRAME -.-> Runtime + Substrate[Substrate Node Libraries] -.-> Node diff --git a/docs/mermaid/state.mmd b/docs/mermaid/state.mmd new file mode 100644 index 000000000000..c72ecbfd1568 --- /dev/null +++ b/docs/mermaid/state.mmd @@ -0,0 +1,16 @@ +flowchart TB + subgraph Node[Node's View Of The State 🙈] + direction LR + 0x1234 --> 0x2345 + 0x3456 --> 0x4567 + 0x5678 --> 0x6789 + :code --> code[wasm code] + end + + subgraph Runtime[Runtime's View Of The State 🙉] + direction LR + ab[alice's balance] --> abv[known value] + bb[bob's balance] --> bbv[known value] + cb[charlie's balance] --> cbv[known value] + c2[:code] --> c22[wasm code] + end diff --git a/docs/mermaid/stf.mmd b/docs/mermaid/stf.mmd new file mode 100644 index 000000000000..dd6c7c36de66 --- /dev/null +++ b/docs/mermaid/stf.mmd @@ -0,0 +1,21 @@ +flowchart LR + %%{init: {'flowchart' : {'curve' : 'linear'}}}%% + subgraph BData[Blockchain Database] + direction LR + BN[Block N] -.-> BN1[Block N+1] + end + + subgraph SData[State Database] + direction LR + SN[State N] -.-> SN1[State N+1] -.-> SN2[State N+2] + end + + BN --> STFN[STF] + SN --> STFN[STF] + STFN[STF] --> SN1 + + BN1 --> STFN1[STF] + SN1 --> STFN1[STF] + STFN1[STF] --> SN2 + + diff --git a/docs/mermaid/stf_simple.mmd b/docs/mermaid/stf_simple.mmd new file mode 100644 index 000000000000..5db20cf6156c --- /dev/null +++ b/docs/mermaid/stf_simple.mmd @@ -0,0 +1,4 @@ +flowchart LR + B[Block] --> STF + S[State] --> STF + STF --> NS[New State] diff --git a/docs/mermaid/substrate_client_runtime.mmd b/docs/mermaid/substrate_client_runtime.mmd index 23c3f849224a..caab2b623028 100644 --- a/docs/mermaid/substrate_client_runtime.mmd +++ b/docs/mermaid/substrate_client_runtime.mmd @@ -1,10 +1,12 @@ graph TB subgraph Substrate direction LR - subgraph Client + subgraph Node end + subgraph Runtime end - Client --runtime-api--> Runtime - Runtime --host-functions--> Client + + Node --runtime-api--> Runtime + Runtime --host-functions--> Node end diff --git a/docs/mermaid/substrate_dev.mmd b/docs/mermaid/substrate_dev.mmd new file mode 100644 index 000000000000..fc331ce311fe --- /dev/null +++ b/docs/mermaid/substrate_dev.mmd @@ -0,0 +1,2 @@ +flowchart LR + T[Using a Template] --> P[Writing Your Own FRAME-Based Pallet] --> C[Custom Node] diff --git a/docs/mermaid/substrate_simple.mmd b/docs/mermaid/substrate_simple.mmd index 475d8be5ef81..a752eaba625f 100644 --- a/docs/mermaid/substrate_simple.mmd +++ b/docs/mermaid/substrate_simple.mmd @@ -1,7 +1,7 @@ graph TB subgraph Substrate direction LR - subgraph Client + subgraph Node end subgraph Runtime end diff --git a/docs/mermaid/substrate_with_frame.mmd b/docs/mermaid/substrate_with_frame.mmd index 12d072a3360c..173c1757b955 100644 --- a/docs/mermaid/substrate_with_frame.mmd +++ b/docs/mermaid/substrate_with_frame.mmd @@ -1,7 +1,7 @@ graph TB subgraph Substrate direction LR - subgraph Client + subgraph Node Database Networking Consensus @@ -15,6 +15,6 @@ subgraph Substrate Identity end end - Client --runtime-api--> Runtime - Runtime --host-functions--> Client + Node --runtime-api--> Runtime + Runtime --host-functions--> Node end diff --git a/polkadot/node/network/protocol/src/request_response/mod.rs b/polkadot/node/network/protocol/src/request_response/mod.rs index 96f7adeb29ba..2df3021343df 100644 --- a/polkadot/node/network/protocol/src/request_response/mod.rs +++ b/polkadot/node/network/protocol/src/request_response/mod.rs @@ -248,8 +248,8 @@ impl Protocol { name, fallback_names, max_request_size: 1_000, - /// Responses are just confirmation, in essence not even a bit. So 100 seems - /// plenty. + // Responses are just confirmation, in essence not even a bit. So 100 seems + // plenty. max_response_size: 100, request_timeout: DISPUTE_REQUEST_TIMEOUT, inbound_queue: tx, diff --git a/polkadot/node/primitives/src/lib.rs b/polkadot/node/primitives/src/lib.rs index dab72bb2a5ed..be62145b9991 100644 --- a/polkadot/node/primitives/src/lib.rs +++ b/polkadot/node/primitives/src/lib.rs @@ -442,7 +442,7 @@ pub struct CollationSecondedSignal { pub relay_parent: Hash, /// The statement about seconding the collation. /// - /// Anything else than [`Statement::Seconded`](Statement::Seconded) is forbidden here. + /// Anything else than [`Statement::Seconded`] is forbidden here. pub statement: SignedFullStatement, } diff --git a/substrate/Cargo.toml b/substrate/Cargo.toml deleted file mode 100644 index 8fb1be5821ba..000000000000 --- a/substrate/Cargo.toml +++ /dev/null @@ -1,30 +0,0 @@ -[package] -name = "substrate" -description = "Next-generation framework for blockchain innovation" -license = "GPL-3.0-or-later WITH Classpath-exception-2.0" -homepage = "https://substrate.io" -repository.workspace = true -authors.workspace = true -edition.workspace = true -version = "1.0.0" -publish = false - -# The dependencies are only needed for docs. -[dependencies] -simple-mermaid = { git = "https://github.com/kianenigma/simple-mermaid.git", rev = "e48b187bcfd5cc75111acd9d241f1bd36604344b" } - -subkey = { path = "bin/utils/subkey" } -chain-spec-builder = { package = "staging-chain-spec-builder", path = "bin/utils/chain-spec-builder" } - -sc-service = { path = "client/service" } -sc-chain-spec = { path = "client/chain-spec" } -sc-cli = { path = "client/cli" } -sc-consensus-aura = { path = "client/consensus/aura" } -sc-consensus-babe = { path = "client/consensus/babe" } -sc-consensus-grandpa = { path = "client/consensus/grandpa" } -sc-consensus-beefy = { path = "client/consensus/beefy" } -sc-consensus-manual-seal = { path = "client/consensus/manual-seal" } -sc-consensus-pow = { path = "client/consensus/pow" } - -sp-runtime = { path = "primitives/runtime" } -frame-support = { path = "frame/support" } diff --git a/substrate/client/allocator/src/lib.rs b/substrate/client/allocator/src/lib.rs index e50d7d54c8e9..70ed764bef8c 100644 --- a/substrate/client/allocator/src/lib.rs +++ b/substrate/client/allocator/src/lib.rs @@ -18,7 +18,7 @@ //! Collection of allocator implementations. //! //! This crate provides the following allocator implementations: -//! - A freeing-bump allocator: [`FreeingBumpHeapAllocator`](freeing_bump::FreeingBumpHeapAllocator) +//! - A freeing-bump allocator: [`FreeingBumpHeapAllocator`] #![warn(missing_docs)] diff --git a/substrate/client/executor/src/lib.rs b/substrate/client/executor/src/lib.rs index 6ee0ab3512ac..25bad81938f3 100644 --- a/substrate/client/executor/src/lib.rs +++ b/substrate/client/executor/src/lib.rs @@ -58,7 +58,7 @@ pub use sc_executor_wasmtime::InstantiationStrategy as WasmtimeInstantiationStra /// Extracts the runtime version of a given runtime code. pub trait RuntimeVersionOf { - /// Extract [`RuntimeVersion`](sp_version::RuntimeVersion) of the given `runtime_code`. + /// Extract [`RuntimeVersion`] of the given `runtime_code`. fn runtime_version( &self, ext: &mut dyn Externalities, diff --git a/substrate/frame/balances/src/lib.rs b/substrate/frame/balances/src/lib.rs index d518f933df8d..843bc351494e 100644 --- a/substrate/frame/balances/src/lib.rs +++ b/substrate/frame/balances/src/lib.rs @@ -49,8 +49,7 @@ //! - **Total Issuance:** The total number of units in existence in a system. //! //! - **Reaping an account:** The act of removing an account by resetting its nonce. Happens after -//! its -//! total balance has become zero (or, strictly speaking, less than the Existential Deposit). +//! its total balance has become zero (or, strictly speaking, less than the Existential Deposit). //! //! - **Free Balance:** The portion of a balance that is not reserved. The free balance is the only //! balance that matters for most operations. @@ -59,24 +58,23 @@ //! Reserved balance can still be slashed, but only after all the free balance has been slashed. //! //! - **Imbalance:** A condition when some funds were credited or debited without equal and opposite -//! accounting -//! (i.e. a difference between total issuance and account balances). Functions that result in an -//! imbalance will return an object of the `Imbalance` trait that can be managed within your runtime -//! logic. (If an imbalance is simply dropped, it should automatically maintain any book-keeping -//! such as total issuance.) +//! accounting (i.e. a difference between total issuance and account balances). Functions that +//! result in an imbalance will return an object of the `Imbalance` trait that can be managed within +//! your runtime logic. (If an imbalance is simply dropped, it should automatically maintain any +//! book-keeping such as total issuance.) //! //! - **Lock:** A freeze on a specified amount of an account's free balance until a specified block -//! number. Multiple -//! locks always operate over the same funds, so they "overlay" rather than "stack". +//! number. Multiple locks always operate over the same funds, so they "overlay" rather than +//! "stack". //! //! ### Implementations //! //! The Balances pallet provides implementations for the following traits. If these traits provide //! the functionality that you need, then you can avoid coupling with the Balances pallet. //! -//! - [`Currency`](frame_support::traits::Currency): Functions for dealing with a +//! - [`Currency`]: Functions for dealing with a //! fungible assets system. -//! - [`ReservableCurrency`](frame_support::traits::ReservableCurrency): +//! - [`ReservableCurrency`] //! - [`NamedReservableCurrency`](frame_support::traits::NamedReservableCurrency): //! Functions for dealing with assets that can be reserved from an account. //! - [`LockableCurrency`](frame_support::traits::LockableCurrency): Functions for @@ -105,7 +103,7 @@ //! ``` //! use frame_support::traits::Currency; //! # pub trait Config: frame_system::Config { -//! # type Currency: Currency; +//! # type Currency: Currency; //! # } //! //! pub type BalanceOf = <::Currency as Currency<::AccountId>>::Balance; @@ -120,26 +118,26 @@ //! use frame_support::traits::{WithdrawReasons, LockableCurrency}; //! use sp_runtime::traits::Bounded; //! pub trait Config: frame_system::Config { -//! type Currency: LockableCurrency>; +//! type Currency: LockableCurrency>; //! } //! # struct StakingLedger { -//! # stash: ::AccountId, -//! # total: <::Currency as frame_support::traits::Currency<::AccountId>>::Balance, -//! # phantom: std::marker::PhantomData, +//! # stash: ::AccountId, +//! # total: <::Currency as frame_support::traits::Currency<::AccountId>>::Balance, +//! # phantom: std::marker::PhantomData, //! # } //! # const STAKING_ID: [u8; 8] = *b"staking "; //! //! fn update_ledger( -//! controller: &T::AccountId, -//! ledger: &StakingLedger +//! controller: &T::AccountId, +//! ledger: &StakingLedger //! ) { -//! T::Currency::set_lock( -//! STAKING_ID, -//! &ledger.stash, -//! ledger.total, -//! WithdrawReasons::all() -//! ); -//! // >::insert(controller, ledger); // Commented out as we don't have access to Staking's storage here. +//! T::Currency::set_lock( +//! STAKING_ID, +//! &ledger.stash, +//! ledger.total, +//! WithdrawReasons::all() +//! ); +//! // >::insert(controller, ledger); // Commented out as we don't have access to Staking's storage here. //! } //! # fn main() {} //! ``` diff --git a/substrate/frame/bounties/src/migrations/v4.rs b/substrate/frame/bounties/src/migrations/v4.rs index 936bac117008..4e6ba9344816 100644 --- a/substrate/frame/bounties/src/migrations/v4.rs +++ b/substrate/frame/bounties/src/migrations/v4.rs @@ -110,7 +110,7 @@ pub fn migrate< } /// Some checks prior to migration. This can be linked to -/// [`frame_support::traits::OnRuntimeUpgrade::pre_upgrade`] for further testing. +/// `frame_support::traits::OnRuntimeUpgrade::pre_upgrade` for further testing. /// /// Panics if anything goes wrong. pub fn pre_migration>( @@ -164,7 +164,7 @@ pub fn pre_migration>( diff --git a/substrate/frame/collective/src/migrations/v4.rs b/substrate/frame/collective/src/migrations/v4.rs index b3326b4251c9..300dff23d8eb 100644 --- a/substrate/frame/collective/src/migrations/v4.rs +++ b/substrate/frame/collective/src/migrations/v4.rs @@ -76,7 +76,7 @@ pub fn migrate>(old_pallet_name: N) { @@ -104,7 +104,7 @@ pub fn pre_migrate>(old_p } /// Some checks for after migration. This can be linked to -/// [`frame_support::traits::OnRuntimeUpgrade::post_upgrade`] for further testing. +/// `frame_support::traits::OnRuntimeUpgrade::post_upgrade` for further testing. /// /// Panics if anything goes wrong. pub fn post_migrate>(old_pallet_name: N) { diff --git a/substrate/frame/election-provider-multi-phase/src/mock.rs b/substrate/frame/election-provider-multi-phase/src/mock.rs index 15a50165ff37..7bdd329d9b04 100644 --- a/substrate/frame/election-provider-multi-phase/src/mock.rs +++ b/substrate/frame/election-provider-multi-phase/src/mock.rs @@ -80,11 +80,7 @@ frame_election_provider_support::generate_solution_type!( /// All events of this pallet. pub(crate) fn multi_phase_events() -> Vec> { - System::events() - .into_iter() - .map(|r| r.event) - .filter_map(|e| if let RuntimeEvent::MultiPhase(inner) = e { Some(inner) } else { None }) - .collect::>() + System::read_events_for_pallet::>() } /// To from `now` to block `n`. diff --git a/substrate/frame/elections-phragmen/src/migrations/v4.rs b/substrate/frame/elections-phragmen/src/migrations/v4.rs index 7e946371f5ca..e78465cd618d 100644 --- a/substrate/frame/elections-phragmen/src/migrations/v4.rs +++ b/substrate/frame/elections-phragmen/src/migrations/v4.rs @@ -69,7 +69,7 @@ pub fn migrate>(new_pallet_name: N) -> Weight { } /// Some checks prior to migration. This can be linked to -/// [`frame_support::traits::OnRuntimeUpgrade::pre_upgrade`] for further testing. +/// `frame_support::traits::OnRuntimeUpgrade::pre_upgrade` for further testing. /// /// Panics if anything goes wrong. pub fn pre_migration>(new: N) { @@ -97,7 +97,7 @@ pub fn pre_migration>(new: N) { } /// Some checks for after migration. This can be linked to -/// [`frame_support::traits::OnRuntimeUpgrade::post_upgrade`] for further testing. +/// `frame_support::traits::OnRuntimeUpgrade::post_upgrade` for further testing. /// /// Panics if anything goes wrong. pub fn post_migration() { diff --git a/substrate/frame/elections-phragmen/src/migrations/v5.rs b/substrate/frame/elections-phragmen/src/migrations/v5.rs index 6fac923703fe..6e360aa8b8c1 100644 --- a/substrate/frame/elections-phragmen/src/migrations/v5.rs +++ b/substrate/frame/elections-phragmen/src/migrations/v5.rs @@ -71,7 +71,7 @@ pub fn pre_migrate_fn(to_migrate: Vec) -> Box() { diff --git a/substrate/frame/examples/kitchensink/src/lib.rs b/substrate/frame/examples/kitchensink/src/lib.rs index 89759dd0bf63..18429bc967d7 100644 --- a/substrate/frame/examples/kitchensink/src/lib.rs +++ b/substrate/frame/examples/kitchensink/src/lib.rs @@ -292,9 +292,8 @@ pub mod pallet { } } - /// Allows you to define an enum on the pallet which will then instruct - /// `construct_runtime` to amalgamate all similarly-named enums from other - /// pallets into an aggregate enum. + /// Allows you to define an enum on the pallet which will then instruct `construct_runtime` to + /// amalgamate all similarly-named enums from other pallets into an aggregate enum. #[pallet::composite_enum] pub enum HoldReason { Staking, diff --git a/substrate/frame/fast-unstake/src/types.rs b/substrate/frame/fast-unstake/src/types.rs index 15d0a327e917..3fb5720861fa 100644 --- a/substrate/frame/fast-unstake/src/types.rs +++ b/substrate/frame/fast-unstake/src/types.rs @@ -39,6 +39,7 @@ impl frame_support::traits::Get for MaxChecking { } } +#[docify::export] pub(crate) type BalanceOf = <::Currency as Currency<::AccountId>>::Balance; /// An unstake request. diff --git a/substrate/frame/grandpa/src/migrations/v4.rs b/substrate/frame/grandpa/src/migrations/v4.rs index 8604296b6e57..9daa818071f7 100644 --- a/substrate/frame/grandpa/src/migrations/v4.rs +++ b/substrate/frame/grandpa/src/migrations/v4.rs @@ -63,7 +63,7 @@ pub fn migrate>(new_pallet_name: N) -> Weight { } /// Some checks prior to migration. This can be linked to -/// [`frame_support::traits::OnRuntimeUpgrade::pre_upgrade`] for further testing. +/// `frame_support::traits::OnRuntimeUpgrade::pre_upgrade` for further testing. /// /// Panics if anything goes wrong. pub fn pre_migration>(new: N) { @@ -99,7 +99,7 @@ pub fn pre_migration>(new: N) { } /// Some checks for after migration. This can be linked to -/// [`frame_support::traits::OnRuntimeUpgrade::post_upgrade`] for further testing. +/// `frame_support::traits::OnRuntimeUpgrade::post_upgrade` for further testing. /// /// Panics if anything goes wrong. pub fn post_migration() { diff --git a/substrate/frame/membership/src/migrations/v4.rs b/substrate/frame/membership/src/migrations/v4.rs index 38e97af51a09..9b80aca86847 100644 --- a/substrate/frame/membership/src/migrations/v4.rs +++ b/substrate/frame/membership/src/migrations/v4.rs @@ -77,7 +77,7 @@ pub fn migrate>(old_pallet_name: N, new_pallet_name: N) { @@ -105,7 +105,7 @@ pub fn pre_migrate>(old_pallet_name: N, new_ } /// Some checks for after migration. This can be linked to -/// [`frame_support::traits::OnRuntimeUpgrade::post_upgrade`] for further testing. +/// `frame_support::traits::OnRuntimeUpgrade::post_upgrade` for further testing. /// /// Panics if anything goes wrong. pub fn post_migrate>(old_pallet_name: N, new_pallet_name: N) { diff --git a/substrate/frame/session/src/historical/offchain.rs b/substrate/frame/session/src/historical/offchain.rs index 1b4d53b74b45..95f4d762949e 100644 --- a/substrate/frame/session/src/historical/offchain.rs +++ b/substrate/frame/session/src/historical/offchain.rs @@ -17,13 +17,11 @@ //! Off-chain logic for creating a proof based data provided by on-chain logic. //! -//! Validator-set extracting an iterator from an off-chain worker stored list containing -//! historical validator-sets. -//! Based on the logic of historical slashing, but the validation is done off-chain. +//! Validator-set extracting an iterator from an off-chain worker stored list containing historical +//! validator-sets. Based on the logic of historical slashing, but the validation is done off-chain. //! Use [`fn store_current_session_validator_set_to_offchain()`](super::onchain) to store the -//! required data to the offchain validator set. -//! This is used in conjunction with [`ProvingTrie`](super::ProvingTrie) and -//! the off-chain indexing API. +//! required data to the offchain validator set. This is used in conjunction with [`ProvingTrie`] +//! and the off-chain indexing API. use sp_runtime::{ offchain::storage::{MutateStorageError, StorageRetrievalError, StorageValueRef}, diff --git a/substrate/frame/src/lib.rs b/substrate/frame/src/lib.rs index 1a8350405a89..59cd92378888 100644 --- a/substrate/frame/src/lib.rs +++ b/substrate/frame/src/lib.rs @@ -15,10 +15,6 @@ // See the License for the specific language governing permissions and // limitations under the License. -//! > Made for [![polkadot]](https://polkadot.network) -//! -//! [polkadot]: https://img.shields.io/badge/polkadot-E6007A?style=for-the-badge&logo=polkadot&logoColor=white -//! //! # FRAME //! //! ```no_compile @@ -34,14 +30,21 @@ //! > **F**ramework for **R**untime **A**ggregation of **M**odularized **E**ntities: Substrate's //! > State Transition Function (Runtime) Framework. //! +//! ## Documentation +//! +//! See [`polkadot_sdk::frame`](../developer_hub/polkadot_sdk/frame_runtime/index.html). +//! //! ## Warning: Experimental //! //! This crate and all of its content is experimental, and should not yet be used in production. //! -//! ## Getting Started +//! ## Underlying dependencies //! -//! TODO: link to `developer_hub::polkadot_sdk::frame`. The `developer_hub` hasn't been published -//! yet, this can be updated once it is linkable. +//! This crate is an amalgamation of multiple other crates that are often used together to compose a +//! pallet. It is not necessary to use it, and it may fall short for certain purposes. +//! +//! In short, this crate only re-exports types and traits from multiple sources. All of these +//! sources are listed (and re-exported again) in [`deps`]. #![cfg_attr(not(feature = "std"), no_std)] #![cfg(feature = "experimental")] @@ -54,9 +57,19 @@ /// `#[pallet::bar]` inside the mod. pub use frame_support::pallet; +pub use frame_support::pallet_macros::{import_section, pallet_section}; + /// The logging library of the runtime. Can normally be the classic `log` crate. pub use log; +/// A list of all macros used within the main [`pallet`] macro. +/// +/// Note: All of these macros are "stubs" and not really usable outside `#[pallet] mod pallet { .. +/// }`. They are mainly provided for documentation and IDE support. +pub mod pallet_macros { + pub use frame_support::{derive_impl, pallet, pallet_macros::*}; +} + /// The main prelude of FRAME. /// /// This prelude should almost always be the first line of code in any pallet or runtime. @@ -78,9 +91,6 @@ pub mod prelude { /// Pallet prelude of `frame-support`. /// /// Note: this needs to revised once `frame-support` evolves. - // `frame-support` will be break down https://github.com/paritytech/polkadot-sdk/issues/127 and its reexports will - // most likely change. These wildcard reexportings can be optimized once `frame-support` has - // changed. #[doc(no_inline)] pub use frame_support::pallet_prelude::*; @@ -156,6 +166,9 @@ pub mod runtime { /// Types to define your runtime version. pub use sp_version::{create_runtime_str, runtime_version, RuntimeVersion}; + /// Macro to implement runtime APIs. + pub use sp_api::impl_runtime_apis; + #[cfg(feature = "std")] pub use sp_version::NativeVersion; } @@ -180,9 +193,6 @@ pub mod runtime { pub use sp_inherents::{CheckInherentsResult, InherentData}; pub use sp_runtime::ApplyExtrinsicResult; - /// Macro to implement runtime APIs. - pub use sp_api::impl_runtime_apis; - pub use frame_system_rpc_runtime_api::*; pub use sp_api::{self, *}; pub use sp_block_builder::*; diff --git a/substrate/frame/support/procedural/src/lib.rs b/substrate/frame/support/procedural/src/lib.rs index ec4118918856..682b135fa143 100644 --- a/substrate/frame/support/procedural/src/lib.rs +++ b/substrate/frame/support/procedural/src/lib.rs @@ -1097,8 +1097,11 @@ pub fn weight(_: TokenStream, _: TokenStream) -> TokenStream { pallet_macro_stub() } -/// Compact encoding for arguments can be achieved via `#[pallet::compact]`. The function must -/// return a `DispatchResultWithPostInfo` or `DispatchResult`. +/// +/// --- +/// +/// **Rust-Analyzer users**: See the documentation of the Rust item in +/// [`frame_support::pallet_macros::call`](../../frame_support/pallet_macros/attr.call.html). #[proc_macro_attribute] pub fn compact(_: TokenStream, _: TokenStream) -> TokenStream { pallet_macro_stub() @@ -1108,7 +1111,7 @@ pub fn compact(_: TokenStream, _: TokenStream) -> TokenStream { /// --- /// /// **Rust-Analyzer users**: See the documentation of the Rust item in -/// `frame_support::pallet_macros::call`. +/// [`frame_support::pallet_macros::call`](../../frame_support/pallet_macros/attr.call.html). #[proc_macro_attribute] pub fn call(_: TokenStream, _: TokenStream) -> TokenStream { pallet_macro_stub() @@ -1117,41 +1120,10 @@ pub fn call(_: TokenStream, _: TokenStream) -> TokenStream { /// Each dispatchable may also be annotated with the `#[pallet::call_index($idx)]` attribute, /// which explicitly defines the codec index for the dispatchable function in the `Call` enum. /// -/// All call indexes start from 0, until it encounters a dispatchable function with a defined -/// call index. The dispatchable function that lexically follows the function with a defined -/// call index will have that call index, but incremented by 1, e.g. if there are 3 -/// dispatchable functions `fn foo`, `fn bar` and `fn qux` in that order, and only `fn bar` -/// has a call index of 10, then `fn qux` will have an index of 11, instead of 1. -/// -/// All arguments must implement [`Debug`], [`PartialEq`], [`Eq`], `Decode`, `Encode`, and -/// [`Clone`]. For ease of use, bound by the trait `frame_support::pallet_prelude::Member`. -/// -/// If no `#[pallet::call]` exists, then a default implementation corresponding to the -/// following code is automatically generated: -/// -/// ```ignore -/// #[pallet::call] -/// impl Pallet {} -/// ``` -/// -/// **WARNING**: modifying dispatchables, changing their order, removing some, etc., must be -/// done with care. Indeed this will change the outer runtime call type (which is an enum with -/// one variant per pallet), this outer runtime call can be stored on-chain (e.g. in -/// `pallet-scheduler`). Thus migration might be needed. To mitigate against some of this, the -/// `#[pallet::call_index($idx)]` attribute can be used to fix the order of the dispatchable so -/// that the `Call` enum encoding does not change after modification. As a general rule of -/// thumb, it is therefore adventageous to always add new calls to the end so you can maintain -/// the existing order of calls. -/// -/// ### Macro expansion -/// -/// The macro creates an enum `Call` with one variant per dispatchable. This enum implements: -/// [`Clone`], [`Eq`], [`PartialEq`], [`Debug`] (with stripped implementation in `not("std")`), -/// `Encode`, `Decode`, `GetDispatchInfo`, `GetCallName`, `GetCallIndex` and -/// `UnfilteredDispatchable`. +/// --- /// -/// The macro implements the `Callable` trait on `Pallet` and a function `call_functions` -/// which returns the dispatchable metadata. +/// **Rust-Analyzer users**: See the documentation of the Rust item in +/// [`frame_support::pallet_macros::call`](../../frame_support/pallet_macros/attr.call.html). #[proc_macro_attribute] pub fn call_index(_: TokenStream, _: TokenStream) -> TokenStream { pallet_macro_stub() diff --git a/substrate/frame/support/procedural/src/pallet/parse/call.rs b/substrate/frame/support/procedural/src/pallet/parse/call.rs index 519e1e618954..f78f2baa9d10 100644 --- a/substrate/frame/support/procedural/src/pallet/parse/call.rs +++ b/substrate/frame/support/procedural/src/pallet/parse/call.rs @@ -26,6 +26,7 @@ use syn::{spanned::Spanned, ExprClosure}; mod keyword { syn::custom_keyword!(Call); syn::custom_keyword!(OriginFor); + syn::custom_keyword!(RuntimeOrigin); syn::custom_keyword!(weight); syn::custom_keyword!(call_index); syn::custom_keyword!(compact); @@ -158,10 +159,10 @@ impl syn::parse::Parse for ArgAttrIsCompact { } } -/// Check the syntax is `OriginFor` or `&OriginFor`. +/// Check the syntax is `OriginFor`, `&OriginFor` or `T::RuntimeOrigin`. pub fn check_dispatchable_first_arg_type(ty: &syn::Type, is_ref: bool) -> syn::Result<()> { - pub struct CheckDispatchableFirstArg(bool); - impl syn::parse::Parse for CheckDispatchableFirstArg { + pub struct CheckOriginFor(bool); + impl syn::parse::Parse for CheckOriginFor { fn parse(input: syn::parse::ParseStream) -> syn::Result { let is_ref = input.parse::().is_ok(); input.parse::()?; @@ -173,14 +174,27 @@ pub fn check_dispatchable_first_arg_type(ty: &syn::Type, is_ref: bool) -> syn::R } } - let result = syn::parse2::(ty.to_token_stream()); - return match result { - Ok(CheckDispatchableFirstArg(has_ref)) if is_ref == has_ref => Ok(()), - _ => { + pub struct CheckRuntimeOrigin; + impl syn::parse::Parse for CheckRuntimeOrigin { + fn parse(input: syn::parse::ParseStream) -> syn::Result { + input.parse::()?; + input.parse::()?; + input.parse::()?; + + Ok(Self) + } + } + + let result_origin_for = syn::parse2::(ty.to_token_stream()); + let result_runtime_origin = syn::parse2::(ty.to_token_stream()); + return match (result_origin_for, result_runtime_origin) { + (Ok(CheckOriginFor(has_ref)), _) if is_ref == has_ref => Ok(()), + (_, Ok(_)) => Ok(()), + (_, _) => { let msg = if is_ref { "Invalid type: expected `&OriginFor`" } else { - "Invalid type: expected `OriginFor`" + "Invalid type: expected `OriginFor` or `T::RuntimeOrigin`" }; return Err(syn::Error::new(ty.span(), msg)) }, @@ -282,8 +296,8 @@ impl CallDef { 0 if dev_mode => CallWeightDef::DevModeDefault, 0 => return Err(syn::Error::new( method.sig.span(), - "A pallet::call requires either a concrete `#[pallet::weight($expr)]` or an - inherited weight from the `#[pallet:call(weight($type))]` attribute, but + "A pallet::call requires either a concrete `#[pallet::weight($expr)]` or an + inherited weight from the `#[pallet:call(weight($type))]` attribute, but none were given.", )), 1 => match weight_attrs.pop().unwrap() { diff --git a/substrate/frame/support/src/dispatch.rs b/substrate/frame/support/src/dispatch.rs index e57227f9b401..c81dba127667 100644 --- a/substrate/frame/support/src/dispatch.rs +++ b/substrate/frame/support/src/dispatch.rs @@ -36,7 +36,8 @@ use sp_weights::Weight; /// returned from a dispatch. pub type DispatchResultWithPostInfo = sp_runtime::DispatchResultWithInfo; -/// Unaugmented version of `DispatchResultWithPostInfo` that can be returned from +#[docify::export] +/// Un-augmented version of `DispatchResultWithPostInfo` that can be returned from /// dispatchable functions and is automatically converted to the augmented type. Should be /// used whenever the `PostDispatchInfo` does not need to be overwritten. As this should /// be the common case it is the implicit return type when none is specified. diff --git a/substrate/frame/support/src/lib.rs b/substrate/frame/support/src/lib.rs index 2ec3b24db0ce..fd348f62b4f7 100644 --- a/substrate/frame/support/src/lib.rs +++ b/substrate/frame/support/src/lib.rs @@ -47,6 +47,8 @@ pub mod __private { pub use sp_core::{OpaqueMetadata, Void}; pub use sp_core_hashing_proc_macro; pub use sp_inherents; + #[cfg(feature = "std")] + pub use sp_io::TestExternalities; pub use sp_io::{self, hashing, storage::root as storage_root}; pub use sp_metadata_ir as metadata_ir; #[cfg(feature = "std")] @@ -2226,13 +2228,159 @@ pub use frame_support_procedural::pallet; /// Contains macro stubs for all of the pallet:: macros pub mod pallet_macros { pub use frame_support_procedural::{ - call_index, compact, composite_enum, config, disable_frame_system_supertrait_check, error, - event, extra_constants, feeless_if, generate_deposit, generate_store, getter, hooks, + composite_enum, config, disable_frame_system_supertrait_check, error, event, + extra_constants, feeless_if, generate_deposit, generate_store, getter, hooks, import_section, inherent, no_default, no_default_bounds, origin, pallet_section, storage_prefix, storage_version, type_value, unbounded, validate_unsigned, weight, whitelist_storage, }; + /// Allows a pallet to declare a set of functions as a *dispatchable extrinsic*. In + /// slightly simplified terms, this macro declares the set of "transactions" of a pallet. + /// + /// > The exact definition of **extrinsic** can be found in + /// > [`sp_runtime::generic::UncheckedExtrinsic`]. + /// + /// A **dispatchable** is a common term in FRAME, referring to process of constructing a + /// function, and dispatching it with the correct inputs. This is commonly used with + /// extrinsics, for example "an extrinsic has been dispatched". See + /// [`sp_runtime::traits::Dispatchable`] and [`crate::traits::UnfilteredDispatchable`]. + /// + /// ## Call Enum + /// + /// The macro is called `call` (rather than `#[pallet::extrinsics]`) because of the + /// generation of a `enum Call`. This enum contains only the encoding of the function + /// arguments of the dispatchable, alongside the information needed to route it to the + /// correct function. + /// + /// ``` + /// #[frame_support::pallet(dev_mode)] + /// pub mod custom_pallet { + /// # use frame_support::pallet_prelude::*; + /// # use frame_system::pallet_prelude::*; + /// # #[pallet::config] + /// # pub trait Config: frame_system::Config {} + /// # #[pallet::pallet] + /// # pub struct Pallet(_); + /// # use frame_support::traits::BuildGenesisConfig; + /// #[pallet::call] + /// impl Pallet { + /// pub fn some_dispatchable(_origin: OriginFor, _input: u32) -> DispatchResult { + /// Ok(()) + /// } + /// pub fn other(_origin: OriginFor, _input: u64) -> DispatchResult { + /// Ok(()) + /// } + /// } + /// + /// // generates something like: + /// // enum Call { + /// // some_dispatchable { input: u32 } + /// // other { input: u64 } + /// // } + /// } + /// + /// fn main() { + /// # use frame_support::{derive_impl, construct_runtime}; + /// # use frame_support::__private::codec::Encode; + /// # use frame_support::__private::TestExternalities; + /// # use frame_support::traits::UnfilteredDispatchable; + /// # impl custom_pallet::Config for Runtime {} + /// # #[derive_impl(frame_system::config_preludes::TestDefaultConfig as frame_system::DefaultConfig)] + /// # impl frame_system::Config for Runtime { + /// # type Block = frame_system::mocking::MockBlock; + /// # } + /// construct_runtime! { + /// pub struct Runtime { + /// System: frame_system, + /// Custom: custom_pallet + /// } + /// } + /// + /// # TestExternalities::new_empty().execute_with(|| { + /// let origin: RuntimeOrigin = frame_system::RawOrigin::Signed(10).into(); + /// // calling into a dispatchable from within the runtime is simply a function call. + /// let _ = custom_pallet::Pallet::::some_dispatchable(origin.clone(), 10); + /// + /// // calling into a dispatchable from the outer world involves constructing the bytes of + /// let call = custom_pallet::Call::::some_dispatchable { input: 10 }; + /// let _ = call.clone().dispatch_bypass_filter(origin); + /// + /// // the routing of a dispatchable is simply done through encoding of the `Call` enum, + /// // which is the index of the variant, followed by the arguments. + /// assert_eq!(call.encode(), vec![0u8, 10, 0, 0, 0]); + /// + /// // notice how in the encoding of the second function, the first byte is different and + /// // referring to the second variant of `enum Call`. + /// let call = custom_pallet::Call::::other { input: 10 }; + /// assert_eq!(call.encode(), vec![1u8, 10, 0, 0, 0, 0, 0, 0, 0]); + /// # }); + /// } + /// ``` + /// + /// Further properties of dispatchable functions are as follows: + /// + /// - Unless if annotated by `dev_mode`, it must contain [`weight`] to denote the + /// pre-dispatch weight consumed. + /// - The dispatchable must declare its index via [`call_index`], which can override the + /// position of a function in `enum Call`. + /// - The first argument is always an `OriginFor` (or `T::RuntimeOrigin`). + /// - The return type is always [`crate::dispatch::DispatchResult`] (or + /// [`crate::dispatch::DispatchResultWithPostInfo`]). + /// + /// **WARNING**: modifying dispatchables, changing their order (i.e. using [`call_index`]), + /// removing some, etc., must be done with care. This will change the encoding of the , and + /// the call can be stored on-chain (e.g. in `pallet-scheduler`). Thus, migration might be + /// needed. This is why the use of `call_index` is mandatory by default in FRAME. + /// + /// ## Default Behavior + /// + /// If no `#[pallet::call]` exists, then a default implementation corresponding to the + /// following code is automatically generated: + /// + /// ```ignore + /// #[pallet::call] + /// impl Pallet {} + /// ``` + pub use frame_support_procedural::call; + + /// Enforce the index of a variant in the generated `enum Call`. See [`call`] for more + /// information. + /// + /// All call indexes start from 0, until it encounters a dispatchable function with a + /// defined call index. The dispatchable function that lexically follows the function with + /// a defined call index will have that call index, but incremented by 1, e.g. if there are + /// 3 dispatchable functions `fn foo`, `fn bar` and `fn qux` in that order, and only `fn + /// bar` has a call index of 10, then `fn qux` will have an index of 11, instead of 1. + pub use frame_support_procedural::call_index; + + /// Declares the arguments of a [`call`] function to be encoded using + /// [`codec::Compact`]. This will results in smaller extrinsic encoding. + /// + /// A common example of `compact` is for numeric values that are often times far far away + /// from their theoretical maximum. For example, in the context of a crypto-currency, the + /// balance of an individual account is oftentimes way less than what the numeric type + /// allows. In all such cases, using `compact` is sensible. + /// + /// ``` + /// #[frame_support::pallet(dev_mode)] + /// pub mod custom_pallet { + /// # use frame_support::pallet_prelude::*; + /// # use frame_system::pallet_prelude::*; + /// # #[pallet::config] + /// # pub trait Config: frame_system::Config {} + /// # #[pallet::pallet] + /// # pub struct Pallet(_); + /// # use frame_support::traits::BuildGenesisConfig; + /// #[pallet::call] + /// impl Pallet { + /// pub fn some_dispatchable(_origin: OriginFor, #[pallet::compact] _input: u32) -> DispatchResult { + /// Ok(()) + /// } + /// } + /// } + pub use frame_support_procedural::compact; + /// Allows you to define the genesis configuration for the pallet. /// /// Item is defined as either an enum or a struct. It needs to be public and implement the diff --git a/substrate/frame/support/src/migrations.rs b/substrate/frame/support/src/migrations.rs index a9eb460421f1..bfd62c8611c6 100644 --- a/substrate/frame/support/src/migrations.rs +++ b/substrate/frame/support/src/migrations.rs @@ -224,8 +224,7 @@ impl PalletVersionToStorageVersionHelper for T { } } -/// Migrate from the `PalletVersion` struct to the new -/// [`StorageVersion`](crate::traits::StorageVersion) struct. +/// Migrate from the `PalletVersion` struct to the new [`StorageVersion`] struct. /// /// This will remove all `PalletVersion's` from the state and insert the current storage version. pub fn migrate_from_pallet_version_to_storage_version< diff --git a/substrate/frame/support/src/storage/child.rs b/substrate/frame/support/src/storage/child.rs index e54002d18db3..76e6f4ee4023 100644 --- a/substrate/frame/support/src/storage/child.rs +++ b/substrate/frame/support/src/storage/child.rs @@ -165,9 +165,9 @@ pub fn kill_storage(child_info: &ChildInfo, limit: Option) -> KillStorageRe /// guarantee that the subsequent call is in a new block; in this case the previous call's result /// cursor need not be passed in an a `None` may be passed instead. This exception may be useful /// then making this call solely from a block-hook such as `on_initialize`. -/// -/// Returns [`MultiRemovalResults`](sp_io::MultiRemovalResults) to inform about the result. Once the -/// resultant `maybe_cursor` field is `None`, then no further items remain to be deleted. + +/// Returns [`MultiRemovalResults`] to inform about the result. Once the resultant `maybe_cursor` +/// field is `None`, then no further items remain to be deleted. /// /// NOTE: After the initial call for any given child storage, it is important that no keys further /// keys are inserted. If so, then they may or may not be deleted by subsequent calls. diff --git a/substrate/frame/support/src/storage/mod.rs b/substrate/frame/support/src/storage/mod.rs index 7f39a3fdad85..c77de1f976f6 100644 --- a/substrate/frame/support/src/storage/mod.rs +++ b/substrate/frame/support/src/storage/mod.rs @@ -1583,7 +1583,7 @@ pub trait StorageTryAppend: StorageDecodeLength + private::Sealed { fn bound() -> usize; } -/// Storage value that is capable of [`StorageTryAppend`](crate::storage::StorageTryAppend). +/// Storage value that is capable of [`StorageTryAppend`]. pub trait TryAppendValue, I: Encode> { /// Try and append the `item` into the storage item. /// @@ -1612,7 +1612,7 @@ where } } -/// Storage map that is capable of [`StorageTryAppend`](crate::storage::StorageTryAppend). +/// Storage map that is capable of [`StorageTryAppend`]. pub trait TryAppendMap, I: Encode> { /// Try and append the `item` into the storage map at the given `key`. /// @@ -1646,7 +1646,7 @@ where } } -/// Storage double map that is capable of [`StorageTryAppend`](crate::storage::StorageTryAppend). +/// Storage double map that is capable of [`StorageTryAppend`]. pub trait TryAppendDoubleMap, I: Encode> { /// Try and append the `item` into the storage double map at the given `key`. /// diff --git a/substrate/frame/support/test/tests/construct_runtime_ui/deprecated_where_block.stderr b/substrate/frame/support/test/tests/construct_runtime_ui/deprecated_where_block.stderr index 08954bb6ab5c..a791c313b4a8 100644 --- a/substrate/frame/support/test/tests/construct_runtime_ui/deprecated_where_block.stderr +++ b/substrate/frame/support/test/tests/construct_runtime_ui/deprecated_where_block.stderr @@ -97,7 +97,7 @@ note: required because it appears within the type `RuntimeEvent` | ||_- in this macro invocation ... | note: required by a bound in `EncodeLike` - --> $CARGO/parity-scale-codec-3.6.4/src/encode_like.rs + --> $CARGO/parity-scale-codec-3.6.5/src/encode_like.rs | | pub trait EncodeLike: Sized + Encode {} | ^^^^^ required by this bound in `EncodeLike` @@ -129,7 +129,7 @@ note: required because it appears within the type `RuntimeEvent` | ||_- in this macro invocation ... | note: required by a bound in `Decode` - --> $CARGO/parity-scale-codec-3.6.4/src/codec.rs + --> $CARGO/parity-scale-codec-3.6.5/src/codec.rs | | pub trait Decode: Sized { | ^^^^^ required by this bound in `Decode` @@ -301,7 +301,7 @@ note: required because it appears within the type `RuntimeCall` | ||_- in this macro invocation ... | note: required by a bound in `EncodeLike` - --> $CARGO/parity-scale-codec-3.6.4/src/encode_like.rs + --> $CARGO/parity-scale-codec-3.6.5/src/encode_like.rs | | pub trait EncodeLike: Sized + Encode {} | ^^^^^ required by this bound in `EncodeLike` @@ -334,7 +334,7 @@ note: required because it appears within the type `RuntimeCall` | ||_- in this macro invocation ... | note: required by a bound in `Decode` - --> $CARGO/parity-scale-codec-3.6.4/src/codec.rs + --> $CARGO/parity-scale-codec-3.6.5/src/codec.rs | | pub trait Decode: Sized { | ^^^^^ required by this bound in `Decode` diff --git a/substrate/frame/support/test/tests/pallet_ui/call_invalid_origin_type.stderr b/substrate/frame/support/test/tests/pallet_ui/call_invalid_origin_type.stderr index c04729a24386..1f814eaa4077 100644 --- a/substrate/frame/support/test/tests/pallet_ui/call_invalid_origin_type.stderr +++ b/substrate/frame/support/test/tests/pallet_ui/call_invalid_origin_type.stderr @@ -1,4 +1,4 @@ -error: Invalid type: expected `OriginFor` +error: Invalid type: expected `OriginFor` or `T::RuntimeOrigin` --> tests/pallet_ui/call_invalid_origin_type.rs:34:22 | 34 | pub fn foo(origin: u8) {} diff --git a/substrate/frame/system/Cargo.toml b/substrate/frame/system/Cargo.toml index b61b4d531e2b..8f7e0052fe09 100644 --- a/substrate/frame/system/Cargo.toml +++ b/substrate/frame/system/Cargo.toml @@ -25,7 +25,7 @@ sp-runtime = { path = "../../primitives/runtime", default-features = false, feat sp-std = { path = "../../primitives/std", default-features = false} sp-version = { path = "../../primitives/version", default-features = false, features = ["serde"] } sp-weights = { path = "../../primitives/weights", default-features = false, features = ["serde"] } -docify = "0.2.0" +docify = "0.2.6" [dev-dependencies] criterion = "0.4.0" diff --git a/substrate/frame/system/src/lib.rs b/substrate/frame/system/src/lib.rs index 1b8dd6a9367f..15dd047fdb05 100644 --- a/substrate/frame/system/src/lib.rs +++ b/substrate/frame/system/src/lib.rs @@ -1018,6 +1018,7 @@ impl_ensure_origin_with_arg_ignoring_arg! { {} } +#[docify::export] /// Ensure that the origin `o` represents a signed extrinsic (i.e. transaction). /// Returns `Ok` with the account that signed the extrinsic or an `Err` otherwise. pub fn ensure_signed(o: OuterOrigin) -> Result @@ -1372,6 +1373,7 @@ impl Pallet { /// NOTE: Events not registered at the genesis block and quietly omitted. pub fn deposit_event_indexed(topics: &[T::Hash], event: T::RuntimeEvent) { let block_number = Self::block_number(); + // Don't populate events on genesis. if block_number.is_zero() { return @@ -1555,12 +1557,7 @@ impl Pallet { /// NOTE: Events not registered at the genesis block and quietly omitted. #[cfg(any(feature = "std", feature = "runtime-benchmarks", test))] pub fn events() -> Vec> { - debug_assert!( - !Self::block_number().is_zero(), - "events not registered at the genesis block" - ); - // Dereferencing the events here is fine since we are not in the - // memory-restricted runtime. + // Dereferencing the events here is fine since we are not in the memory-restricted runtime. Self::read_events_no_consensus().map(|e| *e).collect() } @@ -1581,6 +1578,21 @@ impl Pallet { Events::::stream_iter() } + /// Read and return the events of a specific pallet, as denoted by `E`. + /// + /// This is useful for a pallet that wishes to read only the events it has deposited into + /// `frame_system` using the standard `fn deposit_event`. + pub fn read_events_for_pallet() -> Vec + where + T::RuntimeEvent: TryInto, + { + Events::::get() + .into_iter() + .map(|er| er.event) + .filter_map(|e| e.try_into().ok()) + .collect::<_>() + } + /// Set the block number to something in particular. Can be used as an alternative to /// `initialize` for tests that don't need to bother with the other environment entries. #[cfg(any(feature = "std", feature = "runtime-benchmarks", test))] diff --git a/substrate/frame/tips/src/migrations/v4.rs b/substrate/frame/tips/src/migrations/v4.rs index 35569633d1bb..2404c6de1a16 100644 --- a/substrate/frame/tips/src/migrations/v4.rs +++ b/substrate/frame/tips/src/migrations/v4.rs @@ -90,7 +90,7 @@ pub fn migrate { /// Who this purports to be from and the number of extrinsics have come before diff --git a/substrate/primitives/runtime/src/generic/unchecked_extrinsic.rs b/substrate/primitives/runtime/src/generic/unchecked_extrinsic.rs index 1cdc0b8e4051..6ac381babeea 100644 --- a/substrate/primitives/runtime/src/generic/unchecked_extrinsic.rs +++ b/substrate/primitives/runtime/src/generic/unchecked_extrinsic.rs @@ -43,16 +43,37 @@ const EXTRINSIC_FORMAT_VERSION: u8 = 4; /// The `SingaturePayload` of `UncheckedExtrinsic`. type UncheckedSignaturePayload = (Address, Signature, Extra); -/// A extrinsic right from the external world. This is unchecked and so -/// can contain a signature. +/// An extrinsic right from the external world. This is unchecked and so can contain a signature. +/// +/// An extrinsic is formally described as any external data that is originating from the outside of +/// the runtime and fed into the runtime as a part of the block-body. +/// +/// Inherents are special types of extrinsics that are placed into the block by the block-builder. +/// They are unsigned because the assertion is that they are "inherently true" by virtue of getting +/// past all validators. +/// +/// Transactions are all other statements provided by external entities that the chain deems values +/// and decided to include in the block. This value is typically in the form of fee payment, but it +/// could in principle be any other interaction. Transactions are either signed or unsigned. A +/// sensible transaction pool should ensure that only transactions that are worthwhile are +/// considered for block-building. +#[doc = simple_mermaid::mermaid!("../../../../../docs/mermaid/extrinsics.mmd")] +/// This type is by no means enforced within Substrate, but given its genericness, it is highly +/// likely that for most use-cases it will suffice. Thus, the encoding of this type will dictate +/// exactly what bytes should be sent to a runtime to transact with it. +/// +/// This can be checked using [`Checkable`], yielding a [`CheckedExtrinsic`], which is the +/// counterpart of this type after its signature (and other non-negotiable validity checks) have +/// passed. #[derive(PartialEq, Eq, Clone)] pub struct UncheckedExtrinsic where Extra: SignedExtension, { - /// The signature, address, number of extrinsics have come before from - /// the same signer and an era describing the longevity of this transaction, - /// if this is a signed extrinsic. + /// The signature, address, number of extrinsics have come before from the same signer and an + /// era describing the longevity of this transaction, if this is a signed extrinsic. + /// + /// `None` if it is unsigned or an inherent. pub signature: Option>, /// The function that should be called. pub function: Call, @@ -286,6 +307,7 @@ where } } +#[docify::export(unchecked_extrinsic_encode_impl)] impl Encode for UncheckedExtrinsic where Address: Encode, diff --git a/substrate/primitives/runtime/src/offchain/storage_lock.rs b/substrate/primitives/runtime/src/offchain/storage_lock.rs index 1b795978447d..116e15788151 100644 --- a/substrate/primitives/runtime/src/offchain/storage_lock.rs +++ b/substrate/primitives/runtime/src/offchain/storage_lock.rs @@ -250,7 +250,7 @@ impl Lockable for BlockAndTime { /// /// A lock that is persisted in the DB and provides the ability to guard against /// concurrent access in an off-chain worker, with a defined expiry deadline -/// based on the concrete [`Lockable`](Lockable) implementation. +/// based on the concrete [`Lockable`] implementation. pub struct StorageLock<'a, L = Time> { // A storage value ref which defines the DB entry representing the lock. value_ref: StorageValueRef<'a>, diff --git a/substrate/src/lib.rs b/substrate/src/lib.rs deleted file mode 100644 index b5a583fcfcf1..000000000000 --- a/substrate/src/lib.rs +++ /dev/null @@ -1,242 +0,0 @@ -// This file is part of Substrate. - -// Copyright (C) Parity Technologies (UK) Ltd. -// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0 - -// This program is free software: you can redistribute it and/or modify -// it under the terms of the GNU General Public License as published by -// the Free Software Foundation, either version 3 of the License, or -// (at your option) any later version. - -// This program is distributed in the hope that it will be useful, -// but WITHOUT ANY WARRANTY; without even the implied warranty of -// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -// GNU General Public License for more details. - -// You should have received a copy of the GNU General Public License -// along with this program. If not, see . - -//! # Substrate -//! -//! Substrate is a Rust framework for building blockchains in a modular and extensible way. While in -//! itself un-opinionated, it is the main engine behind the Polkadot ecosystem. -//! -//! [![github]](https://github.com/paritytech/polkadot-sdk/tree/master/substrate/) - [![polkadot]](https://polkadot.network) -//! -//! This crate in itself does not contain any code and is just meant ot be a documentation hub for -//! substrate-based crates. -//! -//! ## Overview -//! -//! Substrate approaches blockchain development with an acknowledgement of a few self-evident -//! truths: -//! -//! 1. Society and technology evolves. -//! 2. Humans are fallible. -//! -//! This, specifically, makes the task of designing a correct, safe and long-lasting blockchain -//! system hard. -//! -//! Nonetheless, in order to achieve this goal, substrate embraces the following: -//! -//! 1. Use of **Rust** as a modern, and safe programming language, which limits human error through -//! various means, most notably memory safety. -//! 2. Substrate is written from the ground-up with a generic, modular and extensible design. This -//! ensures that software components can be easily swapped and upgraded. Examples of this is -//! multiple consensus mechanisms provided by Substrate, as listed below. -//! 3. Lastly, the final blockchain system created with the above properties needs to be -//! upgradeable. In order to achieve this, Substrate is designed as a meta-protocol, whereby the -//! application logic of the blockchain (called "Runtime") is encoded as a Wasm blob, and is -//! stored onchain. The rest of the system (called "Client") acts as the executor of the Wasm -//! blob. -//! -//! In essence, the meta-protocol of all Substrate based chains is the "Runtime as Wasm blob" -//! accord. This enables the Runtime to become inherently upgradeable (without forks). The upgrade -//! is merely a matter of the Wasm blob being changed in the chain state, which is, in principle, -//! same as updating an account's balance. -//! -//! ### Architecture -//! -//! Therefore, Substrate can be visualized as follows: -#![doc = simple_mermaid::mermaid!("../../docs/mermaid/substrate_simple.mmd")] -//! -//! The client and the runtime of course need to communicate. This is done through two concepts: -//! -//! 1. Host functions: a way for the (Wasm) runtime to talk to the client. All host functions are -//! defined in [`sp-io`]. For example, [`sp-io::storage`] are the set of host functions that -//! allow the runtime to read and write data to the on-chain state. -//! 2. Runtime APIs: a way for the client to talk to the Wasm runtime. Runtime APIs are defined -//! using macros and utilities in [`sp-api`]. For example, [`sp-api::Core`] is the most basic -//! runtime API that any blockchain must implement in order to be able to (re) execute blocks. -#![doc = simple_mermaid::mermaid!("../../docs/mermaid/substrate_client_runtime.mmd")] -//! -//! [`FRAME`], Substrate's default runtime development library takes the above even further by -//! embracing a declarative programming model whereby correctness is enhanced and the system is -//! highly configurable through parameterization. -//! -//! All in all, this design enables all substrate-based chains to achieve forkless, self-enacting -//! upgrades out of the box. Combined with governance abilities that are shipped with `FRAME`, this -//! enables a chain to survive the test of time. -//! -//! ## How to Get Stared -//! -//! Most developers want to leave the client side code as-is, and focus on the runtime. To do so, -//! look into the [`frame`] crate, which is the entry point crate into runtime development with -//! FRAME. -//! -//! > Side note, it is entirely possible to craft a substrate-based runtime without FRAME, an -//! > example of which can be found [here](https://github.com/JoshOrndorff/frameless-node-template). -//! -//! In more broad terms, the following avenues exist into developing with substrate: -//! -//! * **Templates**: A number of substrate-based templates exist and they can be used for various -//! purposes, with zero to little additional code needed. All of these templates contain runtimes -//! that are highly configurable and are likely suitable for basic needs. -//! * [`FRAME`]: If need, one can customize that runtime even further, by using `FRAME` and -//! developing custom modules. -//! * **Core**: To the contrary, some developers may want to customize the client side software to -//! achieve novel goals such as a new consensus engine, or a new database backend. While -//! Substrate's main configurability is in the runtime, the client is also highly generic and can -//! be customized to a great extent. -//! -//! ## Structure -//! -//! Substrate is a massive cargo workspace with hundreds of crates, therefore it is useful to know -//! how to navigate its crates. -//! -//! In broad terms, it is divided into three categories: -//! -//! * `sc-*` (short for *substrate-client*) crates, located under `./client` folder. These are all -//! the client crates. Notable examples are crates such as [`sc-network`], various consensus -//! crates, [`sc-rpc-api`] and [`sc-client-db`], all of which are expected to reside in the client -//! side. -//! * `sp-*` (short for *substrate-primitives*) crates, located under `./primitives` folder. These -//! are the traits that glue the client and runtime together, but are not opinionated about what -//! framework is using for building the runtime. Notable examples are [`sp-api`] and [`sp-io`], -//! which form the communication bridge between the client and runtime. -//! * `pallet-*` and `frame-*` crates, located under `./frame` folder. These are the crates related -//! to FRAME. See [`frame`] for more information. -//! -//! ### Wasm Build -//! -//! Many of the Substrate crates, such as entire `sp-*`, need to compile to both Wasm (when a Wasm -//! runtime is being generated) and native (for example, when testing). To achieve this, Substrate -//! follows the convention of the Rust community, and uses a `feature = "std"` to signify that a -//! crate is being built with the standard library, and is built for native. Otherwise, it is built -//! for `no_std`. -//! -//! This can be summarized in `#![cfg_attr(not(feature = "std"), no_std)]`, which you can often find -//! in any Substrate-based runtime. -//! -//! Substrate-based runtimes use [`substrate-wasm-builder`] in their `build.rs` to automatically -//! build their Wasm files as a part of normal build commandsOnce built, the wasm file is placed in -//! `./target/{debug|release}/wbuild/{runtime_name}.wasm`. -//! -//! ### Binaries -//! -//! Multiple binaries are shipped with substrate, the most important of which are located in the -//! `./bin` folder. -//! -//! * [`node`] is an extensive substrate node that contains the superset of all runtime and client -//! side features. The corresponding runtime, called [`kitchensink_runtime`] contains all of the -//! modules that are provided with `FRAME`. This node and runtime is only used for testing and -//! demonstration. -//! * [`chain-spec-builder`]: Utility to build more detailed [chain-spec][`sc-chain-spec`] for the -//! aforementioned node. Other projects typically contain a `build-spec` subcommand that does the -//! same. -//! * [`node-template`]: a template node that contains a minimal set of features and can act as a -//! starting point of a project. -//! * [`subkey`]: Substrate's key management utility. -//! -//! ### Anatomy of a Binary Crate -//! -//! From the above, [`node`] and [`node-template`] are essentially blueprints of a substrate-based -//! project, as the name of the latter is implying. Each substrate-based project typically contains -//! the following: -//! -//! * Under `./runtime`, a `./runtime/src/lib.rs` which is the top level runtime amalgamator file. -//! This file typically contains the [`frame_support::construct_runtime`] macro, which is the -//! final definition of a runtime. -//! -//! * Under `./node`, a `main.rs`, which is the point, and a `./service.rs`, which contains all the -//! client side components. Skimming this file yields an overview of the networking, database, -//! consensus and similar client side components. -//! -//! > The above two are conventions, not rules. -//! -//! ## Parachain? -//! -//! As noted above, Substrate is the main engine behind the Polkadot ecosystem. One of the ways -//! through which Polkadot can be utilized is by building "parachains", blockchains that are -//! connected to Polkadot's shared security. -//! -//! To build a parachain, one could use -//! [`Cumulus`](https://github.com/paritytech/polkadot-sdk/tree/master/cumulus), the library on top -//! of Substrate, empowering any substrate-based chain to be a Polkadot parachain. -//! -//! ## Where To Go Next? -//! -//! Additional noteworthy crates within substrate: -//! -//! - Chain specification of a Substrate node: -//! - [`sc-chain-spec`] -//! - RPC APIs of a Substrate node: [`sc-rpc-api`]/[`sc-rpc`] -//! - CLI Options of a Substrate node: [`sc-cli`] -//! - All of the consensus related crates provided by Substrate: -//! - [`sc-consensus-aura`] -//! - [`sc-consensus-babe`] -//! - [`sc-consensus-grandpa`] -//! - [`sc-consensus-beefy`] -//! - [`sc-consensus-manual-seal`] -//! - [`sc-consensus-pow`] -//! -//! Additional noteworthy external resources: -//! -//! - [Substrate Developer Hub](https://substrate.dev) -//! - [Parity Tech's Documentation Hub](https://paritytech.github.io/) -//! - [Frontier: Substrate's Ethereum Compatibility Library](https://paritytech.github.io/frontier/) -//! - [Polkadot Wiki](https://wiki.polkadot.network/en/) -//! -//! Notable upstream crates: -//! -//! - [`parity-scale-codec`](https://github.com/paritytech/parity-scale-codec) -//! - [`parity-db`](https://github.com/paritytech/parity-db) -//! - [`trie`](https://github.com/paritytech/trie) -//! - [`parity-common`](https://github.com/paritytech/parity-common) -//! -//! Templates: -//! -//! - classic [`substrate-node-template`](https://github.com/substrate-developer-hub/substrate-node-template) -//! - classic [cumulus-parachain-template](https://github.com/substrate-developer-hub/substrate-parachain-template) -//! - [`extended-parachain-template`](https://github.com/paritytech/extended-parachain-template) -//! - [`frontier-parachain-template`](https://github.com/paritytech/frontier-parachain-template) -//! -//! [polkadot]: -//! https://img.shields.io/badge/polkadot-E6007A?style=for-the-badge&logo=polkadot&logoColor=white -//! [github]: -//! https://img.shields.io/badge/github-8da0cb?style=for-the-badge&labelColor=555555&logo=github -//! [`FRAME`]: ../frame/index.html -//! [`sp-io`]: ../sp_io/index.html -//! [`sp-api`]: ../sp_api/index.html -//! [`sp-api`]: ../sp_api/index.html -//! [`sc-client-db`]: ../sc_client_db/index.html -//! [`sc-chain-spec`]: ../sc_chain_spec/index.html -//! [`sc-network`]: ../sc_network/index.html -//! [`sc-rpc-api`]: ../sc_rpc_api/index.html -//! [`sc-rpc`]: ../sc_rpc/index.html -//! [`sc-cli`]: ../sc_cli/index.html -//! [`sc-consensus-aura`]: ../sc_consensus_aura/index.html -//! [`sc-consensus-babe`]: ../sc_consensus_babe/index.html -//! [`sc-consensus-grandpa`]: ../sc_consensus_grandpa/index.html -//! [`sc-consensus-beefy`]: ../sc_consensus_beefy/index.html -//! [`sc-consensus-manual-seal`]: ../sc_consensus_manual_seal/index.html -//! [`sc-consensus-pow`]: ../sc_consensus_pow/index.html -//! [`node`]: ../node_cli/index.html -//! [`node-template`]: ../node_template/index.html -//! [`kitchensink_runtime`]: ../kitchensink_runtime/index.html -//! [`subkey`]: ../subkey/index.html -//! [`chain-spec-builder`]: ../chain_spec_builder/index.html -//! [`substrate-wasm-builder`]: https://crates.io/crates/substrate-wasm-builder - -#![deny(rustdoc::broken_intra_doc_links)] -#![deny(rustdoc::private_intra_doc_links)]