From f58630fd116bb1a22fac34ae8c4f38b89ba95b53 Mon Sep 17 00:00:00 2001 From: Callum Date: Mon, 16 Nov 2020 19:34:13 +0100 Subject: [PATCH 1/8] initial draft of rfc 5 - soft upgrades --- rfc/005-soft-chain-upgrades.md | 134 +++++++++++++++++++++++++++++++++ 1 file changed, 134 insertions(+) create mode 100644 rfc/005-soft-chain-upgrades.md diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md new file mode 100644 index 00000000..741c5bd9 --- /dev/null +++ b/rfc/005-soft-chain-upgrades.md @@ -0,0 +1,134 @@ +# RFC 005: Soft Chain Upgrades + +## Changelog + +- 2020-11-16: Initial draft. + +## Author(s) + +- Callum Waters (@cmwaters) + +## Context + +The current method for upgrading a network across block protocol breaking changes is to hard fork the chain. All nodes must coordinate a halt height/time, where application state can be exported across and an entirely new chain (as far as Tendermint is concerned) is created (see [cosmoshub-3 upgrade](https://blog.cosmos.network/cosmos-hub-3-upgrade-announcement-39c9da941aee)). This means that each block protocol version maps to the entire life of a single chain. The result of this is that either Tendermint is hindered with the flexibility and quantity of changes it can make or that networks have to deal with pain of frequently upgrading. + +Furthermore, although work from [RFC002: Non-Zero Genesis](https://github.com/tendermint/spec/blob/master/rfc/002-nonzero-genesis.md) has enabled unique heights throughout the history of a "logical" chain, data retrievability across heights remains difficult as blockchains that choose to upgrade not only have to coordinate upgrades but they essentially lose all their history. + +The scope of this RFC is to address the need for Tendermint to have flexibility in improving itself whilst offering greater ease of use to the networks running on Tendermint. This is done by introducing a division in the types of ways a network can be upgraded. There will still remain the need for hard upgrades (the starting of a new chain) but this will not be focused on in this RFC(discussions on improving this can be found [here](https://github.com/tendermint/tendermint/issues/5595)). Instead, this RFC will only explore what supporting soft upgrades would entail. + +## Proposal + +Tendermint supports a class of upgrades deemed as soft upgrades. + +Soft and hard upgrades would directly align with semver. For example, a release from 1.2 → 1.3 would be considered a soft upgrade and could be executed on a live chain (more on how that would be executed further down). A major release from 1.5 → 2.0 would be a hard upgrade and require a network to halt and migrate. + +### Protocol Versioning + +Tendermint currently has three protocol versions: Block, P2P and App. + +The block version consists of all block related data structures i.e. Header, Vote, Validator, Commit. This would be the sole focus for soft upgrades (a full list of the exact data structures can be found in [data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md#data-structures). A block version could be incremented between minor releases. However, for backwards compatibility, all prior versions in a given major release would also need to be supported thus TM 1.3 could support block version 9 and 10 and TM 1.4 could introduce block version 11 but would still need to support 9 and 10. In a major release the support of prior versions could be dropped. Continuing with the same example this could mean that TM 2.0 could support block version 11 without supporting versions 9 or 10. + +The other two, P2P and App would be left as is. Any changes to the network protocol would require a hard upgrade and therefore would only be allowed in major releases. Any changes to the app version would be able to be conducted live and at the discretion of the app. + +Depending on the implementation it may be more beneficial to divide block protocol versions into more specific abstractions i.e. Block, State and Consensus to isolate changes from spilling over multiple modules (normally a change to say `commit` would cause us to replicate all the data structures). However, increasing the granularity of abstractions would compound the permutations between versions which could lead to untamable code, hence, given that these changes should be grouped together and occur seldomly it should suffice to have a single block protocol version. + +### Transitions + +Networks would be expected to announce upgrades in advance allowing a window for individual nodes to upgrade to a software version of Tendermint that would comply with the announced block protocol upgrade. Node runners would temporarily halt their node, change to the latest binary and start up again. Tendermint could also offer a tool to easily swap between binaries thus reducing the amount of down-time that nodes experience although this is outside the scope of this RFC. + +The actual switch would occur in the same manner as an App Version change, using the `EndBlockResponse{}` ABCI call to indicate to Tendermint that a different block protocol be used. In the case that a node failed to upgrade, the node would gracefully stop and error. + +Given the delayed execution model, careful consideration would be needed to ensure that the nodes could transition smoothly. It may be that the actual upgrade might not take place till height: h + 2. + +### Current Use of Versioning + +Records of versions are currently stored in `State` and in the `Header` of each block. This already covers a broad base of scenarios. The consensus reactor can use the version from state to ensure it reads `Vote`s and `Partsetheader`s correctly. Similarly, fast sync, the block executor and evidence module, whom all interact with block protocol data structures also all have access to state. The next few sections will explore the aspects of Tendermint that will need to be modified to prepare for versioning. + +#### RPC + +Supporting soft upgrades means that the RPC will need to be able to deliver information across different versions thus the RPC should be versatile to requests over all prior heights and must return the data structure alongside a version number so that the receiver knows how to interpret the information. + +#### Light Clients + +Similar to RPC, light clients will also need to be wary of block protocol changes and be able to handle them accordingly. Light clients should have the same version as the latest block protocol that they support. They will also require a similar method for verifying across block versions. + +#### Evidence + +Evidence is always a reference to a past action. It could be possible that the past action whether it be sending duplicate votes or attempting to fool a light client with an alternative block was of a different block version. To be able to process this correctly, evidence might need to include the version of it's contents or have a method of working out what version the contents should be in order to process them accordingly (upgrade heights could be stored in state). + +#### Spec + +In the future, we expect a versioned spec, where each spec release will list the software versions and protocol versions that correlate to that release. This will reside in `spec/version.md`. A clear guide of the rules behind breaking changes should also be there for easy reference. + +### Implementation Approach + +If this proposal is accepted, the implementation will need to be further fleshed out in an ADR, however, it is helpful to illustrate here the approach that Tendermint would most likely take to support soft upgrades. + +This would mostly likely mimic a similar design to what has been done in the Cosmos SDK with keeping legacy code and using conditionals to handle transitions between the versions. One would expect a directory structure for each module as such: + +```go +consensus // module +|---- state.go +|---- state_test.go +| ... +|---- votes.go // current version (v10) +| ... +|---- legacy + |---- v8 // prior supported versions + |---- votes.go + | ... // other data structures ( and pure functions ) + |---- v9 + |---- votes.go + | ... +``` + +The module, depending on the types, may statically cast each of the different versions or define an interface that it expects all versions to follow. For example: + +```go +func (blockExec *BlockExecutor) CreateProposalBlock(state State) (BlockI, error) { + switch state.Version.Block { + case 9: + return v9.CreateProposalBlock(state, blockExec.Mempool, blockExec.Evpool) + case 10: + return v10.CreateProposalBlock(state, blockExec.Mempool, blockExec.Evpool) + default: + return fmt.Errorf("block version %d not recognised", state.Version.Block) + } +} +``` + +In this case, Block which falls under a data structure of the block protocol could have many versions and is represented here as an interface. `State` and `BlockExecutor` are structs outside of the block protocol. Breaking changes to them could only occur in major releases. + +### Alternative Approach: Migration Scripts + +Although using migration scripts would have been most likely a simpler solution, unfortunately in most cases this would result in breaking the hashes and is therefore not a viable approach. + + +### Concluding Remarks + +This option provides greater flexibility for the developers and potentially reduces the amount of coordination and work required for validators and the frequency with which hard forks need to occur. However, it is nevertheless recommended that block protocol breaking changes still be grouped together and remain as seldom as possible. + +The implementation to support this would ideally need to be done prior to the 1.0 release. + +## Status + +Proposed + +## Consequences + +### Positive + +- Chains get the benefits of upgrades without having to sacrifice their block history + +### Negative + +- There is a large degree of complexity required in ensuring the flexibility of data structures. This could lead to the bloating of the code-base and/or decreasing how easy it is to follow. + +### Neutral + +- There will still need to be hard chain upgrades (albeit fewer). + +## References + +- [Upgrade tooling tracking issue](https://github.com/tendermint/tendermint/issues/5595) +- [Protocol versions](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-016-protocol-versions.md) \ No newline at end of file From 845da986fcb147edbe984f9783b8c5207141a189 Mon Sep 17 00:00:00 2001 From: Callum Date: Tue, 17 Nov 2020 11:09:08 +0100 Subject: [PATCH 2/8] linting --- rfc/005-soft-chain-upgrades.md | 183 +++++++++++++++++++++++++-------- 1 file changed, 138 insertions(+), 45 deletions(-) diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md index 741c5bd9..cd54774c 100644 --- a/rfc/005-soft-chain-upgrades.md +++ b/rfc/005-soft-chain-upgrades.md @@ -1,3 +1,4 @@ + # RFC 005: Soft Chain Upgrades ## Changelog @@ -10,61 +11,138 @@ ## Context -The current method for upgrading a network across block protocol breaking changes is to hard fork the chain. All nodes must coordinate a halt height/time, where application state can be exported across and an entirely new chain (as far as Tendermint is concerned) is created (see [cosmoshub-3 upgrade](https://blog.cosmos.network/cosmos-hub-3-upgrade-announcement-39c9da941aee)). This means that each block protocol version maps to the entire life of a single chain. The result of this is that either Tendermint is hindered with the flexibility and quantity of changes it can make or that networks have to deal with pain of frequently upgrading. - -Furthermore, although work from [RFC002: Non-Zero Genesis](https://github.com/tendermint/spec/blob/master/rfc/002-nonzero-genesis.md) has enabled unique heights throughout the history of a "logical" chain, data retrievability across heights remains difficult as blockchains that choose to upgrade not only have to coordinate upgrades but they essentially lose all their history. - -The scope of this RFC is to address the need for Tendermint to have flexibility in improving itself whilst offering greater ease of use to the networks running on Tendermint. This is done by introducing a division in the types of ways a network can be upgraded. There will still remain the need for hard upgrades (the starting of a new chain) but this will not be focused on in this RFC(discussions on improving this can be found [here](https://github.com/tendermint/tendermint/issues/5595)). Instead, this RFC will only explore what supporting soft upgrades would entail. +The current method for upgrading a network across block protocol breaking +changes is to hard fork the chain. All nodes must coordinate a halt height/ +time, where application state can be exported across and an entirely new chain +(as far as Tendermint is concerned) is created (see [cosmoshub-3 +upgrade](https://blog.cosmos.network/cosmos-hub-3-upgrade-announcement-39c9da941aee)). +This means that each block protocol version maps to the entire life of a single +chain. The result of this is that either Tendermint is hindered with the +flexibility and quantity of changes it can make or that networks have to deal +with pain of frequently upgrading. + +Furthermore, although work from [RFC002: Non-Zero Genesis](https://github.com/tendermint/spec/blob/master/rfc/002-nonzero-genesis.md) +has enabled unique heights throughout the history of a "logical" chain, data +retrievability across heights remains difficult as blockchains that choose to +upgrade not only have to coordinate upgrades but they essentially lose all their +history. + +The scope of this RFC is to address the need for Tendermint to have flexibility +in improving itself whilst offering greater ease of use to the networks running +on Tendermint. This is done by introducing a division in the types of ways a +network can be upgraded. There will still remain the need for hard upgrades (the +starting of a new chain) but this will not be focused on in this +RFC(discussions on improving this can be found [here](https://github.com/tendermint/tendermint/issues/5595)). +Instead, this RFC will only explore what supporting soft upgrades would entail. ## Proposal Tendermint supports a class of upgrades deemed as soft upgrades. -Soft and hard upgrades would directly align with semver. For example, a release from 1.2 → 1.3 would be considered a soft upgrade and could be executed on a live chain (more on how that would be executed further down). A major release from 1.5 → 2.0 would be a hard upgrade and require a network to halt and migrate. +Soft and hard upgrades would directly align with semver. For example, a release +from 1.2 → 1.3 would be considered a soft upgrade and could be executed on a +live chain (more on how that would be executed further down). A major release +from 1.5 → 2.0 would be a hard upgrade and require a network to halt and migrate. ### Protocol Versioning Tendermint currently has three protocol versions: Block, P2P and App. -The block version consists of all block related data structures i.e. Header, Vote, Validator, Commit. This would be the sole focus for soft upgrades (a full list of the exact data structures can be found in [data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md#data-structures). A block version could be incremented between minor releases. However, for backwards compatibility, all prior versions in a given major release would also need to be supported thus TM 1.3 could support block version 9 and 10 and TM 1.4 could introduce block version 11 but would still need to support 9 and 10. In a major release the support of prior versions could be dropped. Continuing with the same example this could mean that TM 2.0 could support block version 11 without supporting versions 9 or 10. - -The other two, P2P and App would be left as is. Any changes to the network protocol would require a hard upgrade and therefore would only be allowed in major releases. Any changes to the app version would be able to be conducted live and at the discretion of the app. - -Depending on the implementation it may be more beneficial to divide block protocol versions into more specific abstractions i.e. Block, State and Consensus to isolate changes from spilling over multiple modules (normally a change to say `commit` would cause us to replicate all the data structures). However, increasing the granularity of abstractions would compound the permutations between versions which could lead to untamable code, hence, given that these changes should be grouped together and occur seldomly it should suffice to have a single block protocol version. +The block version consists of all block related data structures i.e. Header, +Vote, Validator, Commit. This would be the sole focus for soft upgrades (a full +list of the exact data structures can be found in +[data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md#data-structures). +A block version could be incremented between minor releases. However, for +backwards compatibility, all prior versions in a given major release would also +need to be supported thus TM 1.3 could support block version 9 and 10 and TM 1.4 +could introduce block version 11 but would still need to support 9 and 10. In a +major release the support of prior versions could be dropped. Continuing with +the same example this could mean that TM 2.0 could support block version 11 +without supporting versions 9 or 10. + +The other two, P2P and App would be left as is. Any changes to the network +protocol would require a hard upgrade and therefore would only be allowed in +major releases. Any changes to the app version would be able to be conducted +live and at the discretion of the app. + +Depending on the implementation it may be more beneficial to divide block +protocol versions into more specific abstractions i.e. Block, State and +Consensus to isolate changes from spilling over multiple modules (normally a +change to say `commit` would cause us to replicate all the data structures). +However, increasing the granularity of abstractions would compound the +permutations between versions which could lead to untamable code, hence, given +that these changes should be grouped together and occur seldomly it should +suffice to have a single block protocol version. ### Transitions -Networks would be expected to announce upgrades in advance allowing a window for individual nodes to upgrade to a software version of Tendermint that would comply with the announced block protocol upgrade. Node runners would temporarily halt their node, change to the latest binary and start up again. Tendermint could also offer a tool to easily swap between binaries thus reducing the amount of down-time that nodes experience although this is outside the scope of this RFC. +Networks would be expected to announce upgrades in advance allowing a window for +individual nodes to upgrade to a software version of Tendermint that would +comply with the announced block protocol upgrade. Node runners would temporarily +halt their node, change to the latest binary and start up again. Tendermint +could also offer a tool to easily swap between binaries thus reducing the amount +of down-time that nodes experience although this is outside the scope of this +RFC. -The actual switch would occur in the same manner as an App Version change, using the `EndBlockResponse{}` ABCI call to indicate to Tendermint that a different block protocol be used. In the case that a node failed to upgrade, the node would gracefully stop and error. +The actual switch would occur in the same manner as an App Version change, using +the `EndBlockResponse{}` ABCI call to indicate to Tendermint that a different +block protocol be used. In the case that a node failed to upgrade, the node +would gracefully stop and error. -Given the delayed execution model, careful consideration would be needed to ensure that the nodes could transition smoothly. It may be that the actual upgrade might not take place till height: h + 2. +Given the delayed execution model, careful consideration would be needed to +ensure that the nodes could transition smoothly. It may be that the actual +upgrade might not take place till height: h + 2. ### Current Use of Versioning -Records of versions are currently stored in `State` and in the `Header` of each block. This already covers a broad base of scenarios. The consensus reactor can use the version from state to ensure it reads `Vote`s and `Partsetheader`s correctly. Similarly, fast sync, the block executor and evidence module, whom all interact with block protocol data structures also all have access to state. The next few sections will explore the aspects of Tendermint that will need to be modified to prepare for versioning. +Records of versions are currently stored in `State` and in the `Header` of each +block. This already covers a broad base of scenarios. The consensus reactor can +use the version from state to ensure it reads `Vote`s and `Partsetheader`s +correctly. Similarly, fast sync, the block executor and evidence module, whom +all interact with block protocol data structures also all have access to state. +The next few sections will explore the aspects of Tendermint that will need to +be modified to prepare for versioning. #### RPC -Supporting soft upgrades means that the RPC will need to be able to deliver information across different versions thus the RPC should be versatile to requests over all prior heights and must return the data structure alongside a version number so that the receiver knows how to interpret the information. +Supporting soft upgrades means that the RPC will need to be able to deliver +information across different versions thus the RPC should be versatile to +requests over all prior heights and must return the data structure alongside a +version number so that the receiver knows how to interpret the information. #### Light Clients -Similar to RPC, light clients will also need to be wary of block protocol changes and be able to handle them accordingly. Light clients should have the same version as the latest block protocol that they support. They will also require a similar method for verifying across block versions. +Similar to RPC, light clients will also need to be wary of block protocol +changes and be able to handle them accordingly. Light clients should have the +same version as the latest block protocol that they support. They will also +require a similar method for verifying across block versions. #### Evidence -Evidence is always a reference to a past action. It could be possible that the past action whether it be sending duplicate votes or attempting to fool a light client with an alternative block was of a different block version. To be able to process this correctly, evidence might need to include the version of it's contents or have a method of working out what version the contents should be in order to process them accordingly (upgrade heights could be stored in state). +Evidence is always a reference to a past action. It could be possible that the +past action whether it be sending duplicate votes or attempting to fool a light +client with an alternative block was of a different block version. To be able to +process this correctly, evidence might need to include the version of it's +contents or have a method of working out what version the contents should be in +order to process them accordingly (upgrade heights could be stored in state). #### Spec -In the future, we expect a versioned spec, where each spec release will list the software versions and protocol versions that correlate to that release. This will reside in `spec/version.md`. A clear guide of the rules behind breaking changes should also be there for easy reference. +In the future, we expect a versioned spec, where each spec release will list the +software versions and protocol versions that correlate to that release. This +will reside in `spec/version.md`. A clear guide of the rules behind breaking +changes should also be there for easy reference. ### Implementation Approach -If this proposal is accepted, the implementation will need to be further fleshed out in an ADR, however, it is helpful to illustrate here the approach that Tendermint would most likely take to support soft upgrades. +If this proposal is accepted, the implementation will need to be further fleshed +out in an ADR, however, it is helpful to illustrate here the approach that +Tendermint would most likely take to support soft upgrades. -This would mostly likely mimic a similar design to what has been done in the Cosmos SDK with keeping legacy code and using conditionals to handle transitions between the versions. One would expect a directory structure for each module as such: +This would mostly likely mimic a similar design to what has been done in the +Cosmos SDK with keeping legacy code and using conditionals to handle transitions +between the versions. One would expect a directory structure for each module as +such: ```go consensus // module @@ -74,41 +152,53 @@ consensus // module |---- votes.go // current version (v10) | ... |---- legacy - |---- v8 // prior supported versions - |---- votes.go - | ... // other data structures ( and pure functions ) - |---- v9 - |---- votes.go - | ... + |---- v8 // prior supported versions + |---- votes.go + | ... // other data structures ( and pure functions ) + |---- v9 + |---- votes.go + | ... ``` -The module, depending on the types, may statically cast each of the different versions or define an interface that it expects all versions to follow. For example: +The module, depending on the types, may statically cast each of the different +versions or define an interface that it expects all versions to follow. For +example: ```go -func (blockExec *BlockExecutor) CreateProposalBlock(state State) (BlockI, error) { - switch state.Version.Block { - case 9: - return v9.CreateProposalBlock(state, blockExec.Mempool, blockExec.Evpool) - case 10: - return v10.CreateProposalBlock(state, blockExec.Mempool, blockExec.Evpool) - default: - return fmt.Errorf("block version %d not recognised", state.Version.Block) - } +func (blockExec *BlockExecutor) CreateProposalBlock(state State) (BlockI, error) +{ + switch state.Version.Block { + case 9: + return v9.CreateProposalBlock(state, blockExec.Mempool, blockExec.Evpool) + case 10: + return v10.CreateProposalBlock(state, blockExec.Mempool, blockExec.Evpool) + default: + return fmt.Errorf("block version %d not recognised", state.Version.Block) + } } ``` -In this case, Block which falls under a data structure of the block protocol could have many versions and is represented here as an interface. `State` and `BlockExecutor` are structs outside of the block protocol. Breaking changes to them could only occur in major releases. +In this case, Block which falls under a data structure of the block protocol +could have many versions and is represented here as an interface. `State` and +`BlockExecutor` are structs outside of the block protocol. Breaking changes to +them could only occur in major releases. ### Alternative Approach: Migration Scripts -Although using migration scripts would have been most likely a simpler solution, unfortunately in most cases this would result in breaking the hashes and is therefore not a viable approach. - +Although using migration scripts would have been most likely a simpler solution, +unfortunately in most cases this would result in breaking the hashes and is +therefore not a viable approach. ### Concluding Remarks -This option provides greater flexibility for the developers and potentially reduces the amount of coordination and work required for validators and the frequency with which hard forks need to occur. However, it is nevertheless recommended that block protocol breaking changes still be grouped together and remain as seldom as possible. +This option provides greater flexibility for the developers and potentially +reduces the amount of coordination and work required for validators and the +frequency with which hard forks need to occur. However, it is nevertheless +recommended that block protocol breaking changes still be grouped together and +remain as seldom as possible. -The implementation to support this would ideally need to be done prior to the 1.0 release. +The implementation to support this would ideally need to be done prior to the +1.0 release. ## Status @@ -118,11 +208,14 @@ Proposed ### Positive -- Chains get the benefits of upgrades without having to sacrifice their block history +- Chains get the benefits of upgrades without having to sacrifice their block +history ### Negative -- There is a large degree of complexity required in ensuring the flexibility of data structures. This could lead to the bloating of the code-base and/or decreasing how easy it is to follow. +- There is a large degree of complexity required in ensuring the flexibility of +data structures. This could lead to the bloating of the code-base and/or +decreasing how easy it is to follow. ### Neutral @@ -131,4 +224,4 @@ Proposed ## References - [Upgrade tooling tracking issue](https://github.com/tendermint/tendermint/issues/5595) -- [Protocol versions](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-016-protocol-versions.md) \ No newline at end of file +- [Protocol versions](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-016-protocol-versions.md) From 7be072a72a673e3a336b490128f39f8a10f489f5 Mon Sep 17 00:00:00 2001 From: Callum Date: Tue, 24 Nov 2020 15:35:14 +0100 Subject: [PATCH 3/8] added chain migration proposal for comparison --- rfc/005-soft-chain-upgrades.md | 380 ++++++++++++++++++++++---------- rfc/images/chain_migation_1.png | Bin 0 -> 33334 bytes 2 files changed, 267 insertions(+), 113 deletions(-) create mode 100644 rfc/images/chain_migation_1.png diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md index cd54774c..139dbb72 100644 --- a/rfc/005-soft-chain-upgrades.md +++ b/rfc/005-soft-chain-upgrades.md @@ -3,6 +3,7 @@ ## Changelog +- 2020-11-23: Added chain migration proposal for comparison - 2020-11-16: Initial draft. ## Author(s) @@ -18,8 +19,8 @@ time, where application state can be exported across and an entirely new chain upgrade](https://blog.cosmos.network/cosmos-hub-3-upgrade-announcement-39c9da941aee)). This means that each block protocol version maps to the entire life of a single chain. The result of this is that either Tendermint is hindered with the -flexibility and quantity of changes it can make or that networks have to deal -with pain of frequently upgrading. +flexibility of changes it can make or that networks have to deal +with the pain of frequently upgrading and all its shortfalls. Furthermore, although work from [RFC002: Non-Zero Genesis](https://github.com/tendermint/spec/blob/master/rfc/002-nonzero-genesis.md) has enabled unique heights throughout the history of a "logical" chain, data @@ -29,60 +30,46 @@ history. The scope of this RFC is to address the need for Tendermint to have flexibility in improving itself whilst offering greater ease of use to the networks running -on Tendermint. This is done by introducing a division in the types of ways a -network can be upgraded. There will still remain the need for hard upgrades (the -starting of a new chain) but this will not be focused on in this -RFC(discussions on improving this can be found [here](https://github.com/tendermint/tendermint/issues/5595)). -Instead, this RFC will only explore what supporting soft upgrades would entail. +on Tendermint. This is done by introducing a division in the way a +network can upgrade: either soft (live) or hard (stop, do some changes then +coordinate a restart). There may still remain the need for hard upgrades but +this will be focused [elsewhere](https://github.com/tendermint/tendermint/issues/5595)). +Instead, this RFC will describe what supporting soft upgrades would entail +and explore two different methods: **Multi Data Structure Support** or +**Chain Migrations**. ## Proposal Tendermint supports a class of upgrades deemed as soft upgrades. -Soft and hard upgrades would directly align with semver. For example, a release -from 1.2 → 1.3 would be considered a soft upgrade and could be executed on a -live chain (more on how that would be executed further down). A major release -from 1.5 → 2.0 would be a hard upgrade and require a network to halt and migrate. +The implementation to support this would ideally need to be done prior to the +1.0 release. ### Protocol Versioning Tendermint currently has three protocol versions: Block, P2P and App. -The block version consists of all block related data structures i.e. Header, -Vote, Validator, Commit. This would be the sole focus for soft upgrades (a full -list of the exact data structures can be found in -[data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md#data-structures). -A block version could be incremented between minor releases. However, for -backwards compatibility, all prior versions in a given major release would also -need to be supported thus TM 1.3 could support block version 9 and 10 and TM 1.4 -could introduce block version 11 but would still need to support 9 and 10. In a -major release the support of prior versions could be dropped. Continuing with -the same example this could mean that TM 2.0 could support block version 11 -without supporting versions 9 or 10. - -The other two, P2P and App would be left as is. Any changes to the network -protocol would require a hard upgrade and therefore would only be allowed in -major releases. Any changes to the app version would be able to be conducted -live and at the discretion of the app. - -Depending on the implementation it may be more beneficial to divide block -protocol versions into more specific abstractions i.e. Block, State and -Consensus to isolate changes from spilling over multiple modules (normally a -change to say `commit` would cause us to replicate all the data structures). -However, increasing the granularity of abstractions would compound the -permutations between versions which could lead to untamable code, hence, given -that these changes should be grouped together and occur seldomly it should -suffice to have a single block protocol version. +Block protocol version would initially be the sole focus for soft upgrades. +This consists of all block related data structures i.e. `Header`, +`Vote`, `Validator`, `Commit` (a full list of the exact data structures can be +found in [data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md#data-structures). +Data structures are dictated by the spec thus requiring a Tendermint software +version to indicate the block version/s it supports. + +The app version can already be changed via consensus and is left to the +discretion of the application to handle. The P2P version would also be left as +it is, with major revisions requiring a hard upgrade. +However, this may be revisited in the future. ### Transitions Networks would be expected to announce upgrades in advance allowing a window for -individual nodes to upgrade to a software version of Tendermint that would -comply with the announced block protocol upgrade. Node runners would temporarily -halt their node, change to the latest binary and start up again. Tendermint -could also offer a tool to easily swap between binaries thus reducing the amount -of down-time that nodes experience although this is outside the scope of this -RFC. +node operators to upgrade to a software version of Tendermint that would +comply with the announced block protocol upgrade. This can be done asynchronously +with node operators temporarily halting their node, changing to the new binary +and starting up again. Tendermint could also offer a tool to easily swap between +binaries thus reducing the amount of down-time that nodes experience although +this is outside the scope of this RFC. The actual switch would occur in the same manner as an App Version change, using the `EndBlockResponse{}` ABCI call to indicate to Tendermint that a different @@ -93,55 +80,23 @@ Given the delayed execution model, careful consideration would be needed to ensure that the nodes could transition smoothly. It may be that the actual upgrade might not take place till height: h + 2. -### Current Use of Versioning - -Records of versions are currently stored in `State` and in the `Header` of each -block. This already covers a broad base of scenarios. The consensus reactor can -use the version from state to ensure it reads `Vote`s and `Partsetheader`s -correctly. Similarly, fast sync, the block executor and evidence module, whom -all interact with block protocol data structures also all have access to state. -The next few sections will explore the aspects of Tendermint that will need to -be modified to prepare for versioning. - -#### RPC - -Supporting soft upgrades means that the RPC will need to be able to deliver -information across different versions thus the RPC should be versatile to -requests over all prior heights and must return the data structure alongside a -version number so that the receiver knows how to interpret the information. - -#### Light Clients - -Similar to RPC, light clients will also need to be wary of block protocol -changes and be able to handle them accordingly. Light clients should have the -same version as the latest block protocol that they support. They will also -require a similar method for verifying across block versions. +As we expect upgrades to always increment the block version by one, rather +than having the version number being passed in `EndBlockResponse{}`, another +option could be to have the height where we want the upgrade to occur. With this +approach validators are not reaching consensus on the version but the window of +time (as a height) that nodes will have to perform or prepare for the upgrade. -#### Evidence +We will now cover the two main methods of executing a soft upgrade. -Evidence is always a reference to a past action. It could be possible that the -past action whether it be sending duplicate votes or attempting to fool a light -client with an alternative block was of a different block version. To be able to -process this correctly, evidence might need to include the version of it's -contents or have a method of working out what version the contents should be in -order to process them accordingly (upgrade heights could be stored in state). +## Method 1: Multi Data Structure Support -#### Spec +This would require Tendermint to support all prior block protocol version. This +would be done by somehow finding what the version of the structure is and +processing it accordingly. This mimics a similar design model to the Cosmos SDK +where all legacy code is stored in the repository and conditionals are used to +indicate how messages are processed. -In the future, we expect a versioned spec, where each spec release will list the -software versions and protocol versions that correlate to that release. This -will reside in `spec/version.md`. A clear guide of the rules behind breaking -changes should also be there for easy reference. - -### Implementation Approach - -If this proposal is accepted, the implementation will need to be further fleshed -out in an ADR, however, it is helpful to illustrate here the approach that -Tendermint would most likely take to support soft upgrades. - -This would mostly likely mimic a similar design to what has been done in the -Cosmos SDK with keeping legacy code and using conditionals to handle transitions -between the versions. One would expect a directory structure for each module as +One could imagine a directory structure for each module as such: ```go @@ -180,46 +135,245 @@ func (blockExec *BlockExecutor) CreateProposalBlock(state State) (BlockI, error) In this case, Block which falls under a data structure of the block protocol could have many versions and is represented here as an interface. `State` and -`BlockExecutor` are structs outside of the block protocol. Breaking changes to -them could only occur in major releases. +`BlockExecutor` are structs outside of the block protocol so would be +internalized to allow for changes (so long as this wouldn't change the overall +behavior). -### Alternative Approach: Migration Scripts +### Current Use of Versioning -Although using migration scripts would have been most likely a simpler solution, -unfortunately in most cases this would result in breaking the hashes and is -therefore not a viable approach. +Records of versions are currently stored in `State` and in the `Header` of each +block. This already covers a broad base of scenarios. The consensus reactor can +use the version from state to ensure it reads `Vote`s and `Partsetheader`s +correctly. Similarly, fast sync, the block executor and evidence module, whom +all interact with block protocol data structures also all have access to state. +The next few sections will explore the aspects of Tendermint that will need to +be modified to prepare for multi versioning support. -### Concluding Remarks +### RPC -This option provides greater flexibility for the developers and potentially -reduces the amount of coordination and work required for validators and the -frequency with which hard forks need to occur. However, it is nevertheless -recommended that block protocol breaking changes still be grouped together and -remain as seldom as possible. +Supporting soft upgrades means that the RPC will need to be able to deliver +information across different versions. The RPC will therefore need to know which +heights corresponds to which version and then be able to communicate this to +the clients. This would most likely require clients to be versatile to handling +the different block versions. For example block explorers would also take on the +burden of supporting all block versions across a chain. -The implementation to support this would ideally need to be done prior to the -1.0 release. +### Light Clients -## Status +Similar to RPC, light clients will also need to be wary of block protocol +changes and be able to handle them accordingly. Depending on these changes, +there might be the need for special verification logic across signed headers +with different versions. -Proposed +### Implications -## Consequences +**Positive:** -### Positive +- It requires no changes to the database therefore keeps the immutable aspect + of the blockchain. +- Async upgrading. Nodes choose when they want to upgrade the binary and +Tendermint handles the actual transition. -- Chains get the benefits of upgrades without having to sacrifice their block -history +**Negative:** -### Negative +- Increased burden on RPC clients to be able to parse multiple versions +- Bloating of code base and binary. We would also need to figure out methods +of maintaining good code hygiene with respect to having all these versions. +- If we separate versions i.e. ABCI and Block and P2P this could potentially +compound the problem if we need to consider the different permutations and +how they can be supported. -- There is a large degree of complexity required in ensuring the flexibility of -data structures. This could lead to the bloating of the code-base and/or -decreasing how easy it is to follow. -### Neutral +## Method 2: Chain Migration -- There will still need to be hard chain upgrades (albeit fewer). +Chain migration might sound counter intuitive in the context of an immutable +blockchain; any change to a block would simply break the hashes. +This would then make the signatures for each block invalid and the whole thing +would just fall apart. Chain migration, however, would mean that the network +only supports a single version at the time which would greatly simplify things. + +The general strategy for a chain migration would be to get the validator set at +the height of the migration to not finalize the last block but to instead +finalize the entire migration from a prior version to the new version. + +![Chain Migration](images/chain_migration_1.png) + +Practically speaking, this would mean that `LastBlockID` ( The blue arrow at +height h) wouldn't be the hash of the last block but the hash of the newly +migrated block (v2 at height h - 1). This would tie in with the migrated block +before it and the block before that all the way down to +genesis (or essentially the entire migration). + +To achieve this, migration would start at height 1 and form the new block. +It would then increment to the next height and take the original block plus the +hash of the migrated block at the height below (as we can see with the green +arrows). + +The problem with this is that new nodes and light +clients which aren't at consensus and don't know what the original blocks were +would not be able to safely verify the derived blocks because the signatures +(the red lines) only link between the original versions (V1). This outlines +some basic imperatives with chain migration. + +## Imperatives + +- All state modifying data from the original block must be preserved +- Light client and fast sync verification models must be unaffected + +### Simple Approach + +One basic approach is that each derived version should be capable of producing +its original i.e. If the chain is at version 4 and a node is fast syncing and is +at block 100 which corresponds originally to version 2 then when it receives a +derived block it should be able to extract out the original for verification +purposes. However, the derived block should also reflect the structure of the +latest version. + +The shortcomings of this is that we would expect the derived block stored to be +larger than normal as it needs to hold the relevant data for both. This would go +against a lot of the proposed plans on the horizon which would aim to reduce the +overall size of the block. Verification would take a little longer but we +wouldn't need to worry about any byzantine behavior (either the derived +block can produce the original block or it can't). + +### Advanced Approach + +There is a second more complicated approach in which we don't need to be able to +rebuild the original block to verify all the way up to the migration height. +Currently we have what can be seen as a core verification path. These are a set +of fields or features that are essential for Tendermint's verification process. +This allows a node to be able to trust a header. The header is then filled with +whatever other hashes that allows the node to verify the other components i.e. +data, evidence, last commit, consensus, app. This guarantees state replication. + +To quickly recap how verification works we start with a trusted validator set. +From this we search for a commit and header that correlates to the height +directly above our current state. We rebuild the votes from the commit and +verify that the signatures are all for the hash of the header we received by +calling `VerifyCommit`: + +```go +func (vals *ValidatorSet) VerifyCommit(chainID string, blockID BlockID, + height int64, commit *Commit) error +``` + +We can then trust the header and as aforementioned trust the rest of the block. +Then Tendermint delivers the txs to the application which will in turn update +the validator set (with an extra height delay). We use NextValidatorsHash to +verify that we have the correct trusted validator set for the next height. + +If we were to migrate data to a new block, then we could copy the original +blockID across. This would mean that we could verify this BlockID but we would +not be able to trust the rest of the contents of the header / block which means +we wouldn't be able to know whether the state transition was correct or if the +new validator set could be trusted (which is critical to continue verification). + +A solution to this would be as follows: + +We have a block that has a set of unalterable fields. They can't change and are +essential to the core verification path. + +```go +type Block { + ... + DerivedBlockHash // derives from the original block that was signed by + // the validator set + NextValidatorHash + AppHash + ... + LastBlockID // This refers to the migrated block header not the original + LastCommit // or anything that houses the signatures + ... +} +``` + +I'm assuming the following relationship: + +```go +f(DerivedBlockHash, AppHash, NextValidatorHash) = Hash of the original header +``` + +Part of the migration script would then be to calculate this `DerivedBlockHash` +or if it was the migration of an already migrated block then just to carry it +across. + +Verification would be as follows: + +1. Starting from a trusted validator set (this could be from genesis) +2. Grab the migrated block at the next height or more specifically the +signatures, next validator hash and derived block hash +3. Calculate the original header hash that the signatures should be signing with +the next validator hash and the derived block hash. +4. Check that the LastBlockID is equal to the hash of the trusted header +(no need to check this is height is 1) +5. Verify that 2/3 signed the original header hash by running `VerifyCommit`. +If no error then we can trust at least that the original block ID is correct and +thus based on collision probability of hashed also assume that we can trust the +`NextValidatorHash` and the `AppHash`. +6. Apply block to current state to get the new state. +7. Check that the state's `ValidatorHash` (at height + 1 now) matches the +`NextValidatorsHash`. If so it means we can trust the new validator set. Check +that the `AppHash` matches. This means that application state is also correct. +If one of these doesn't match we drop the block and peer and continue looking +for another. (Remmeber light clients only have `NextValidatorHash` and have no +record of app state) +8. Go back to 1 and recur until we reach a point where the `DerivedBlockHash` +is nil. This indicates the crossover from the migrated blocks to the original +ones. +9. When validating this block we return to the normal verification process. +This means that instead of using the `DerivedBlockHash` we take the signatures +in the commit and verify it against the hash of the entire header. +This means that we not only trust the `NextValidatorHash` and `AppHash` but can +trust the entire header. This includes LastBlockID. +10. Verify that LastBlockID equates to the hash of the last migrated block. If +this is equal then we have essentially verified the entire migration history and +we can trust that the state transitions that have been applied are correct. +11. Proceed as normal until the node can switch to consensus. + +Another way of viewing `DerivedBlockHash` is as the remnants of the hashes/data +in the old header that we don't want to bring across because we have changed it. + +In terms of the actual upgrade, when consensus is reached on a block that causes +the block version to increment, the next block proposed should not only have the +new block structure but the LastBlockID should be that of the migrated block not +the original one. This means that before the next block of the new version can +be agreed upon everyone participating in consensus must have migrated the entire +history in order to generate the "golden hash", the cumulative hash of the +migrated history. If this is three years of blocks this migration could take a +long time before consensus can continue hence it's probably an important aspect +to attempt to make migration asynchronous so nodes already know this golden hash +before the upgrade actually happens. + +This could be done by executing a migration script that works "offline" by +iterating through the block store and making copies of the new header only so +that nodes know what the migrated header hash would be and when consensus +reached the upgrade height, could smoothly transition to the newly migrated +blocks + +### Implications + +**Positive** + +- The network only needs to concern itself with the latest block version +- Some migrations would be backwards compatible meaning we could restore the +prior block version. This would allow external clients on prior block versions +to still be able to process the blocks. +- Async upgrading. Nodes choose when they want to upgrade the binary and +Tendermint handles the actual transition. + +**Negative** + +- Nodes may have to go through millions of blocks requiring extra memory and +computation over the transition phase +- External clients would most likely need to support both block versions over +the transition period +- Bootstrapping nodes and light clients require an extra step for verification. +- We've cornered off some untouchable fields in the block structure. If we +really wanted to change these we would need a hard upgrade. + +## Status + +Proposed ## References diff --git a/rfc/images/chain_migation_1.png b/rfc/images/chain_migation_1.png new file mode 100644 index 0000000000000000000000000000000000000000..1c0dff209a4db6859aece7c1434aa1de9dfb0200 GIT binary patch literal 33334 zcmbrm1yq#Z`acTe&=NyRh;)i5ozgAcATS`(As{88^pKLG2na~$fP|C`T?*1EAc82} z9e0m@&pqdN?)u-g?)|ZrxaQ@Jz4yEKv!CY^6RD-4M2Js=kAZ98X9Y zz~l=ItZ{6usk{^1rh{?QD$90k``M3`0`|il{vq~^;3PL@>7ZCzzBy^A{N>iwrKV!c zjqJd1;#(TZf9}!EARUyaOA-VnfkWS7*foXW9$&}AFCD}!sQ>4AIN&3Fx+wnN2mb&1 zYm_DKHhHCG=rA1<{2GMZh*tV>>Q83W5&X-_Yf91}YgiVFEt+BLh!qk84^R=0WrMge zf>smUF@FI4+fxQ1CH?I^Ge6fhBb3qtCs+%_0e!1mn^bPFBJ6&p44#LC7aR zE-ZHb?S?eeO6~=OmR6%F3U^8FMFHYB3HXCN`WPW&pa!&!0W{|~<$oW;edQtpMMw*P zrw@@VuKep>q|z@Se@Bbt&ujYMoUl!PF95vmAUj+w_GejB!t|P;Ut^3=O|J@0hO!>s%J|u*g_)Y?$$D%OY5RCbdJk$R^ z#%$Dp1!I5=fiYKm8 z8+>=3*2{}E{=nQm#S-@krcjKQBGQJe4|kmEd{|iy^qEH18p_{O@#dT_STM=ZIm_s(9$Nz@aXHFa=!Zi#2yIBDJ}VRzhfmg$hvBE3u# zwg7Pw+F6=-5XgzN*;OsMF86`}X&JgpT{?yS#e1BWAjF8$?GVCT8*w?o>U*$A-I~_R zT~6AR>`C`WdjZcejyPlDQh@uo3?KZt6te7R?MO}xGRJ>8JMg;YVn!G9T znvVd#l=H#Qx{!a3*)e_T7mTw~Km5`+p4yswHe?c;dScytplPJT%0LFIq@;yWbBz9MP0YS$X%| z+*sVQ4cfXjD0!6LSa#I9KK``cJWb;rr%~b5Y%D5#u1@vmyXVC!i`2I}4%da8SuduE zbgDn^kQ;fAq>iqivmnoowo*L?B@L^ps#p$MYzfW%HUxjqH8(bGw-F4N8uN3SHB^&x z8?4`WAa%Yi=eu6nZGE))EjT1RNLtz~DJkjwW{GY=Tnsfo8|R(UrYXK4Ldhiow?+Ar z;~y^{pYH!Q7)S|VeSrb3Jt?K)MB>2>CwaWmnJ%pcZnGB4Sau#h85({(DsWznVfrA2 zyMb9&(E@9u5y?FE@4^X7(G|Gh8n z^8rUigN()fuXvJTX@!4A*3u#hPz^y?v@6mAT=0nV84zPPg6Z4qHwJZq-<}?YA zk}>s-hEhAUT*%2~I&rVk(uegrT!!VP%28~{k2c`d41aWcG*$ID+S=AuZEd{l@l36= zPPQzTAxu5Ulj7#MW;n!AGXK}mDz4YoqJmDjLVpvYW})<=8e^vYb)(+_S&K~2&3jg% z3plo(;n%4Xm9(_r{mGo~yGE`KI7=qvHM7w}gzkao+IR|=esQ5~ zor@E>k#LpO_jC>Lxex^Ff}HyHy*zW5$KY*GaLr%>CpYUPp(jG+PR+ar)r0Kw_ck+w~Yqay>Kek}O>J zl!644S_%!c;qPUWzhcF}$%12r6~G?unD2NO9l{ln;X8f*wMmd0wfihXoNi>3gLrmP z;R!z5|D>#>hJp~ZaBuNL1CxRgPp@z3U&a&Glry;c0_usn(xzSr+5Lo|b zct+85aZ|zDae3fGCxS^9AR;#DzzKh(pyRy`{0rND3;M|gZwe4!7+!L`OiTwZC!evJG z4|w1gc#51x_}A+}kJadcrVaPwol9(3=Yqn;Jv=-FW7-&>GNEt{*r$nNI*;HtF*4-X z!lXe|-TyTtm;c}m6Z<=Vcbxz%*G=K?JZ~hBk=u?@tWT-Pr4ADOXYg8<-|XoK}9NPb?}B=C^#8 zV4^LBFHe$I?{H`MdiZ|p$?#PM$ZU66MWw}SknrW@b;AT1$*u^`<& zD#|?MOEI`07hGV$j0Zg?A2%y~D@luAI(4oCcV0||l^&qN6)Rfj%A%|okw4A^6Up&Q z4>~Hl5$Fxnn+VKLGAD#D!dz2gN4~X$i$q_(@^SwwebY{#J8#!NOkySp^^lPp$u)I~ z+i1a?+x2Dn^H6M1TG!zil&+>EYX}bUEA97+y&z2FTK6XI&bYSp?*ATH{&cpPlr0z0 z7tbh1nmP8Om)82>yYR#`Qss}tbIQ;~1&|%{TG2OVLe}oJ`Ae9|SG4MWVpUDnf|xe> zxT%p)i}=2e*%$%q2rq4&#Q-e%N~YzZH4Y>_>%huL*Z2IbPiGn$;u`{L@vobf}?SkB45 zQIqr!Lz=3Rj4k+m!xwgF-Gs*~CGb=&`k0?_yEnn-u0LQBl5p?*LBin8+*U zdzZ`o1lEIXsk1*whz42W6PXG|OL=jE)vNcv! zl~z0|?*28V@UYRI%cATdW243SJpZJ8U3iGly%(5qQWB#F?k7K+ zs{&UrMH4gLN0kW|nQKf7E7t{F(xKksmi``AVPRJquEx$@){13U#~ zV{OxaF;csPxBhGH!072W((upgp<5u6E^!7s%SI4-vb73Mf;k+0`HC^>+fJ-NiMDyU z)xHC-+q&c}$Sjl4EF%Z4UdQ6Mj~xR;qA`o{7G@QLk*}b$#w>c z35)ja6uR!2cFEereHG{^o)<@))$V#zc-WR~I}bIS@a4cAHzD#o%^|fu^TxZ!OTzb? zn%o7k#;$WK`y zE)b+$Jj%TjGdZ+RH2j^y0@_9ovafyvwPz}Dy;SYT1Qy#q-xS34dy`-@>FtymYKR|C%^Cpk~v>iO*e#imV0lAUI*Y#V>>J&`6 z8@n%UDP1^E1V_j~Zn3*+TJpfG5Hzzc4qcl4chh@u$6r3wg8%DcCHef1pS%(G zc4-R;oqKKf{H4S$;{1GXkk4(gBX>n6NS#L3Yk>{ggw$dUPA&3c4GxPu@`;@1Z8_H& zkh(he_e`Dl+iK3m^4gfL;d7e2>wEOgnB2T zn0g$`g1jmAG#*_u;4SeFN-%#(6HHoPck=y}9bvuPm(sTivcaAt&HJ3}nLbwZ_y`1V z{SxWP&$q5iecTnvnAfm-GQ8(F!4FrOe5n6~nRjpZt>>kiQ%$}f_BUn>?;kH*J8%W{ zNn=#8hr4^OMN8mtjpKy>1&bqmPWP<}rcsBn;$F}PW{_7#G=YuSeYcmr z}d0* z0o-r@7h{dXm`?E7n4v`x$Bjo3hewxZTOeLm+b5Aul$qA47HDKx&wmO~J(oIL=l4BX zPF=2OxzzblYc~`OdE0ZVAeBxOc6J{4vIxDd<}b!rf|@~c{djAxxsL^xCUCyluddo@ zO6P*AE!=CV2ga$L6H5B!tAQ5W?P|i}c?MObBI%@|MS#9b{W2x%`mqd4EQdgKi_)zC zJ+=q08Ei8=4d&vnP6nSRe_cEU+pg-_;l`Nld&StqcY&vo=oYY370YYw4M^tiIu@JF z?cDaWU1be?QQo70!iM(J1Rbr%iuHGglx`Ib1`j^H6?klUvKY;)XHDxwvIUA( z0;VlSx$qt^Ze{mnLodF0&uhcU++QmjynwM166K2T2cfGR&VCg%F@`IeTbC{3~7ES@O5Awi1~6_hDx58msQGgHtW z-0`Dw8%~Ud^YLMo4<8C~wK-{{&+5T^{ij)xZt;iTY2R$N|g21ZQbf3Bg#S z--Pl3K4G4n!w$00XY39whftN;t|LnssXn}{=PL~VTa)kJ=>5~Rik3~>oTt`G@Cl(p z!;_oMRTWFBaic1(jD%dZ_Z97$9a4oBCGV4*9Eb$L@+;KgSLb`0p4K$PX**0?w60Cx z|Ks9s4Xzob^EE8I7qgFlf624#6!Mi_({HSXZS4g0RCMD#P=qt*p`vg|)A8GXe^5=~ z>W$|8q#P8BBVkU<4>30{>?)&SiKrstGS!6?t+sd$^QN9sn5ykmlb*P@bX~{g-G@TV z4^1^O*1xhmibCH@)p{VBl1E3Wn!1%On0_lV9D}krq2KkTm+|nst*c++Q%6Hb?wbm> zJ)%?@Z=OC+5W|J)b8FYVYP-He^vz(QZ4jRL|wb*>X6oi&m?CE2WxF zfZnD|4*t@;JLVGSCs6Up4-T`hp+Y+NrSjL`qu7FJwjwDo%?MI%Zng3S$EG~qD<4^| zYltVncPi4!(=%XcGEGCEn98h7Owyip>ldI*1YI6LXiZjl1+WY_;60^5&Ts)1{jLw= zPhue@{hb%0aphrst4Hv<4O8|t+00feN0EJG64>~D3l5lW7zAqLt)zPlz?RMw*jNQv zeC>vFBUPVImKgF<YWp$j) zdJO6Z$KGRjrVvf}B^sXKhG(2bdFLSqA+3;}RrHyb4bdJ!*(b8w*e`tOjA6#x$iclK-F9IxPQsw>NNE<@kMv`WZeC&roG#d>6}Q#%lgv^4q;xsu*x>D zwAVf$XTJnSlRhC~!6hFUWO%nlz}u9CD@QFR|I+VpUArxm&s8`70UkFJAqxt2^ZhBn`uhPl1I4)#I!Qv#(K$Zowr^_qUX zTW&vIyR1;Ph5EN3G!ouZiH5*@2QtJB0jR}HW$u&RCm6#eig?=Y43mSmgSJT;EJ3@* zZ!-;pRb-@qZ`cf9KjkImWo z4+759dd?p2d;E|AR~JWIx9+*q=)%d4)!Z7H+2N$Fcrqw-lcERw$T92oqE-0|qe7I) zU~Dv@gzTZ(1r%3At+Kh|o~xrsNOVZh_u}!a2Xq4`3_6N3Uh)+r_tRw+hXc-Mzy?^{ z)!~@=nxNKO+Y+a(!FSJwbv*$}3FFNyVzN?5eD04ats>4M865xemXvve*G931*aU!a zoJ9@GABws@MEK{#!T4gp;3jD zGaZwg3dxZ?woZgpw_OdtNKI}J@IS+b>qnOC;n=%uQggzJM>PL z(!Z+ub@(QYKjX&Fzs}wr4I?+w9O~>LYvQ8?d5b zjoU|5UW9yVjGfGhsv=VyNR6DqN**OtXc!kANB;EV`fEPba%9J`hH&Cr_vtE!HL3&_ z&1Nmz!;6j{W&k{peQ0I!6?2wj_9CZMf6<<2M8mldY0vZX9i@&wksj;NRiRq}`w6Nn zL7&wTdPDJbR40*c>L7Q}_e=|h-A)vFx;1C^jX*rX26hr9$841RUQUn>AwX)F#mf>X zJ@&S(O9>NmlrqBThwbiCFIUSpYwE)s9fq7?qMuQs%n4W~zo>%lJ?bi)-=QwXY6fU$ z@MvsLbc{isDn-f(-q=$n{Uy%NZk)(Zmmh=%eLxOAA0jyDonEb+Ipf%E?~+kB z)cah&Q9Gk2CTA@udq5)}aV(t66pEYkfb*1t1NrN$X2M+eO)|*S$0R{+rBg%_V%kB{ zhu+@vROF4tS$-&KSm8S5?l4YzfAdTwmD3w+tr?Wqi&6EwLDMQLO+Yqj`!2xX3vGRb zo7B1tC1(kA+|1EBK3I--_uI|pXNyq6?<3Nsj z+28d{*}f}t>E80`I_V(OEF_O?b@w!_b<&Q!BEY|09tLM>;muLYWe8P{P(&i=!I*2~ z;Zrg_{LIdQ{N5$CQ9Yt}K!@P$LLHL?%1`g$-gf5(wT0=_c-3Py{UIEE|F2q+1@e)k zs@0%Pkw2@+C+%=|JTw^>!o&KAP@pCvDs7r97&)gjpvDP~l0PnZ7x*j_wdl>#t0W(_ zDrVPnD<(?(JCUg68Rbdn4!`XHU5ec;CdwGNb_9hsQ}JPzr70zZj(qaT4>>)Mg%{%I zGZXXG=C~p8V77%QS)Y5n2mu}P!Z z+nLdeQ%=b6+kL~9jTG9>T5$9p$fQYbrlkGC=1W@9%Da#3!^g^#FLTyD;w(n2bkGVp zX(@cGY!8)%z9uN%WRMM4ew4{^{bgkv-tF2@i)X)CJ*DukE!5wQ`V44+ZHweCA$lcClZBVzrIkH7xz`Q?*< z2Iqfi0XQZ74)xk1HpW4rqiOT`b#`VwHWnq!r6usf`bSqRSCzv2a(`-SLFO~A9F#0V zTLot^s?0(D9h#GQRvYagF7hBO-0e?ND*DK@UbjGF)!0=eNI|N#{XyY)NSPDeVjE+D zGKKGr)^LV}YUY^(oW8)9OmKARr_drexDMQZ zP*$c+{g{0Twg*N~*f-q8qQqUuf@LNx`m$%P<@ktaZY)|?q|#!I_piHc#^K%8gpHh( z%D=G4DwQ9R4A6#uYsa#+0wgOX7hnf5jn|ooG0g(idJX`DIuJ18@X?M z1TtP*q?pSctZ_Gd5ln8_-9KOgcm0vO#nMFox{O!*VlPR6wfZfvp!x-C*| zH7)9=!2>NCeL5WZecn;@GJcJ}58G;4tf3ZQL11_(w;(3`=lu8755zT?e`FZsipSJU zSyEvM8SMLJu#;>L%Tf?$ER9<}Jd5!0)m0)=NEdwNC&54S<7)Y=d^OJkqr{_foxM3o~U? zx8v~M>Zi;)|C4a1bF2DMiGs-LkI46|WV!PukVi&Agk_2$QT*s^uG(=z!z?|5?WFJlO~1O=WCYCDrv4t=tuLXLkcG@ zL0R*l{^SZK`YAy97OCK_T?qS zG2^A(+V5zpl^#RxS1JPu%%sZdDIU|$#G^K(+(I7w0%*y7DkDYY;W-}Tvu{m)dDb?z z%Bl?d3Y9!O2&><9<2(AVQ2-TbFLt_Fi%9rMGSgb97-IV&68Cl#4uhza{9zjb?`M9< zOpD`qsW$80S1mQlL61}()BI28?~lc~Ru<|V?6{o*0SfLfucnzjAt^hzOX`7Sg{pg!_?L#DFf@*pl&u4gYCMrv?@#m1L z>MAKQ%n^C=CMq@h=}Ac$`yY%>%bo}HtMmh|E4$_SiOy6rJX3FVSbgAARaxn(Pwd0W zCz1X&?qUA-yyMd6>!Z_0%A3{S&rEFmn>Rtak{qs!Lu7iyz`X z>ys>F>p8P>`6UdzZ0V&(lAbe3uWh=v;cUXBYWmchuAT0;IOyNc8^ z1(+a{4854WM;b~OPNTz;#QUGYx=P&dacq7p9o?O$H^leqx_nCwRMCr+9Y~`@FxB?m0W!z5GH?b2 z7R^l(O*yI8okYj^f&lBtYq2gV0hC(mnKD6R(CU0kpj#mTWQ@80khr^-u^&IDt|BbhVrjp;ue9N^)KdFh#z;*tTpr9*)uZGV4B zsOw?;F+}K*i?TzI^$isB!}ll$lb#ruQ4`KgT|#c|-J~S8iJj%UuLy2m9Nv(^r)rhe ziy*#r2P7o*syW2j-zpzP6lZ?$Tr)`70vlSrs(<{)TTEezPkUI;LHzDTTDDscq>FO7 z-rWp&A@^GAoX$!DRMGx>D5IA1-DpyaT)k4G%ADf(5OB#O37{PDyigbZ#$uc97G9i7 zrU6gbzxptEKMBNKV}OS8>U-kUSZO!o)L1eOtYrF?lPbhuHbznMq*1jpw~t^^J!BxD zx}%r3$x!=|I5_qsTk7))0KiiLzT4|q(VQPjYjHcCom&5yR%sR--PKPqq(*FMUzY?g zZ+1}z<<0-}D0~8;E%gc^cqw)1!B<|~Bdb(#Ly0xeh)_hc5`f{6<>tJPW~l_eT4Fj;7H|r=_n@rTVhWO zt6iT9)L}r0hq#!GaPMq(`@;v%Pxka#qUy2gdJ^_-bYkAMLAFxrK^pdl)%2I?N6Qr2 z{o`|BFbStmhtrNJBK*OgPnvU3Vz~Ds4(Y%$Ipd5zyiVCU2=mm)5YyMMXcclqT9#1Z zT-^C|Y{hELfsB(tO`0oW(K;Piy`@@=4(FL|xwu6VK>7=0$Sv$I0#5qJ+ldhP85C80XJu+(W6Wlb2kZ^yEh5 z_^4VpWCQU}-0vM;f*rJ`D=4N(ExW*r(}PtUn0Sr}!nw5zcRWzafLRMJh*@TTBa^9? z$_~(`$2s2VIh}_*s=YE!vPnBX_Oeh`{k5ibE~`__wFpKw4MW z%8D-=v-P_AJFT@mOb>7c`(`ZtXcPFHIze@lE*n0Hv(fz;XjXKN9!1|cuw^Vo3^T50 zL!vjf0K_fDI~?u&+qORi zPV9DH|L=l#<#&icCBbw5Z2g0SA0^HkB-ZtWB7?HDC<9o)A`2gp^mi>1)wb`_FeX*` z%wun-Y%2m0AYOT*_>uC_toM9?Z|5iskNC!2>AqJyC8uphBv18zwZ$3M4EOVTfj3_)W-Mz9kS`K-Zr)23RY&Qhyz8c%T9Pd}6ir9kxJO$u@Xb z6~1NF>Z$JXW5%*{K%o@+Dsav8xqIIe{8F~1W-g@brUXNO-Fk}G!APF{f&1|DMeH|U zTN0d3kG!j%j?mkiUOluoXl`O77mcwXQ8oY}GjNC8dqyQ_tmCC2*1wos3Q-j`!k+9H zCr)DteZCLwEaL1+UHj#{tWhW8J1uNfyJL5I@o!&@8*VO|JOZ3(;Z89Mk|{j*jn{Gh zOc7~^MZ4N6RKI`Qetug*9oPMa6k~iTYS{H+iD$DW1yT}8%S%uig`Mx5P|7UoYnPV` zwSHRhEXG{$v_ul?QBN{VHeJ-E33i z(1IWBKuO$8Zp4V(H}w-$&G%+`iqOMje-lX7JW<{!OFk)29l3YEg1|}9rgm1!SuGP4 zw`n#wp)hem@09)=!_C|!1+?o2)O=)FSrt}98kf(xM9XGkP$It~AEZuRFS51QfNGn4 zjIi?dXa3@loLb^f*9;+0M7mvKcKFuoL)2P$=#G_BY@Sdh3NW-nNjVt!hd~~rR~3N< zHm)lv&%Ib8S4Q3{awp=W&t@V{l@J`7{p)3$>~lgB7uVhkzNvgh*3pSFG&5L3%D zm&b%yKa@>t2QX^kExj>3=N?wNa#Er$8GZU(Q~5)-Tu=GL7&N`tD3#eK%R%=-l=(OI zE;VyAGpS{$BiP8Vw#k}jq5sV$v*2TGF+;RrNF6Y`-@hw{=YmFSDPY^h3b5#940uIg}at2sK`O?sk!PU{C_k0@J*7U_E@KPSrTKQN2 zgi^|;WY9}K!*!>Udl*E#d*_kHF+>}Aq5*FYA~~q|7EIvMgsl2}@iY}nvNGBZcfmKx z{tIV|Afb}*%#YA{#bSZIU+-TQySpf(USbD1Oq82tqL^%-Aom~U_K5=hqz>M;9QT)V zP)+SlK6+WSADj(211QYW@?qFQj+HvVc%geQCUIzo8Du{L*edr~j2<4&Ni-VTfRYfx zcy!pAzF68t*HnEiHU6N7Cn-PC8crTfGQGR}@(a#_T99X3!ngAX=iylfc{VgP$97`Q zS1Z!Wn|Q8NED*9``5AAf)ozl=1xNxjQ8`U?Tr*iR^6mNqGMwV)XTtOTNjP6XEUR$G zVm2Y9Fj|sU?1;t=>fGXxjQBv*gGJ}m6LLM_I)#z^#{{nLS#jJs=JFu#BD|2Cbgi~U zwl;m{$cvE0m~0~wO0Ep~$QMkZ&G7mo2K!#O^KkPYs_KA-Vu zruD+1DCbqM=Zh1&$ajJ!E!4Z*hR`}bW}J*g-1Kz3Cp;NjX1x&11jV9dp0vK?@~K!O zd1944j2T?y^&2%n@CXb|CSW=2u}zyehz)^O$P~Y z7Oqn~%M3iVuOTBT*M>K5PjriR#k?pFQw3TrGq29;0=(4#6W@tmNhje+j*)BkP(rXW z37HY*NVqR4Qv(eq9pI>ovAHw*7Z2zw(p9ohtGkqO@3GKS(dH4{b1E;4g~)x~Ag7<+ zOlDT=8U>8ozd|+)e>ElJYrCYF9o>EZj|=)t=>}cIwQsdazH!Gw)+q8t249nWWtESC zKgtQA>O{P_(-P|@*7TVPWVV!;y)Lu=K)dzrj_L2PPCPTOiQ9mr|C;{c?;DW(J9>c9 zfBpWW6MT1#Pxf{i5!N^p!&Iy|$zMdp+b4GK14GMz2fX__2huC6@BU4>zZf!|2YD*t zgTR+I#`vkwHm>4$sd}&;cUy688I3(24dg&-GcID8{{uKTEtz@^qyWv?#y+oez(_)k z%tnuN!|^hwmC`RkFDQ z-vH|5t1AnQexP3XlcSP$Uh;XzjRxO=1jBH6Yf9z!?uw0`WzQuKymJ2U^HeM237W}usGZ7vq+yMKFWeHbpeG^4L z9u}+|Ug8s?7oEWMGYgx0m+-w7M)3-RiNZ2VLKDbDbQBomE0ljAZtK~RtlkeezY9>1g0Nk=ZClpD43+UI9f z?iB}rM8^e!jbL1p$0tfe4^TAgzd;q>5hcx`J<;svU88WlQhqKmR ztILaBFbhi7)qul5Yw8`4jxzx`F}@o30Di>qTmSt0X4OL;*+UxEC(LpC3D67m1KRaR z*nI68NG!2=#JwPx6?gRB8y==S>`f5+&66wBIo^$1X1L+rhSMaHZ*d%kQY?PG2e6^S z+nTLQ=fdBfCXwnc5tp0RmG=?H%(kO7Oryp6YULjn@mY~1W8d2&`a#KC&UGejr;Upu z1i=rRQUs@;1T1CafGD4}Po7nQ6QsRgzUUi0d^Puw#=t3wy}Dt&DhAh!oC*^7y$%^A zigDdHB;lDGU^S}TmsRzriGrhPDshPElL04NWQ7S-`v%9daP5|L7j0ALqHJr zoL*=H>-Y(3w%$WuCos98{na+G4e$WYA$K=miPyZ&C8HtVV`ZQi!>?6Nqwp+h*9YjP zJf`&o1NDMxRyP%_j7h@IU()i`{< z-<<9@MD0GHU^^{j&JfR(qS0(rUg3!YP@Lq9tx4WB9ZSeet_qpHR4s@Z#a)gg`RZc^ zcQY&J-1~XBw4e7TvZgF7zTqz3*MckPEkoAxT25C>iccl*S8}Q_xo$({{LJobQYnpx z70h()@U_Iizx`A{{rTf@ODs-a8k$n>@DIg#mq1Y^Jv) z{8YaU-F(l?4t;Cuvdh&XqL9uks*_rPko@6jpfOq!=*WuAc1V$gG6zCzG?x7M>TIip z_xR>a(~gitVHa!-4LNLNzw^xx7%XS{5*tUy>p0giS8~29Y2cWvFUWOm7bWugZr}sRJM;S18OTjCkANxyD9KP0i;zC!udS z7pbV^1@R1cl!lh1{ocFo{Zc;g+ZWB84<8=D->YWBLN$)+gcvw@W zo9B+>fgf_{>(FDvYTs=SKN5zP!-dtS6(&$GunP_Yg3j}lTkoM8*L7Z^HK(W68Fp*8 z)Tmm#ns7{Sm^7>p58hCMLV&1J3IWswZ`759Vu#KTf1xGdQ-QfNVrGFQGk02EDg_-h z;@q%l`9qsf{|v-?D;G!>J}@S%12ZYj5$&$y+;k>#Qc+R#JGQA ziD7I}d9ku+%89(g7NHw`>mS+YJ4^m8hL-T%v}hXl`wma7KlBQvSm&Couoi#zX% zOGz!ahOV*JicA##CDIM`WCPqWf$aex7jK$Q$GQHy&> zlj*Q5ddlkKotB!%yXyi>0HG0!&_)LhfOKlR{P=he@K$be0YZPAuEO6Q{votRBvmQ` z8qN(W^yy+DYPas<|7|E=-$~-#@N+iI%xKp@2AmwcuMa8{#HyZ)0f}vQ@@d=Cn}KTt z3})83ThMX5b&oi@gqQEqZv$xzuaj>;%g3vV%(Rqw)k{Z=Mtz{qltV z0p~l!6~9=CYXcN((PFdvrcCPGb*W(gCZ$gfe0%q4(>*-!B2->b9gNRD1V-7>$1*{k zfcmASDt^BXGHSnO5e}q5tZWOAVShL?Zuvo5(=@0Ag?M<-}>{IKXwq@t3MpFQafC3PRxCa5N1t#!p08uw5#n3Ob zrE6>ZHO+uGHTkO!T(BMk%Yf`FT^BBBl;YQ4D$JYj8Wf1=2Xs6%)kV`tg|vaC!NEPq znK@kLz%N4Cp*Jy2vD)^={r)N(!^5;&;MO1LD|f$L?4F9BWn_d#D{A7AClvo??=Rc9 zq?jlLQ?51TxDzu`XB5_TKA8SBtmK77pGN%jbNhqtbEYex+S^MmlrbmqBAc2?s+;da*c=X)k`>RYbdHgB9m#0jsivrdOmrZZM+_#WL@= zVu6?XNs!&(r9&dl| z&ayi?Pd|RN)AriyM|sIreuiVusqdCC`X0PK6}}$1D2FKERQ`!0HibL!TdLD;&)Xqz z=;w0EAa$$QvJBmXDtOBVImXHs{@8!URX%!2K)rp^^7#6=kwd`a8(=PcL-)>+hRuYf z(4MA?NG?e5m5Bv4m`d`(|6`{3F@vFb2s}M_c0`Y$U`&|p_M2hg;yXZY@b77NI)@7Y zPpT}G10=2e2I%HEE=_O#GSbBLD?z4Y_s zEA`ZS>OF1%l>;xY0bp0g%|A{0`GE7%B-+V?2=mj`)Clcl2Qu342G^ugp^;`ZFO~sW zDHK3m?!er~S6Gc0AyfaGO(3 z1`lDpH&oa7@xu}rWlqap-t1w`*M$qhzIz4{#$rPhqI%vppWkc-a`5KoA_jQqAi2r$RJwJDoJ(2=~)!{;IX8WNWg->Td>tj~`JO8aU01LbK z&w7|&qJ4f0XrSLC_DpPx}sc2LK=a1AgEI*F(?C7-gy#Ac3DWti&?`wBk7NC!0_Vp zs&8Wg+S{;JR@YaoU$O*BhvdesPo~&JbcG8>NO({Tt+0l>yZaK5Fr-G3v8w{#!mbPj zP3t9aQVavR@Knq57PM{TkA>p?fanSxHJ>#C&~&m2U(vkW=X6nB^T3nG^;ef?%cUO% z5<&Ic14bruLYQeX?xwgZT73%?r}4viYW<@{x_O`+Af;*jba6Pt=d%q*n;2L9mUn&CfphN;Z*TyhEJm+>s40Q?^T-|3LJ>(m`P#od ziK^Z$y9%PLyHN$9B|Ck>+yVrnIby*2asnE?oD+RVM>e`_3IM)6`RZ?zKe~Kt&=>Pu zWdQ_T(${aPT0sHd!>E>knmZYfjEw`ovXkImGiX|h79eb;BX2^hKoRKPNnxmL33F$drW#O;w9Q%B`6(_jeyix$|INhNcWB>x#;XjTw3At)ZGEe3fAh9g( z^ymW?DOrV5k}km9n-m*XWWFrY3JSB6y$woZZewHPRcUe}i88ZBgK(moTxd^2M9SBR zYWoqrw^aZlMF~7>=-%qH*ek)OW~r8E_*lK+xV@V$e3k2*OYFo0PN84 zn*^|E_?n_7A>CVuHaUwCnGU~U74~I?hN0Q`KVM0+$@y1YD!EKq{;m|1T#_W=>hTaq z@a;Xq-G9ow^UVsGv;P3S{$-pSg@pg9KK`pD1Wr0F&7cFUFPHa*|0n~md~CkIN<`Nw z|8*w#5*``w?H^PEXWhzXvK_h|mphy{U#;$%iaW~?SvD1i9BJ<~T%Y47G+1nH%mehs zhrW9ctL=t976dJkfI|Gyq=RVB^O))gtPgj?h!UL=fs=> zb%4G1bM{mBbzQ%QkiSq3(kGi2tHisY3^@PW!V8tR@iiTBd;_p?EX?n+#W5eD&?TQaxQ6eM||JI%JdT#vvQs5#*J5tF>;Mc#n{Jk(h%ouX_@84qSJ!M}`7r6Uv(D*j@ zlH45B20*s*pmb*$Dv-BXCRHu(0ge9o(qFR4d&VJ}={z?p)btQ8ky&}`s^I93!>E{c z%4HeC>PS>Upn~5pLL`pD1V`#Q`cP#wPE!h8usqa5?riyF+Rnm@7x=qrsNFu!Jva^8 zm!a6h&tf){^pMbqkm2PY#)Iw}O8kQ#ejr_ z{SAY!0Ed*hGIe(`o1a}j*P~NHBto1fRHdWoL01^I-;iTU?rDx&dm(5u_b3_7X-%vd z&odbDS8u<~)%ns0UIiyfPEM6q<>3>aEuP}0(&R0CB%G6t2bCCLC+x92tsGfy zygs!ualct*!}J?*y#6=haDV@`PK;ICQJ+LFi6_vH-%!X<3E<_H#@h|MPwl|^p@QSHtCOc)qloQhVB5g< zDu7i?-Ak_f6FTVC?2DkHU6u9|GjqqUx1kVN0?ELguhA^!6Zeyyq%6O2@@My2aHv0nf$99k{e-I5OMKo@YurN-=tN$j>m|Js^?i8 zyhz(`&0Kn==xy%r;b2yylmF3wObF9bGaf(qs}B%UP|J$}n3gN}z63&Hn$IaMZ?i9j zBQ-OVi_AxtKtNfXgF`7<>OyI_-Yc}CcsD**C#a1#eywu4+VZ=8k?LoYarhGKwy}-D z=6KS0{b<>s!`6v6-z03?ZQY$6w|98CG4>J7b4B9R?;L+dy25cYLJ}FT7{qd?D(fAZ%QaRNzC8aLOPh5Pg37TkEzjf|tF*|P?om7HUElh* z=qV2&=>J{gari-IyAK9FHpuyqFQHc%l*YY>pg=~Z($3$H3EIC;8C;v#bP)wXzs&2s zh-&gh?c>THh3CFOElnuyl*k1x#Bz^z)H{4qRq2rP-%AE7vjzlCmMf~;Q&FiAKAA#6 zODu+}S`z1L4?spYSb!3$$@kIcX(;ZI+>v&PIUn*7T0&PNh3o^B?`)0YjncK#_jc9- z&r*>U%nKj<)_10TU#O+iS1A%>(TLyBClH#{tERSJVD)8^@iuxLfy1^r{vsz%mXOlS zs5O+Xm@Te*J4SYxb&gDh>YiE++QskDzfpB?zSJHulVb!4iV0#AN^EL=2XHc?;8(+c zfebfFB)bvVuU=x#jx=*dAt&}-u!(QVcD|LQ`|mwL|7ZwWx3lUKGjk@6X7=7Gx3b>n znBDc}?*QeV(4r7$c=2PQQU(Htf&D#~&SQ00eC`;x^YWB;l}llBOk(1aJY)I@L_xGOY^9WSW+p1<$SjaR33j?L}0exWVeq7U~ z5znb*i?{bD@9nH#LYE-c4XwvF^Ij)H6{yFv1{fUtS?+wo7a_&1OiCdP12ODos7{vq|SS(L7t=IXJ|B zOcNM=`9fgw3*O7w%Ira3t;Rsp?L!&Ob|`Rj(x^QZHvwVzns58QAcnR1vjyyrMy(4y zU8C!-URO_J{VK1hN`+UWAAf>f?gcFIDoI~euK|h1L?uxlWGzOrCyz&fqKm0iE+3Gdy_U72A z{_AzD51A3F)Av;NU%-bVB}VE?EdXWiH$$yCcA%swfag9UuypX3$OlF z#~SvZR~zE>pBzK3F%^W$6((5%)1Nr{uB&X^&VOuW{2-@!SZ3Z49OU;U6gT}*QQ>WY zommdQ2eRbOO~-G%jXoB38X(e6WIQ%c;Du5}R(YZ8Uf0f==OVm*F*K5=2yrF9&=&W5 zlFU{z%CJR`OyQer1Db*ezCh|hu`_c+jD7-B(CM#fxAMj;ifY&ayI6(owg59 zEg$mVmgm5C_V#WlFTuR2-f)|XinFs{1$49+1r^U=>(fsYv#}ngu7gKBuAH8?OwfQ8162kxA z?g=zu)nnM*?j>SDG_l;+pq-zmsvLr$wF2VEc{*>vg;7!!pc<~vhPVKqc;RbVPK8sU zEtXRY_xF0;togP2wFkwoU>4=eW3!&Bc}Xp7ZTV|^zPfxpqG2#cvVtnUklJS<8`oO* zbwGPx7Nc%)XKy%7wU^vKON@W?$Yq63hblX#D3;JNeZxyYttd-Iwd?pS(Xn1SqA)xZ z9}_Ch6Cqig=@h1rue9l~#t02LQc&XSO1mztLDpsEF(_Uh_W?HWrFVym_ypPc)(~6j z_P(e^I#>@%z*>i@iFlntodmm84%~MM;CFZn6_o_dZx5HKo+XDz%|G~ic*j`fX~4l2 zpLCC}dO?gGB{l;V%>}JOdJFLYw zbV0>07JZK=!Kd8TsfE{cv#=IZ7&i5o{KCJ!<3JaLZLnc;Fbu=IQg?m~O3pu<=_IQ{ z?MChG6mN4bFMdb(lG}5#<=*0!#}p4MR}Y>ik)wc-Dk(&fIajQBp}lu zr2TxyyK&Y#XRYx|rr7eWVFpM69#f^?}MZ7PmPTW4tH1h$yG86U^CJW z#gp{Th4|NS50&Nk3f;FHs3y(y7R2)vI*B1ikAQj!lbK(QuH+BGP$=2BVH>-AWYy-m zOL9N#ULiekgD;z`dZcd`Htmwa38=a6ZatJM8{sr5H`0iuQl@VX)3?H9r%pf+RwP0C zPe1$V{l#8`W_NpWi4oSQB$iyfZ-)@U)`AVzn7kE4lK>3IExrwvae5YZlCMMuxpWxu zcnm|{PlGd6B ztdMQ*AB$@R9bPMdb+3)~hgF%-M11Oa;ejje%2$KkYmJ(hO*HtedKc#Qy(Rxw8P9+3 zLZVF$(uDBGE^xH`Cd^3 z0-D3dm1g?A4_T_Q@X1*yeiS$PgWqaYY!qs4&(O&?p}v=Wh}D2xlWL+-!65TP*`iTZ zM&m1Y0I9-P)GK!Lvg|fAmFwKgxhGzaRsh^M+QvExRmoIXKM)eU#>JKx`Mt{EUC2&W zWJ%?xR+1o0L1qS}D9_d8(JCto7us7nvNwtW2HO`lbRguX@=G94gU@pKhS2Kpr!eNc z^)SLQ{SB`U+;zsqBel7mWro_H@yiCgxm+U)uah#hLxLHYR3WBR%#qsbVH`tA!kZyv zC0)>2#2rIQBC6mF;2AqG!t<3AOW;wm>&ZSFW5Zh35QPJW`y%*|S4yjX=42J-|C_<4tslIxDMASqH0%93!Zd(rY9Mh_s`Z zF}`9&6Zb1{wZ5giq2QrN4i)UZK>;FX@vbp4&~i`4SLGZXthwW%Tpd6woC<%5Gdc6y zAx|cuaVkmIG@%OG@cOq|eoiqNB?FA-d~q%g3RGGhiCexXb*?7>8(YG);@y>cT#_F`q+nN}%uu2lwpND(l9U?}N@VEsRnK4YxkR?+i-Sn-_qxx3Ul z%(Ht^{R1D;b8|OJ3gB6Twh$!QsAbsi>UlHXf}wog6`w7>l{zP3rxr`$!8@;baK~Sy z2!DH@xkoagaa2Dsb-5*I7L;C63;Bb#pi_(5U!NjJ>UPEqxL-y7OCgzD{o@*4`Nz5D zvQ$UnC@s-{(q=BlIgXOF5)-&<*lnW1!K!8Nr531491I!@yeC=swcfVYy zs)x{`uK)Jx{?QAjz?z|Rva;Rq3aW8DH6$T9V9gG3qlI^K`yj}Tg?r4=h;W$cfc!@U z*fjLA&d>3}0TbaMX_9^bB}&SJP7Fl4$6dwqFF=6WL31T{`oFD0{ZBbHY9%<94fK~vHZlGFMuDJ$&Y+%}00O=7Nv3J(0*c{6wGoT9B^vuqz~Jlt6) znCf+rn>0E-JtSevcqnVwY2-sENfG;xzPLHjC(Dbe+U3eeE=G_1LI*)4a>0?BdOww}C*yAUi$VbqIr#mfjTFh1Pz2p7AVTFPrC5<2jMIa_sA4i}%|A z;U&-+JKWWps)D9i5^giQ-390g^JBjUwm_EXzUT4Bips1}toyJBWk(!{#WQ(J8{O3T zttXT1$vZc}oJ%rs_yD`pzVws5n)MvzX8n`Dbwwk1Q*|=?FA#*je9(9K=df{h#~kv~ z3YJqd*QPA%?8TxK-XZw6cB1a64ai@baE-|OUVyPqw8c2a^$%9TCHcd$acROMz31}H z|M`Vb_zR#%ldpbq1pt)pSFiU#NxSvY10i$13OAFP+;uAeOHBu!%&jZf;}>SGl#l@1LnCGr}hnQ+a&klKD*QKigsvX5_$CxvVRZDHc{(UzBFOt z%e0=0u}-(gI7#yFVM$ni*cL2J#8;{mh+#`etG2yXLfss|rj@RL4Sf!EZPIIBSE=Uz zTY9q)B_*ijyi`lsh*Am`0+6mZXA=BYcPzv{)(*u%np3lCNj-0mX5@+OdP!ua{I6ei z|M3D~II);+Orf9G?wMF8zOX@)b&=h${X7nphm$H!ipp1k|H@^r6}3X^BWS!AX#U-) zVDRFeY+v;!qGPw?`xQ9-g_YrJEHEql7qe6&)tJPK-1?d!R)hAhNkZH?TJAA)FtnSo zF?5)@Vl|l1k>i!9ml_IZCW!~U9b-D<lm`6Gv`b>^sEt4m7NOgZ_mP=ryH-2N6LoxjNcQD&ZFy6jm)L>#XrC1PO96~71JFj z?v|Q^M8obYsT^K@aZ=tN6{|})xTA7Rw?=w|b;$#IINC&Xt%BEyFX!fP&WGzz1W3E5 z0wl{DfMRlHSu$a7%fM4OxhE(>2L}dH zQJ6ZYJW*+api_fNs$7S9_(UNX_?S2|ulH3PtG;7hG;S5(ocQtqh=)rz*pZ8GA7+7< zcmO0&PH}PLt)@ki4E)9HJNfsSz->06Z~Hx>ZNROLNED2W2JXaL3CE8$syH$JKt)X* z$YdzVl%N!lJ_|X7wD(R^O=m1Esna!aq(mS>D{v^(c zlnnx5R`re z0rE+dwHl?xl1%|r>EoFCaEg_V?nv6Fp6apfE@;K#cU2|$2wZ%&Yr&LZgZq+DPy-<# z3#!c;HMTuFL{-4pcAjY(_CouZVx<;aN;v|*NiG1knNecdiEN>cea1`71xQYebxxwy zK5JY4zs+a<%AO%)wokH+l3)E!avCjb5l291Y4^%q6we|<>eKC6jo`rrvAe_^=O(`Y znBxQyQvV0MM@^8i{;$c90G%``U{6wgB-C0o7+x?Pj&=w@*rx%dozjD7ZyTvliY4lK zF%`0`hQD)$g zFtmM=Iou~P{rNuqKVt1U?`NG!^AlEoHP;hyQlAj+OPWZ93Mt6-B8<3JD^Y} zt5g1sJ{Q+U)+?2-WzFA!1wJ(sOeLNPzLC+W!CEA2KiSY1Q4WJOrXHQR3PGj5=Lp@I zk+SG5vq~rbF(mf&Nf@PA0qh>03)D-$y+JS=_=C6wgug5(Co2FGjFzYU@{{cTm`i+`ygwT0|Fyd9!=z z42bIP+P}tCYG3cWx z`XOhPYA7m#4?})itzJ(&FhG0a!=FzCMB3nOz_zOd1n!-!)@@CoS}digF{gk&VGVZd zj9Xa0pde^^+5qKul*9Yz)VRWA8DAjc0A-_qFhDlgBaTi{eu~F#E#JA$vH?KR2G>WF zEBP&c`U#H5Q6FvYbzuVwt>am<3cciLG;Ag*n*l7D9z)A-qKzCo0oz2M%y!$Y*b05H z2&UvUVWU?I82wxTnC#C5#M-{~S6U20@~}rJj3QqxD&G@#8^|;F5;_%+Ap%~mOhKG- z8{Ul_ESl+omv@Ff6nni*V$x(h*qAC-CLOXQuczTclJ!Hh%wiwCtLGIe;oW_o@aQ9y z$OXZyZhXKd8h!0nz6n5_sd;KiC(hWq8*f^e254HtJaP zxFyJX!a^sAMxzoHW!Lk{kC?&dU>USOc6gl}7hBx-3-50k5_9-^sKUuPRG+Oh`;Nnr zKo$8b?DVG_FNabiHD_V3b^Z#Sp>=0pH(l68RX~fV`H&Cgem#=?*-g2#**IT*=h;_w zv$irZhmnh%snAxY`U?P;<(?cD|~sRZb~B;@3&A`A|;EK z(0r?J%7*z~t_Hfnpdr_stEGn=O?f|N4xXfdl~SxerNW-H>pr3% z4A2-V7^27?voiy;zU#t;TRcb!!}{&9p4yd+NKFK$gM&eP54Az_ilD?pN1Fp5oj@(F zeK+>t2N$3Hp{fZ=XsNgZJ9nSz^!FP?8u%r^kT1p72iKjI>>qmxI7cb_!(e|3@K5?v zppG%mqY|f{cIUQwO|88P<5YVU%35r45P$@_PJX^p@x+(Pv%P=hZi4CUADZvMxRTUN zhX3d1X99Q3x6`&S0>$orWX%X8-|k}fi{u58Jx3afVJ>X*SXgQ)y%AATs?q#Suw+2+ zv_kpjB9*}QW2ow=U+B8Abt7!m>(f?7YQ{-N2kn^&yA``ZJ6m*ZpULi?=yQF9i=Btn zK~-v6kTRUOk{Yy{T#~`dp+>@#n)_bcL3aNNZ+@qwBEwS#!wDaq8P>ETNdvX5shALB zJXU|ER>vn_h$}|ZjjrVGKTPNG&$Y9^qj~+h_Xx^#J&f%QUcuD7ych02*|jKBl~}(k zLC~6qlJh&1fpftR#HV`?;ylKOO(+K*Ob=ceo*2d+KQ2@eb4B_B62vGXA%fygzTfS+ zh-Bmw^zE<9Yxop%jUYQ=%EjPyJe8Vpni0QTmtP36n{vu2bcEP=Jutp-0 z=xWnw_KfS7+eL6e2eJM^u5A99PpvNyN-xW(&IE(_(pTzNc8GU1J5xR~cC&ZlOS=wT zAoSI(6*KHsOelawUKB|nK}H`K8^sF9GDW||M9K02DfPXF0LYTLxWOVH+y=23)n(G& zef8t{QY~5q_MjQuE>(AYJO?R#g2BFs1l_(Fte?t4wlPz$BMD1kE-x1@aZs*jz;n!# z_A)w@-g`LH_nQ%eKmkx|ZJO@-KwFhvyidxvMK#VsadJzWimlUIe1}@;Zb};l=e~h~ zfsqLM5B|j0J%UM+7~_Qq6J)L-)pf}NUMO&dXO7IpiX~sD+#~5TzgIzL(vMl zS~_1O*%hztZ9#j0iOPk3j%RPr{IaWxl2ZyfC$I<1Ji7Up!H}$4~$jV!V!QJAt+D=QOL8-~GuHYKy=A6&&AC$1pFYUqh4EK!ep+ZfZV$Lj-}%7D{y zRI$zi>4Uccd%*NZ>{*iK!(YfqBZT*oDM^UGn!BLTpO@GwpZSMtUM91M5!g~I^o$V; z4#S8)?xeNyOi7>@=Mx?w{=kvix!@tI)*~U(@RV&1RrI%bQ;0$5gvFrypz!2PJ5aEn z-Fg2AGk?+cPj+jXx~m3dZt$jZkIp;u05`yiJz#h^IdM z#;_ix0QK@atO$xyo;+1z`R?8r%EP0Z>c@pND!C5hH9l=(_wIaXKFAF^u>Nxe-=R1E z<1kLVL@N@OOFGx&F?b?#Nr`>S8%aJ`VYJFSxu#h8ehKmNH zo?gI>FjZpMx*!Y)2}mwiSiLOn6@W?9S{O|=7L!4uy9_w^B9 zO7AXMf%5C)^*;Z^ks%Eq47ZIF@t*N~WqGoorQn)+y8PU2sWI=*hf9-50cC2f{$RqO zQn7i%+cEF`u+HYy4aX_Il$I9?Az?yevqg0xA?PiVIy_cqjuYl#1#)a8pk-jBeY9Vj!-p}d2&>b5Q9?baSKqCNwbkG>{ zolCM2PIvm-z&lA@Syo*UI`KD1%FogFYXZxJdc-?6z3gk&I9ju}t3^+M32-_VKgan7d(ra7886*%S&ZR6wP3dAOh;~7Zi#|3{YD_jy=XSBUoZk;G8o<39mcYD^h zNCum#`{_s4RTZJj4?gohFdJo`Kb>oI`7On&hZQPIoY^v1l^nbjaQyo|>L{)RfNz@* z%K^r(|Ms623PGmN+s3BBj++A%ThMSVr%sG2cUxNgw1xT}zyZSbpt>qQUC(N~;s{}?J@?*aHi?@RA zK70P0*RSvS48p6veE!WEW&7tKRf~!51Wm_mPwTAQ`7TuUmWVh~dYwGFAzyXXVj?e; z{_gP+S*AC}$hOHln@_wKD!m7W<$3lBc{ediNF5y{nXJZcU%50XP$xAJfN}M@k>9nK zk4>4K1rkErX^dpWaEZpts)lbbI~KX0P}Nkl6$$1_b0@WX(L8xFeG7>lYCm~Ic28Wt zIhb_Z*m_?-v9+jrkiHY)vPSP>QpLq*f*j5?V~cGqpH>XJ^Dz$Vl3CmvD)b73wh4TX~gt+&bbn?<%{~lZ{S>l-4&NR1zvgFFau za9Z$LLoq_hyJpyhus-df_t3BlMSVCaznP5@wae{W9{h3-$tJ|9OpkV$ovrL8d^}9O z)(NG=Ct+2h3k0&Ph!T0HxB1hqtu7yhBwAbwe-#*`77~qVs_S%>!sWI|W?p@5p0n2N zS*2ZnR<$-x+m0zYC45rJKibtYx!T?(cKf|LjV27-zg&9|LHg9;`|7c_iZ!S4Ba@Sp6UJ&5>xZ5{m@0K0$+QNs zDz6)v?3SC~`|3jZUfszEfi6e_W^_#+fZcj%HOM_N_tsmWZjCH z&iNJfm%q&v=U!GW{4^e&;- zwS`rumlT)I9>n+=<=Eec)@fxdR=gbaLsu%i3%^mju-(>X3=+d2T-viFHnG5m6ulN0 zJ2?gi2d&wkB)>}MY`1+wS^X5zu(NC@-w#=H^!Pb`-xOnVgG-wi*{yC@LXJ%N%W49uyl1|oh*b z^nRE{1Vt;d{i)J(`Pn?u81pm+_uY)Kb^Y+lnj`)bxB8{yc9QUY3znjH>YZcC8BkGB zD*et^UAXY&R5%Llh~`cGT(K>3-z4PEiYEL(4}pZ~R1wR2<5^YG&mw3<<0AK5O$E*e zm$(;E_f*y9@6`=Z@Z#h|(yaLFo{eY1+k^x9A*<_nb&JyoWKyU{t1bT7TWIVo9w8=u z6jf}TsV?$w06u#Q{cQA-FqCc7o-obhgnJR?PE%{0kBqhzY`Me3l8i5EmE*c@7xpyI zQNq~!ld9&Rl*}lkYE6QN2PdotK^&t_Fi(#9gjv8iiyPh4E9cJp1o6x0MBaB*bIHk2 zeTIKBB#u2q&uYvCjYm5&rt5KPn%&vkUS!+B83HdmdU*YFr#f?URDYp%DI`&64018S zz;QC1xcjp9Y^?pP*n=oeW)4qO{N|fl=csJ58~0QccZlkL|8w@(43T?u&Qzjr(@OX` z7I+tY9S2m(VAxU-npu>=pc5EF#D!GE#vLGZGIVCs?i)9H>0YiyyENH{)tFxeA5d5B zYr5wYRJ3^IQ2V?=JV!S?AB{wf1H~_fw~Ke~{x%>QzjYr^Q0Dy0sO5LwCD;=$hY8rN zdT{hZK6RwCh0FgihY)<%HbJsISE*DwADo=j6*#V-2e}X=c6}Zd|NO7-H|XJBO!{wg zG(QG$Rq8VZ|Hb>JUKZ*pVey4 z`NbdGDTBjLp;Hu{XuP&%2?rhT8XATdM@ZK=v7BTS8Wt-W`a2lS-mRrFp#G0HnuvyD zA-Ok0hd~Ah$qG<*A_Ob6Tv7jL)sw`abDsU^v6zhigX(_Y8o3PC3|S{)_TD$4Uc4rl zgz2u1gkGbqfR*Km&@;+5Z4l3%7%5q882s5c}|AVcZfU zv>dLy5EDj1orqU`x=I z+m`HHo_cRgiugg@F|~kJz>F1za)Q$QI{@*7chP^8+oOW3G|u^J!Etbb%}_v3GXnZ4 zH$*Q|!JrqFFa1+Z5iMMteeZp?*fRJpb^q?~?e+csxd_LB3c7bLWT)#5Hr9a>6(u6m(KYv5>G+V47420Ut6;Qs@`;! zq>kHDOijJSEDJou*0zZy*OBBusAzwX7B`{TV~a8-k59smBby z2&5{OWNNRTSL>~}B<6K@Q)FSU=MFa83;mBT+rrC6Z*T;~Pl^*B9eMhz;2b1^BTB-> zORC3=_{KIN>cxwS5~6`9UH1c zvr54baW9qp(%I`C0iBFkfzvhu)#pOLKyU2QQ0+V$OUN)nPtj}jKgS>=?w1+> literal 0 HcmV?d00001 From 5d358f50dd7d5a53e2381accbce05a1e69d3ca0e Mon Sep 17 00:00:00 2001 From: Callum Date: Tue, 24 Nov 2020 15:37:27 +0100 Subject: [PATCH 4/8] fix image link --- rfc/005-soft-chain-upgrades.md | 4 ++-- .../{chain_migation_1.png => chain-migration.png} | Bin 2 files changed, 2 insertions(+), 2 deletions(-) rename rfc/images/{chain_migation_1.png => chain-migration.png} (100%) diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md index 139dbb72..792c82a7 100644 --- a/rfc/005-soft-chain-upgrades.md +++ b/rfc/005-soft-chain-upgrades.md @@ -196,7 +196,7 @@ The general strategy for a chain migration would be to get the validator set at the height of the migration to not finalize the last block but to instead finalize the entire migration from a prior version to the new version. -![Chain Migration](images/chain_migration_1.png) +![Chain Migration](images/chain-migration.png) Practically speaking, this would mean that `LastBlockID` ( The blue arrow at height h) wouldn't be the hash of the last block but the hash of the newly @@ -215,7 +215,7 @@ would not be able to safely verify the derived blocks because the signatures (the red lines) only link between the original versions (V1). This outlines some basic imperatives with chain migration. -## Imperatives +### Imperatives - All state modifying data from the original block must be preserved - Light client and fast sync verification models must be unaffected diff --git a/rfc/images/chain_migation_1.png b/rfc/images/chain-migration.png similarity index 100% rename from rfc/images/chain_migation_1.png rename to rfc/images/chain-migration.png From e6ca31327ca59965e8f2c38583e66f25b45cab9b Mon Sep 17 00:00:00 2001 From: Callum Waters Date: Wed, 25 Nov 2020 16:42:36 +0100 Subject: [PATCH 5/8] minor word fixes --- rfc/005-soft-chain-upgrades.md | 74 +++++++++++++++++----------------- 1 file changed, 36 insertions(+), 38 deletions(-) diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md index 792c82a7..81df3ecc 100644 --- a/rfc/005-soft-chain-upgrades.md +++ b/rfc/005-soft-chain-upgrades.md @@ -18,22 +18,17 @@ time, where application state can be exported across and an entirely new chain (as far as Tendermint is concerned) is created (see [cosmoshub-3 upgrade](https://blog.cosmos.network/cosmos-hub-3-upgrade-announcement-39c9da941aee)). This means that each block protocol version maps to the entire life of a single -chain. The result of this is that either Tendermint is hindered with the -flexibility of changes it can make or that networks have to deal -with the pain of frequently upgrading and all its shortfalls. - -Furthermore, although work from [RFC002: Non-Zero Genesis](https://github.com/tendermint/spec/blob/master/rfc/002-nonzero-genesis.md) -has enabled unique heights throughout the history of a "logical" chain, data -retrievability across heights remains difficult as blockchains that choose to -upgrade not only have to coordinate upgrades but they essentially lose all their -history. - -The scope of this RFC is to address the need for Tendermint to have flexibility -in improving itself whilst offering greater ease of use to the networks running -on Tendermint. This is done by introducing a division in the way a -network can upgrade: either soft (live) or hard (stop, do some changes then -coordinate a restart). There may still remain the need for hard upgrades but -this will be focused [elsewhere](https://github.com/tendermint/tendermint/issues/5595)). +chain. The result is that networks either forego new features for long periods +of time or have to deal with the pains of upgrading such as the coordination +effort, state migration, intra-chain security challenges and most importantly, +the loss of transaction history. + +The scope of this RFC is to address the need for Tendermint to have better +flexibility in improving itself whilst offering greater ease of use to the +networks running on Tendermint. This is done by introducing a division in the +way a network can upgrade: either soft (live) or hard (stop, do some changes +then coordinate a restart). There may still remain the need for hard upgrades +but this will be focused [elsewhere](https://github.com/tendermint/tendermint/issues/5595)). Instead, this RFC will describe what supporting soft upgrades would entail and explore two different methods: **Multi Data Structure Support** or **Chain Migrations**. @@ -49,27 +44,29 @@ The implementation to support this would ideally need to be done prior to the Tendermint currently has three protocol versions: Block, P2P and App. -Block protocol version would initially be the sole focus for soft upgrades. +The block protocol version would initially be the sole focus for soft upgrades. This consists of all block related data structures i.e. `Header`, `Vote`, `Validator`, `Commit` (a full list of the exact data structures can be found in [data structures](https://github.com/tendermint/spec/blob/master/spec/core/data_structures.md#data-structures). Data structures are dictated by the spec thus requiring a Tendermint software -version to indicate the block version/s it supports. +version to indicate the block version/s it supports. When a new spec release +makes changes to any of the aforementioned data structures, this will result in +incrementing the block protocol version. The app version can already be changed via consensus and is left to the discretion of the application to handle. The P2P version would also be left as -it is, with major revisions requiring a hard upgrade. -However, this may be revisited in the future. +it is, with major revisions requiring a hard upgrade. However, this may be +revisited in the future. ### Transitions Networks would be expected to announce upgrades in advance allowing a window for node operators to upgrade to a software version of Tendermint that would -comply with the announced block protocol upgrade. This can be done asynchronously -with node operators temporarily halting their node, changing to the new binary -and starting up again. Tendermint could also offer a tool to easily swap between -binaries thus reducing the amount of down-time that nodes experience although -this is outside the scope of this RFC. +comply with the announced block protocol upgrade. This can be done +asynchronously with node operators temporarily halting their node, changing to +the new binary and starting up again. Tendermint could also offer a tool to +easily swap between binaries thus reducing the amount of down-time that nodes +experience although this is outside the scope of this RFC. The actual switch would occur in the same manner as an App Version change, using the `EndBlockResponse{}` ABCI call to indicate to Tendermint that a different @@ -190,11 +187,12 @@ Chain migration might sound counter intuitive in the context of an immutable blockchain; any change to a block would simply break the hashes. This would then make the signatures for each block invalid and the whole thing would just fall apart. Chain migration, however, would mean that the network -only supports a single version at the time which would greatly simplify things. +only supports a single version at a time which would greatly simplify things. The general strategy for a chain migration would be to get the validator set at the height of the migration to not finalize the last block but to instead -finalize the entire migration from a prior version to the new version. +finalize the entire migration of all blocks from a prior version to the new +version. ![Chain Migration](images/chain-migration.png) @@ -225,15 +223,15 @@ some basic imperatives with chain migration. One basic approach is that each derived version should be capable of producing its original i.e. If the chain is at version 4 and a node is fast syncing and is at block 100 which corresponds originally to version 2 then when it receives a -derived block it should be able to extract out the original for verification -purposes. However, the derived block should also reflect the structure of the -latest version. +derived block (at v4) it should be able to extract out the original (v2) for +verification. However, the derived block should also reflect the structure of +the latest version. The shortcomings of this is that we would expect the derived block stored to be larger than normal as it needs to hold the relevant data for both. This would go against a lot of the proposed plans on the horizon which would aim to reduce the -overall size of the block. Verification would take a little longer but we -wouldn't need to worry about any byzantine behavior (either the derived +overall size of the block's components. Verification would take a little longer +but we wouldn't need to worry about any byzantine behavior (either the derived block can produce the original block or it can't). ### Advanced Approach @@ -257,7 +255,7 @@ func (vals *ValidatorSet) VerifyCommit(chainID string, blockID BlockID, height int64, commit *Commit) error ``` -We can then trust the header and as aforementioned trust the rest of the block. +We can then trust the header and therefore trust the rest of the block. Then Tendermint delivers the txs to the application which will in turn update the validator set (with an extra height delay). We use NextValidatorsHash to verify that we have the correct trusted validator set for the next height. @@ -301,21 +299,21 @@ Verification would be as follows: 1. Starting from a trusted validator set (this could be from genesis) 2. Grab the migrated block at the next height or more specifically the -signatures, next validator hash and derived block hash +signatures, next validator hash, app hash and derived block hash 3. Calculate the original header hash that the signatures should be signing with -the next validator hash and the derived block hash. +the next validator hash, app hash and the derived block hash. 4. Check that the LastBlockID is equal to the hash of the trusted header (no need to check this is height is 1) 5. Verify that 2/3 signed the original header hash by running `VerifyCommit`. If no error then we can trust at least that the original block ID is correct and -thus based on collision probability of hashed also assume that we can trust the -`NextValidatorHash` and the `AppHash`. +thus, based on the collision probability of hashes, also assume that we can +trust the `NextValidatorHash` and the `AppHash`. 6. Apply block to current state to get the new state. 7. Check that the state's `ValidatorHash` (at height + 1 now) matches the `NextValidatorsHash`. If so it means we can trust the new validator set. Check that the `AppHash` matches. This means that application state is also correct. If one of these doesn't match we drop the block and peer and continue looking -for another. (Remmeber light clients only have `NextValidatorHash` and have no +for another. (Remember light clients only have `NextValidatorHash` and have no record of app state) 8. Go back to 1 and recur until we reach a point where the `DerivedBlockHash` is nil. This indicates the crossover from the migrated blocks to the original From be66d67e15d67bbfd440fae19901d5ce2063f2cd Mon Sep 17 00:00:00 2001 From: Callum Waters Date: Fri, 27 Nov 2020 16:04:33 +0100 Subject: [PATCH 6/8] create appendix to clarify some terminology --- rfc/005-soft-chain-upgrades.md | 63 ++++++++++++++++++++++++++++++++-- 1 file changed, 61 insertions(+), 2 deletions(-) diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md index 81df3ecc..f81210b0 100644 --- a/rfc/005-soft-chain-upgrades.md +++ b/rfc/005-soft-chain-upgrades.md @@ -3,6 +3,7 @@ ## Changelog +- 2020-11-27: Add appendix to distinguish between soft and hard upgrades - 2020-11-23: Added chain migration proposal for comparison - 2020-11-16: Initial draft. @@ -27,8 +28,9 @@ The scope of this RFC is to address the need for Tendermint to have better flexibility in improving itself whilst offering greater ease of use to the networks running on Tendermint. This is done by introducing a division in the way a network can upgrade: either soft (live) or hard (stop, do some changes -then coordinate a restart). There may still remain the need for hard upgrades -but this will be focused [elsewhere](https://github.com/tendermint/tendermint/issues/5595)). +then coordinate a restart). More information on the distinction as well as +general upgrade terminology can be found in Appendix A. There may still remain +the need for hard upgrades but this will be focused [elsewhere](https://github.com/tendermint/tendermint/issues/5595)). Instead, this RFC will describe what supporting soft upgrades would entail and explore two different methods: **Multi Data Structure Support** or **Chain Migrations**. @@ -358,6 +360,8 @@ prior block version. This would allow external clients on prior block versions to still be able to process the blocks. - Async upgrading. Nodes choose when they want to upgrade the binary and Tendermint handles the actual transition. +- In the future it may be possible to support application migrations. This is +where the application can upgrade the Tx data structures. **Negative** @@ -377,3 +381,58 @@ Proposed - [Upgrade tooling tracking issue](https://github.com/tendermint/tendermint/issues/5595) - [Protocol versions](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-016-protocol-versions.md) + +## Appendix A: Upgrade terminology + +It's important to define upgrades and classify the distinctions so as to ensure +that everyone has the same understanding of what it means and that we can use +the same terminology to speak about the different concepts. This appendix aims +at achieving this. + +An upgrade, in the loosest sense, is a change to the Tendermint codebase. For +simplicity to users, we group changes together and form releases. Every release +corresponds to a unique software version. Tendermint follows SemVer which +has strict rules about versioning and makes a distinction between different +types of releases. + +From the node operators perspective, upgrades just mean, stopping the node, +changing the binary and starting the node again. + +A patch release (the last number in x.x.x), is a backwards compatible bug-fix. +Nodes should be able to perform this upgrade at any time without any affect to +the network. + +A minor release (the middle number in x.x.x), is also backwards compatible but +it indicates changes in existing features or new features. One can think of +performance optimizations as a good example. Nodes should also be able to +perform this upgrade at any time and nodes with different minor versions should +have no problem interacting with one another. + +We call the above two releases as constituting minor upgrades. + +A major release (the first number in x.x.x), is a set of incompatible changes. +This means that the public api has changed and/or the messages that nodes send +to one another and persist to disk have changed in a way that is incompatible +with prior releases. A block protocol change is always a major release. + +When this happens nodes can't simply stop and restart when they want but must +coordinate a restart together. There can't be nodes with different versions on +the same network. Not only this, but upgraded nodes can't read nor verify data +structures from a prior major version. Thus, an upgrade for a major release +has so far meant having to create a new chain. + +Soft and hard upgrades both refer to major releases but have very different +properties. + +A soft upgrade like minor upgrades can happen asynchronously across the +network. They are initiated by the application and coordinated by Tendermint's +consensus mechanism. In one way or another they allow the chain to remain fully +accessible and verifiable to the nodes with the latest version. In the context +of this RFC, an example of a soft upgrade could be the removing of timestamps +in the commit sig. + +A hard upgrade must be initiated and coordinated socially. The latest versioned +node is incapable of parsing and/or verifying the previous versioned data +structures. Once could imagine a wall between the chain of blocks. An example of +a hard upgrade in the context of this RFC could be switching from delayed +execution to immediate execution. From 717f01de8afb459efa21b184f6349018524cd9d9 Mon Sep 17 00:00:00 2001 From: Callum Waters Date: Tue, 8 Dec 2020 10:47:10 +0100 Subject: [PATCH 7/8] implement suggestions --- rfc/005-soft-chain-upgrades.md | 65 +++++++++++++++++++++++++--------- 1 file changed, 48 insertions(+), 17 deletions(-) diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md index f81210b0..46fe63a2 100644 --- a/rfc/005-soft-chain-upgrades.md +++ b/rfc/005-soft-chain-upgrades.md @@ -27,21 +27,42 @@ the loss of transaction history. The scope of this RFC is to address the need for Tendermint to have better flexibility in improving itself whilst offering greater ease of use to the networks running on Tendermint. This is done by introducing a division in the -way a network can upgrade: either soft (live) or hard (stop, do some changes -then coordinate a restart). More information on the distinction as well as -general upgrade terminology can be found in Appendix A. There may still remain -the need for hard upgrades but this will be focused [elsewhere](https://github.com/tendermint/tendermint/issues/5595)). -Instead, this RFC will describe what supporting soft upgrades would entail -and explore two different methods: **Multi Data Structure Support** or +way a network can upgrade: either soft or hard. More information on the +distinction as well as general upgrade terminology can be found in Appendix A. +There may still remain the need for hard upgrades but this will be focused +[elsewhere](https://github.com/tendermint/tendermint/issues/5595)). Instead, +this RFC will describe what supporting soft upgrades would entail and explore +two different methods: **Multi Data Structure Support** or **Chain Migrations**. ## Proposal Tendermint supports a class of upgrades deemed as soft upgrades. +A soft upgrade, in the simplest terms, is a change to Tendermint's binary that +would not impede with the use of prior data / existing chains. + The implementation to support this would ideally need to be done prior to the 1.0 release. +### Stakeholders + +Let's first set out to define the different needs that the various stakeholders +of the Tendermint software have in the context of upgrading: + +- Application Developers: want to be able to get as much of the benefits of + new features (like state sync and light client) with minimal work or loss to + their current infrastructure. Ideally, Tendermint is also somewhat + accommodating to application upgrades. + +- Node Operators: Primarily they want safety and reliability. This means minimal + down-time and intuitive / easy UX when it comes to upgrading. + +- Wallets, Block Explorers and other clients: data retrievability which means + ensuring that helpful/informative data is always available and that minimal + infrastructure is needed to support serving data across the entire lifespan of + the chain. + ### Protocol Versioning Tendermint currently has three protocol versions: Block, P2P and App. @@ -89,11 +110,11 @@ We will now cover the two main methods of executing a soft upgrade. ## Method 1: Multi Data Structure Support -This would require Tendermint to support all prior block protocol version. This -would be done by somehow finding what the version of the structure is and -processing it accordingly. This mimics a similar design model to the Cosmos SDK -where all legacy code is stored in the repository and conditionals are used to -indicate how messages are processed. +This would require Tendermint to support all prior block protocol version since +the last hard fork. This would be done by somehow finding what the version of +the structure is and processing it accordingly. This mimics a similar design +model to the Cosmos SDK where all legacy code is stored in the repository and +conditionals are used to indicate how messages are processed. One could imagine a directory structure for each module as such: @@ -154,15 +175,22 @@ Supporting soft upgrades means that the RPC will need to be able to deliver information across different versions. The RPC will therefore need to know which heights corresponds to which version and then be able to communicate this to the clients. This would most likely require clients to be versatile to handling -the different block versions. For example block explorers would also take on the -burden of supporting all block versions across a chain. +the different block versions unless it was possible for the data to be +transformed to an earlier version. For example, block explorers would also take +on the burden of supporting all block versions across a chain. Clients would +indicate the latest version they support in the RPC call itself e.g. +`/v3/block?height=100`. The node would know what version the block at height 100 +was. If it was greater than v3 it would return a version error indicating the +version the block explorer would need to support. If it was less or equal to v3 +then it would return the block (which contains version information). ### Light Clients Similar to RPC, light clients will also need to be wary of block protocol changes and be able to handle them accordingly. Depending on these changes, there might be the need for special verification logic across signed headers -with different versions. +with different versions. If a light client bisected to a height where it didn't +have the version it would return an error. ### Implications @@ -175,12 +203,13 @@ Tendermint handles the actual transition. **Negative:** -- Increased burden on RPC clients to be able to parse multiple versions +- Increased burden on RPC clients to be able to parse multiple versions. - Bloating of code base and binary. We would also need to figure out methods of maintaining good code hygiene with respect to having all these versions. - If we separate versions i.e. ABCI and Block and P2P this could potentially compound the problem if we need to consider the different permutations and -how they can be supported. +how they can be supported. Correctness analysis and proper testing would be +required to mitigate errors. ## Method 2: Chain Migration @@ -361,7 +390,8 @@ to still be able to process the blocks. - Async upgrading. Nodes choose when they want to upgrade the binary and Tendermint handles the actual transition. - In the future it may be possible to support application migrations. This is -where the application can upgrade the Tx data structures. +where the application can upgrade the Tx data structures. Generally, this option +might offer a broader set of changes that can be soft upgraded. **Negative** @@ -372,6 +402,7 @@ the transition period - Bootstrapping nodes and light clients require an extra step for verification. - We've cornered off some untouchable fields in the block structure. If we really wanted to change these we would need a hard upgrade. +- There is greater risk to correctness in the migration logic. ## Status From 7885fa6a9e08daab1b1d8bda1d8d0916ee10e312 Mon Sep 17 00:00:00 2001 From: Callum Waters Date: Tue, 8 Dec 2020 11:01:11 +0100 Subject: [PATCH 8/8] add marko's note on using a separate repo --- rfc/005-soft-chain-upgrades.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/rfc/005-soft-chain-upgrades.md b/rfc/005-soft-chain-upgrades.md index 46fe63a2..ea6046fb 100644 --- a/rfc/005-soft-chain-upgrades.md +++ b/rfc/005-soft-chain-upgrades.md @@ -159,6 +159,9 @@ could have many versions and is represented here as an interface. `State` and internalized to allow for changes (so long as this wouldn't change the overall behavior). +Another similar approach is to have the prior versions in a different repo and +import it as a library. This may minimize the amount of code in the repo. + ### Current Use of Versioning Records of versions are currently stored in `State` and in the `Header` of each @@ -412,6 +415,7 @@ Proposed - [Upgrade tooling tracking issue](https://github.com/tendermint/tendermint/issues/5595) - [Protocol versions](https://github.com/tendermint/tendermint/blob/master/docs/architecture/adr-016-protocol-versions.md) +- [Go Modules: v2 and Beyond](https://blog.golang.org/v2-go-modules) ## Appendix A: Upgrade terminology