diff --git a/404.html b/404.html index 5ff41ed..770ff00 100644 --- a/404.html +++ b/404.html @@ -91,7 +91,7 @@ diff --git a/approved/0001-agile-coretime.html b/approved/0001-agile-coretime.html index a017cbe..02ab8d3 100644 --- a/approved/0001-agile-coretime.html +++ b/approved/0001-agile-coretime.html @@ -90,7 +90,7 @@ diff --git a/approved/0005-coretime-interface.html b/approved/0005-coretime-interface.html index 4a6c725..b7304be 100644 --- a/approved/0005-coretime-interface.html +++ b/approved/0005-coretime-interface.html @@ -90,7 +90,7 @@ diff --git a/approved/0007-system-collator-selection.html b/approved/0007-system-collator-selection.html index 3da0523..c724ba3 100644 --- a/approved/0007-system-collator-selection.html +++ b/approved/0007-system-collator-selection.html @@ -90,7 +90,7 @@ diff --git a/approved/0008-parachain-bootnodes-dht.html b/approved/0008-parachain-bootnodes-dht.html index 29ebb49..82b606b 100644 --- a/approved/0008-parachain-bootnodes-dht.html +++ b/approved/0008-parachain-bootnodes-dht.html @@ -90,7 +90,7 @@ diff --git a/approved/0010-burn-coretime-revenue.html b/approved/0010-burn-coretime-revenue.html index 67c84e6..5e17dbe 100644 --- a/approved/0010-burn-coretime-revenue.html +++ b/approved/0010-burn-coretime-revenue.html @@ -90,7 +90,7 @@ diff --git a/approved/0012-process-for-adding-new-collectives.html b/approved/0012-process-for-adding-new-collectives.html index c603544..8cb6114 100644 --- a/approved/0012-process-for-adding-new-collectives.html +++ b/approved/0012-process-for-adding-new-collectives.html @@ -90,7 +90,7 @@ diff --git a/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html b/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html index f49e5a7..0db771e 100644 --- a/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html +++ b/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html @@ -90,7 +90,7 @@ diff --git a/approved/0014-improve-locking-mechanism-for-parachains.html b/approved/0014-improve-locking-mechanism-for-parachains.html index 5ee782f..e9833d1 100644 --- a/approved/0014-improve-locking-mechanism-for-parachains.html +++ b/approved/0014-improve-locking-mechanism-for-parachains.html @@ -90,7 +90,7 @@ diff --git a/approved/0022-adopt-encointer-runtime.html b/approved/0022-adopt-encointer-runtime.html index cdd3d67..7e9d1b2 100644 --- a/approved/0022-adopt-encointer-runtime.html +++ b/approved/0022-adopt-encointer-runtime.html @@ -90,7 +90,7 @@ diff --git a/approved/0026-sassafras-consensus.html b/approved/0026-sassafras-consensus.html index a895ccd..7456cb6 100644 --- a/approved/0026-sassafras-consensus.html +++ b/approved/0026-sassafras-consensus.html @@ -90,7 +90,7 @@ diff --git a/approved/0032-minimal-relay.html b/approved/0032-minimal-relay.html index abd7e4a..2c3f789 100644 --- a/approved/0032-minimal-relay.html +++ b/approved/0032-minimal-relay.html @@ -90,7 +90,7 @@ diff --git a/approved/0042-extrinsics-state-version.html b/approved/0042-extrinsics-state-version.html index c55674c..050b922 100644 --- a/approved/0042-extrinsics-state-version.html +++ b/approved/0042-extrinsics-state-version.html @@ -90,7 +90,7 @@ diff --git a/approved/0043-storage-proof-size-hostfunction.html b/approved/0043-storage-proof-size-hostfunction.html index c71dcc6..0edd935 100644 --- a/approved/0043-storage-proof-size-hostfunction.html +++ b/approved/0043-storage-proof-size-hostfunction.html @@ -90,7 +90,7 @@ diff --git a/approved/0045-nft-deposits-asset-hub.html b/approved/0045-nft-deposits-asset-hub.html index 2a4e720..e9cd642 100644 --- a/approved/0045-nft-deposits-asset-hub.html +++ b/approved/0045-nft-deposits-asset-hub.html @@ -90,7 +90,7 @@ diff --git a/approved/0047-assignment-of-availability-chunks.html b/approved/0047-assignment-of-availability-chunks.html index 53d12aa..633541b 100644 --- a/approved/0047-assignment-of-availability-chunks.html +++ b/approved/0047-assignment-of-availability-chunks.html @@ -90,7 +90,7 @@ diff --git a/approved/0048-session-keys-runtime-api.html b/approved/0048-session-keys-runtime-api.html index 184c44a..1ce20bb 100644 --- a/approved/0048-session-keys-runtime-api.html +++ b/approved/0048-session-keys-runtime-api.html @@ -90,7 +90,7 @@ diff --git a/approved/0050-fellowship-salaries.html b/approved/0050-fellowship-salaries.html index 6925f9a..792d020 100644 --- a/approved/0050-fellowship-salaries.html +++ b/approved/0050-fellowship-salaries.html @@ -90,7 +90,7 @@ diff --git a/approved/0056-one-transaction-per-notification.html b/approved/0056-one-transaction-per-notification.html index 0901c54..d6f102c 100644 --- a/approved/0056-one-transaction-per-notification.html +++ b/approved/0056-one-transaction-per-notification.html @@ -90,7 +90,7 @@ diff --git a/approved/0059-nodes-capabilities-discovery.html b/approved/0059-nodes-capabilities-discovery.html index 70e811f..224a22f 100644 --- a/approved/0059-nodes-capabilities-discovery.html +++ b/approved/0059-nodes-capabilities-discovery.html @@ -90,7 +90,7 @@ diff --git a/approved/0078-merkleized-metadata.html b/approved/0078-merkleized-metadata.html index 9560d32..14d9b3b 100644 --- a/approved/0078-merkleized-metadata.html +++ b/approved/0078-merkleized-metadata.html @@ -90,7 +90,7 @@ diff --git a/approved/0084-general-transaction-extrinsic-format.html b/approved/0084-general-transaction-extrinsic-format.html index 2650112..bd6b738 100644 --- a/approved/0084-general-transaction-extrinsic-format.html +++ b/approved/0084-general-transaction-extrinsic-format.html @@ -90,7 +90,7 @@ diff --git a/approved/0091-dht-record-creation-time.html b/approved/0091-dht-record-creation-time.html index 7f59b91..d52b098 100644 --- a/approved/0091-dht-record-creation-time.html +++ b/approved/0091-dht-record-creation-time.html @@ -90,7 +90,7 @@ diff --git a/approved/0097-unbonding_queue.html b/approved/0097-unbonding_queue.html index f491f14..0c7f490 100644 --- a/approved/0097-unbonding_queue.html +++ b/approved/0097-unbonding_queue.html @@ -90,7 +90,7 @@ diff --git a/approved/0099-transaction-extension-version.html b/approved/0099-transaction-extension-version.html index 886ac8e..22fce81 100644 --- a/approved/0099-transaction-extension-version.html +++ b/approved/0099-transaction-extension-version.html @@ -90,7 +90,7 @@ diff --git a/approved/0100-xcm-multi-type-asset-transfer.html b/approved/0100-xcm-multi-type-asset-transfer.html index 7a8dd8e..5109567 100644 --- a/approved/0100-xcm-multi-type-asset-transfer.html +++ b/approved/0100-xcm-multi-type-asset-transfer.html @@ -90,7 +90,7 @@ diff --git a/approved/0101-xcm-transact-remove-max-weight-param.html b/approved/0101-xcm-transact-remove-max-weight-param.html index 4f9cb01..c8ea0af 100644 --- a/approved/0101-xcm-transact-remove-max-weight-param.html +++ b/approved/0101-xcm-transact-remove-max-weight-param.html @@ -90,7 +90,7 @@ diff --git a/approved/0103-introduce-core-index-commitment.html b/approved/0103-introduce-core-index-commitment.html index 271bc39..9ea9090 100644 --- a/approved/0103-introduce-core-index-commitment.html +++ b/approved/0103-introduce-core-index-commitment.html @@ -90,7 +90,7 @@ diff --git a/approved/0105-xcm-improved-fee-mechanism.html b/approved/0105-xcm-improved-fee-mechanism.html index 62f0c69..ca2ceae 100644 --- a/approved/0105-xcm-improved-fee-mechanism.html +++ b/approved/0105-xcm-improved-fee-mechanism.html @@ -90,7 +90,7 @@ diff --git a/approved/0107-xcm-execution-hints.html b/approved/0107-xcm-execution-hints.html index fcf41d7..dd2e4fa 100644 --- a/approved/0107-xcm-execution-hints.html +++ b/approved/0107-xcm-execution-hints.html @@ -90,7 +90,7 @@ diff --git a/approved/0108-xcm-remove-testnet-ids.html b/approved/0108-xcm-remove-testnet-ids.html index 08837cc..ecaf30c 100644 --- a/approved/0108-xcm-remove-testnet-ids.html +++ b/approved/0108-xcm-remove-testnet-ids.html @@ -90,7 +90,7 @@ diff --git a/approved/0122-alias-origin-on-asset-transfers.html b/approved/0122-alias-origin-on-asset-transfers.html index 2e21fda..ac6d661 100644 --- a/approved/0122-alias-origin-on-asset-transfers.html +++ b/approved/0122-alias-origin-on-asset-transfers.html @@ -90,7 +90,7 @@ diff --git a/index.html b/index.html index 365e635..0face9d 100644 --- a/index.html +++ b/index.html @@ -90,7 +90,7 @@ @@ -185,7 +185,7 @@ detailing proposed changes to the technical implementation of the Polkadot netwo
This book contains the Polkadot Fellowship Requests for Comments (RFCs) detailing proposed changes to the technical implementation of the Polkadot network.
- -Table of Contents
- -:pending_code as intermediate storage key for the runtime code| Start Date | 14.10.2024 |
|---|---|
| Description | Store a runtime upgrade in :pending_code before moving it to :code. |
| Authors | Bastian Köcher |
The code of a runtime is stored in its own state, and when performing a runtime upgrade, this code is replaced. The new runtime can contain runtime migrations that adapt the state to the state layout as defined by the runtime code. This runtime migration is executed when building the first block with the new runtime code. Anything that interacts with the runtime state uses the state layout as defined by the runtime code. So, when trying to load something from the state in the block that applied the runtime upgrade, it will use the new state layout but will decode the data from the non-migrated state. In the worst case, the data is incorrectly decoded, which may lead to crashes or halting of the chain.
-This RFC proposes to store the new runtime code under a different storage key when applying a runtime upgrade. This way, all the off-chain logic can still load the old runtime code under the default storage key and decode the state correctly. The block producer is then required to use this new runtime code to build the next block. While building the next block, the runtime is executing the migrations and moves the new runtime code to the default runtime code location. So, the runtime code found under the default location is always the correct one to decode the state from which the runtime code was loaded.
-While the issue of having undecodable state only exists for the one block in which the runtime upgrade was applied, it still impacts anything that reads state data, like block explorers, UIs, nodes, etc. For block explorers, the issue mainly results in indexing invalid data and UIs may show invalid data to the user. For nodes, reading incorrect data may lead to a performance degradation of the network. There are also ways to prevent certain decoding issues from happening, but it requires that developers are aware of this issue and also requires introducing extra code, which could introduce further bugs down the line.
-So, this RFC tries to solve these issues by fixing the underlying problem of having temporary undecodable state.
-The runtime code is stored under the special key :code in the state. Nodes and other tooling read the runtime code under this storage key when they want to interact with the runtime for e.g., building/importing blocks or getting the metadata to read the state. To update the runtime code the runtime overwrites the value at :code, and then from the next block on, the new runtime will be loaded.
-This RFC proposes to first store the new runtime code under :pending_code in the state for one block. When the next block is being built, the block builder first needs to check if :pending_code is set, and if so, it needs to load the runtime from this storage key. While building the block the runtime will move :pending_code to :code to have the runtime code at the default location. Nodes importing the block will also need to load :pending_code if it exists to ensure that the correct runtime code is used. By doing it this way, the runtime code found at :code in the state of a block will always be able to decode the state.
-Furthermore, this RFC proposes to introduce system_version: 3. The system_version was introduced in RFC42. Version 3 would then enable the usage of :pending_code when applying a runtime code upgrade. This way, the feature can be introduced first and enabled later when the majority of the nodes have upgraded.
Because the first block built with the new runtime code will move the runtime code from :pending_code to :code, the runtime code will need to be loaded. This means the runtime code will appear in the proof of validity of a parachain for the first block built with the new runtime code. Generally this is not a problem as the runtime code is also loaded by the parachain when setting the new runtime code.
-There is still the possibility of having state that is not migrated even when following the proposal as presented by this RFC. The issue is that if the amount of data to be migrated is too big, not all of it can be migrated in one block, because either it takes more time than there is assigned for a block or parachains for example have a fixed budget for their proof of validity. To solve this issue there already exist multi-block migrations that can chunk the migration across multiple blocks. Consensus-critical data needs to be migrated in the first block to ensure that block production etc., can continue. For the other data being migrated by multi-block migrations the migrations could for example expose to the outside which keys are being migrated and should not be indexed until the migration is finished.
Testing should be straightforward and most of the existing testing should already be good enough. Extending with some checks that :pending_code is moved to :code.
The performance should not be impacted besides requiring loading the runtime code in the first block being built with the new runtime code.
-It only alters the way blocks are produced and imported after applying a runtime upgrade. This means that only nodes need to be adapted to the changes of this RFC.
-The change will require that the nodes are upgraded before the runtime starts using this feature. Otherwise they will fail to import the block build by :pending_code.
-For Polkadot/Kusama this means that also the parachain nodes need to be running with a relay chain node version that supports this new feature. Otherwise the parachains will stop producing/finalizing nodes as they can not sync the relay chain any more.
The issue initially reported a bug that led to this RFC. It also discusses multiple solutions for the problem.
-None
-move function. When using the V1 trie layout every value bigger than 32 bytes is put into the db separately. This means a low level move function would only need to move the hash of the runtime code from :code to :pending_code.Table of Contents
This RFC proposes the definition of version 5 extrinsics along with changes to the specification and encoding from version 4.
-RFC84 introduced the specification of General transactions, a new type of extrinsic besides the Signed and Unsigned variants available previously in version 4. Additionally, RFC99 introduced versioning of transaction extensions through an extra byte in the extrinsic encoding. Both of these changes require an extrinsic format version bump as both the semantics around extensions as well as the actual encoding of extrinsics need to change to accommodate these new features.
The introduction of General transactions allows the authorization of any and all origins through
extensions. This means that, with the appropriate extension, General transactions are capable of
@@ -363,31 +297,31 @@ consumers downstream in the transition period between these extrinsic versions,
General variant starting with v5 extrinsicsThe metadata will have to accommodate two distinct extrinsic format versions at a given point in time in order to provide the new functionality in a non-breaking way for users and tooling.
-There is no impact on testing, security or privacy.
-This change makes the authorization through signatures configurable by runtime devs in version 5
extrinsics, as opposed to version 4 where the signing payload algorithm and signatures were
hardcoded. This moves the responsibility of ensuring proper authentication through
TransactionExtension to the runtime devs, but a sensible default which closely resembles the
present day behavior will be provided in VerifySignature.
There is no performance impact.
-Tooling will have to adapt to be able to tell which authorization scheme is used by a particular
transaction by decoding the extension and checking which particular TransactionExtension in the pipeline is enabled to do the origin authorization. Previously, this was done by simply checking whether the transaction is signed or unsigned, as there was only one method of authentication.
As long as extrinsic version 4 is still exposed in the metadata when version 5 will be introduced, the changes will not break existing infrastructure. This should give enough time for tooling to support version 5 and to remove version 4 in the future.
-This is a result of the work in Extrinsic Horizon and RFC99.
-There is no clear way to expose two different extrinsic versions in the current metadata framework. A non-exhaustive list of options discussed so far:
ExtrinsicMetadata, but for future versions of the extrinsic
format and add it to the metadata.Following this change, extrinsic version 5 will be introduced as part of the Extrinsic Horizon effort, which will shape future work.
@@ -444,16 +378,16 @@ work.An off-chain approximation protocol should assign rewards based upon the approvals and availability work done by validators.
All validators track which approval votes they actually use, reporting the aggregate, after which an on-chain median computation gives a good approximation under byzantine assumptions. Approval checkers report aggregate information about which availability chunks they use too, but in availability we need a tit-for-tat game to enforce honesty, because approval committees could often bias results thanks to their small size.
-We want all polkadot subsystems be profitable for validataors, because otherwise operators might profit from running modified code. In particular, almost all rewards in Kusama/Polkadot should come from work done securing parachains, primarily approval checking, but also backing, availability, and support of XCMP.
Among these task, our highest priorities must be approval checks, which ensure soundness, and sending availability chunks to approval checkers. We prove backers must be paid strictly less than approval checkers.
At present though, validators' rewards have relatively little relationship to validators operating costs, in terms of bandwidth and CPU time. Worse, polkadot's scaling makes us particular vulnerable "no-shows" caused by validators skipping their approval checks.
We're particularly concernned about hardware specks impact upon the number of parachain cores. We've requested relatively low spec machines so far, only four physical CPU cores, although some run even lower specs like only two physical CPU cores. Alone, rewards cannot fix our low speced validator problem, but rewards and outreach together should far more impact than either alone.
In future, we'll further increase validator spec requirements, which directly improve polkadot's throughput, and which repeats this dynamic of purging underspeced nodes, except outreach becomes more important because de facto too many slow validators can "out vote" the faster ones
-We alter the validators rewards protocol, but with negligable impact upon rewards for honest validators who comply with hardware and bandwidth recommendations.
We shall still reward participation in relay chain concensus of course, which de facto means block production but not finality, but these current reward levels shall wind up greatly reduced. Any validators who manipulate block rewards now could lose rewards here, simply because of rewards being shifted from block production to availability, but this sounds desirable.
We've discussed roughly this rewards protocol in https://hackmd.io/@rgbPIkIdTwSICPuAq67Jbw/S1fHcvXSF and https://github.com/paritytech/polkadot-sdk/issues/1811 as well as related topics like https://github.com/paritytech/polkadot-sdk/issues/5122
@@ -539,7 +473,7 @@ for (mmu,atm) in my_missing_uploads.iter_mut().zip(approvals_tally_messages) {We distribute rewards on-chain using approval_usages_medians and reweighted_total_used_downloads. Approval checkers could later change from who they download chunks using my_missing_uploads.
In theory, validators could adopt whatever strategy they like to penalize validators who stiff them on availability redistribution rewards, except they should not stiff back, only choose other availability providers. We discuss one good strategy below, but initially this could go unimplemented.
-Polkadot's efficency creates subtle liveness concerns: Anytime one node cannot perform one of its approval checks then Polkadot loses in expectation 3.25 approval checks, or 0.10833 parablocks. This makes back pressure essential.
We cannot throttle approval checks securely either, so reactive off-chain back pressure only makes sense during or before the backing phase. In other words, if nodes feel overworked themselves, or perhaps beleive others to be, then they should drop backing checks, never approval checks. It follows backing work must be rewarded less well and less reliably than approvals, as otherwise validators could benefit from behavior that harms the network.
@@ -590,7 +524,7 @@ At this point, we compute $\beta\prime_w = \sum_v \beta\prime_{w,v}$ on-chain foWe discuss approvals being considered by the tit-for-tat in earlier drafts. An adversary who successfuly manipulates the rewards median votes would've alraedy violated polkadot's security assumptions though, which requires a hard fork and correcting the dot allocation. Incorrect report wrong approval_usages remain interesting statistics though.
Adversarial validators could manipulates their availability votes though, even without being a supermajority. If they still download honestly, then this costs them more rewards than they earn. We do not prevent validators from preferentially obtaining their pieces from their friends though. We should analyze, or at least observe, the long-term consequences.
A priori, whale nominator's validators could stiff validators but then rotate their validators quickly enough so that they never suffered being skipped back. We discuss several possible solution, and their difficulties, under "Rob's nominator-wise skipping" in https://hackmd.io/@rgbPIkIdTwSICPuAq67Jbw/S1fHcvXSF but overall less seems like more here. Also frequent validator rotation could be penalized elsewhere.
-We operate off-chain except for final rewards votes and median tallies. We expect lower overhead rewards protocols would lack information, thereby admitting easier cheating.
Initially, we designed the ELVES approval gadget to allow on-chain operation, in part for rewards computation, but doing so looks expensive. Also, on-chain rewards computaiton remains only an approximation too, but could even be biased more easily than our off-chain protocol presented here.
@@ -598,11 +532,11 @@ At this point, we compute $\beta\prime_w = \sum_v \beta\prime_{w,v}$ on-chain foWe alraedy teach validators about missed parachain blocks, but we'll teach approval checking more going forwards, because current efforts focus more upon backing.
JAM's block exports should not complicate availability rewards, but could impact some alternative schemes.
-None
-Provide specific questions to discuss and address before the RFC is voted on by the Fellowship. This should include, for example, alternatives to aspects of the proposed design where the appropriate trade-off to make is unclear.
-Any rewards protocol could simply be "out voted" by too many slow validators: An increase the number of parachain cores increases more workload, but this creates no-shows if too few validators could handle this workload.
We could add a synthetic parachain flag, only settable by governance, which treats no-shows as positive approval votes for that parachain, but without adding rewards. We should never enable this for real parachains, only for synthetic ones like gluttons. We should not enable the synthetic parachain flag long-term even for gluttonsm, because validators could easily modify their code. Yet, synthetic approval checks might enable pushing the hardware upgrades more agressively over the short-term.
@@ -652,12 +586,12 @@ At this point, we compute $\beta\prime_w = \sum_v \beta\prime_{w,v}$ on-chain foChange the upgrade process of a parachain runtime upgrade to become an off-chain process with regards to the relay chain. Upgrades are still contained in parachain blocks, but will no longer need to end up in relay chain blocks nor in relay chain state.
-Having parachain runtime upgrades go through the relay chain has always been seen as a scalability concern. Due to optimizations in statement distribution and asynchronous backing it became less crucial and got @@ -672,13 +606,13 @@ this we would hope for far more parachains to get registered, thousands potentially even ten thousands. With so many PVFs registered, updates are expected to become more frequent and even attacks on service quality for other parachains would become a higher risk.
-The issues with on-chain runtime upgrades are:
The major drawback of this solution is the same as any solution the moves work off-chain, it adds complexity to the node. E.g. nodes needing the PVF, need to store them separately, together with their own pruning strategy as well.
-Implementations adhering to this RFC, will respond to PVF requests with the actual PVF, if they have it. Requesters will persist received PVFs on disk for as long as they are replaced by a new one. Implementations must not be lazy @@ -826,8 +760,8 @@ only chunk, but also PVF available), it is important to have enough validators upgraded, before we allow collators to make use of the new runtime upgrade mechanism. Otherwise we would risk disputes to not being able to succeed.
This RFC has no impact on privacy.
-This proposal lightens the load on the relay chain and is thus in general beneficial for the performance of the network, this is achieved by the following:
@@ -842,7 +776,7 @@ upgrade per relay chain block, occupying almost all of the blockspace. push back on an upgrade for whatever reason, no network bandwidth and core time gets wasted because of this.End users are only affected by better performance and more stable block times.
Parachains will need to implement the introduced request/response protocol and
adapt to the new signalling mechanism via an UMP message, instead of sending
@@ -853,7 +787,7 @@ upgrade gets passed to pre-checking. This is especially important for on-demand
chains or bulk users not occupying a full core. Further more that behaviour of
requiring multiple blocks to fully initiate a runtime upgrade needs to be well
documented.
We will continue to support the old mechanism for code upgrades for a while, but will start to impose stricter limits over time, with the number of registered parachains going up. With those limits in place parachains not migrating to the @@ -874,7 +808,7 @@ validators. "hot" PVF).
Off-chain runtime upgrades have been discussed before, the architecture described here is simpler though as it piggybacks on already existing features, namely:
@@ -883,7 +817,7 @@ namely:https://github.com/paritytech/polkadot-sdk/issues/971
-TODO: Fully resolve these questions and incorporate in RFC text.
-By no longer having code upgrade go through the relay chain, occupying a full relay chain block, the impact on other parachains is already greatly reduced, if we @@ -976,9 +910,9 @@ sharing if multiple parachains use the same data (e.g. same smart contracts).
This RFC proposes a solution to replicate an existing pure proxy from one chain to others. The aim is to address the current limitations where pure proxy accounts, which are keyless, cannot have their proxy relationships recreated on different chains. This leads to issues where funds or permissions transferred to the same keyless account address on chains other than its origin chain become inaccessible.
-A pure proxy is a new account created by a primary account. The primary account is set as a proxy for the pure proxy account, managing it. Pure proxies are keyless and non-reproducible, meaning they lack a private key and have an address derived from a preimage determined by on-chain logic. More on pure proxies can be found here.
For the purpose of this document, we define a keyless account as a "pure account", the controlling account as a "proxy account", and the entire relationship as a "pure proxy".
The relationship between a pure account (e.g., account ID: pure1) and its proxy (e.g., account ID: alice) is stored on-chain (e.g., parachain A) and currently cannot be replicated to another chain (e.g., parachain B). Because the account pure1 is keyless and its proxy relationship with alice is not replicable from the parachain A to the parachain B, alice does not control the pure1 account on the parachain B.
Given that these mistakes are likely, it is necessary to provide a solution to either prevent them or enable access to a pure account on a target chain.
-Runtime Users, Runtime Devs, wallets, cross-chain dApps.
-One possible solution is to allow a proxy to create or replicate a pure proxy relationship for the same pure account on a target chain. For example, Alice, as the proxy of the pure1 pure account on parachain A, should be able to set a proxy for the same pure1 account on parachain B.
To minimise security risks, the parachain B should grant the parachain A the least amount of permission necessary for the replication. First, Parachain A claims to Parachain B that the operation is commanded by the pure account, and thus by its proxy, and second, provides proof that the account is keyless.
The replication process will be facilitated by XCM, with the first claim made using the DescendOrigin instruction. The replication call on parachain A would require a signed origin by the pure account and construct an XCM program for parachain B, where it first descends the origin, resulting in the ParachainA/AccountId32(pure1) origin location on the receiving side.
There are two disadvantages to this approach:
We could eliminate the first disadvantage by allowing only the spawner of the pure proxy to recreate the pure proxies, if they sign the transaction on a remote chain and supply the witness/preimage. Since the preimage of a pure account includes the account ID of the spawner, we can verify that the account signing the transaction is indeed the spawner of the given pure account. However, this approach would grant exclusive rights to the spawner over the pure account, which is not a property of pure proxies at present. This is why it's not an option for us.
As an alternative to requiring clients to provide a witness data, we could label pure accounts on the source chain and trust it on the receiving chain. However, this would require the receiving chain to place greater trust in the source chain. If the source chain is compromised, any type of account on the trusting chain could also be compromised.
A conceptually different solution would be to not implement replication of pure proxies and instead inform users that ownership of a pure proxy on one chain does not imply ownership of the same account on another chain. This solution seems complex, as it would require UIs and clients to adapt to this understanding. Moreover, mistakes would likely remain unavoidable.
-Each chain expressly authorizes another chain to replicate its pure proxies, accepting the inherent risk of that chain potentially being compromised. This authorization allows a malicious actor from the compromised chain to take control of any pure proxy account on the chain that granted the authorization. However, this is limited to pure proxies that originated from the compromised chain if they have a chain-specific seed within the preimage.
There is a security issue, not introduced by the proposed solution but worth mentioning. The same spawner can create the pure accounts on different chains controlled by the different accounts. This is possible because the current preimage version of the proxy pallet does not include any non-reproducible, chain-specific data, and elements like block numbers and extrinsic indexes can be reproduced with some effort. This issue could be addressed by adding a chain-specific seed into the preimages of pure accounts.
-The replication is facilitated by XCM, which adds some additional load to the communication channel. However, since the number of replications is not expected to be large, the impact is minimal.
-The proposed solution does not alter any existing interfaces. It does require clients to obtain the witness data which should not be an issue with support of an indexer.
-None.
-None.
-None.
-This RFC proposes a new host function, secp256r1_ecdsa_verify_prehashed, for verifying NIST-P256 signatures. The function takes as input the message hash, r and s components of the signature, and the x and y coordinates of the public key. By providing this function, runtime authors can leverage a more efficient verification mechanism for "secp256r1" elliptic curve signatures, reducing computational costs and improving overall performance.
“secp256r1” elliptic curve is a standardized curve by NIST which has the same calculations by different input parameters with “secp256k1” elliptic curve. The cost of combined attacks and the security conditions are almost the same for both curves. Adding a host function can provide signature verifications using the “secp256r1” elliptic curve in the runtime and multi-faceted benefits can occur. One important factor is that this curve is widely used and supported in many modern devices such as Apple’s Secure Enclave, Webauthn, Android Keychain which proves the user adoption. Additionally, the introduction of this host function could enable valuable features in the account abstraction which allows more efficient and flexible management of accounts by transaction signs in mobile devices. Most of the modern devices and applications rely on the “secp256r1” elliptic curve. The addition of this host function enables a more efficient verification of device native transaction signing mechanisms. For example:
This RFC proposes a new host function for runtime authors to leverage a more efficient verification mechanism for "secp256r1" elliptic curve signatures.
Proposed host function signature:
#![allow(unused)] @@ -1132,19 +1066,19 @@ Most of the modern devices and applications rely on the “secp256r1” elliptic ) -> bool; }
The host function MUST return true if the signature is valid or false otherwise.
N/A
-The changes are not directly affecting the protocol security, parachains are not enforced to use the host function.
-N/A
-The host function proposed in this RFC allows parachain runtime developers to use a more efficient verification mechanism for "secp256r1" elliptic curve signatures.
-Parachain teams will need to include this host function to upgrade.
-A followup of the RFC-0014. This RFC proposes adding a new collective to the Polkadot Collectives Chain: The Unbrick Collective, as well as improvements in the mechanisms that will allow teams operating paras that had stopped producing blocks to be assisted, in order to restore the production of blocks of these paras.
-Since the initial launch of Polkadot parachains, there has been many incidients causing parachains to stop producing new blocks (therefore, being bricked) and many occurrences that requires Polkadot governance to update the parachain head state/wasm. This can be due to many reasons range @@ -1204,14 +1138,14 @@ damage to the parachain and users.
Polkadot Fellowship), due to the nature of their mission, are not fit to carry these kind of tasks.In consequence, the idea of a Unbrick Collective that can provide assistance to para teams when they brick and further protection against future halts is reasonable enough.
-The Unbrick Collective is defined as an unranked collective of members, not paid by the Polkadot Treasury. Its main goal is to serve as a point of contact and assistance for enacting the actions @@ -1279,31 +1213,31 @@ of the new PVF being set). Therefore, they must have the technical capacity to p
The ability to modify the Head State and/or the PVF of a para means a possibility to perform arbitrary modifications of it (i.e. take control the native parachain token or any bridged assets in the para).
This could introduce a new attack vectorm, and therefore, such great power needs to be handled carefully.
-The implementation of this RFC will be tested on testnets (Rococo and Westend) first.
An audit will be required to ensure the implementation doesn't introduce unwanted side effects.
There are no privacy related concerns.
-This RFC should not introduce any performance impact.
-This RFC should improve the experience for new and existing parachain teams, lowering the barrier to unbrick a stalled para.
-This RFC is fully compatible with existing interfaces.
-WhitelistedUnbrickCaller track?Unbrick origin?In an attempt to mitigate risks derived from unwanted behaviours around long decision periods on referenda, this proposal describes how to finalize and decide a result of a poll via a mechanism similar to candle auctions.
-Referenda protocol provide permissionless and efficient mechanisms to enable governance actors to decide the future of the blockchains around Polkadot network. However, they pose a series of risks derived from the game theory perspective around these mechanisms. One of them being where an actor @@ -1376,7 +1310,7 @@ on a poll as early as possible. This proposal's approach suggests using a Candle be determined right after the confirm period finishes, thus decreasing the chances of actors to alter the results of a poll on confirming state, and instead incentivizing them to cast their votes earlier, on deciding state.
-Finalizing state.Currently, the process of a referendum/poll is defined as an sequence between an ongoing state (where accounts can vote), comprised by a with a preparation period, a decision period, and a confirm period. If the poll is passing before the decision period ends, it's possible to push @@ -1467,43 +1401,43 @@ candle was "lit-off".
cf --> [*] rj --> [*] -This approach doesn't include a mechanism to determine whether a change of the poll status in the confirming period is due to a legitimate change of mind of the voters, or an exploitation of its aforementioned vulnerabilities (like a sniping attack), instead treating all of them as potential attacks.
This is an issue that can be addressed by additional mechanisms, and heuristics that can help determine the probability of a change of poll status to happen as a result of a legitimate behaviour.
-The implementation of this RFC will be tested on testnets (Paseo and Westend) first. Furthermore, it should be enabled in a canary network (like Kusama) to ensure the behaviours it is trying to address is indeed avoided.
An audit will be required to ensure the implementation doesn't introduce unwanted side effects.
There are no privacy related concerns.
-The added steps imply pessimization, necessary to meet the expected changes. An implementation MUST exit from the Finalization period as early as possible to minimize this impact.
-This proposal does not alter the already exposed interfaces or developers or end users. However, they must be aware of the changes in the additional overhead the new period might incur (these depend on the implemented VRF).
-This proposal does not break compatibility with existing interfaces, older versions, but it alters the previous implementation of the referendum processing algorithm.
An acceptable upgrade strategy that can be applied is defining a point in time (block number, poll index) from which to start applying the new mechanism, thus, not affecting the already ongoing referenda.
-polkadot-runtime-commont: Defines the mechanism of candle auctions.A proposed implementation of this change can be seen on this Pull Request.
Table of Contents
@@ -1536,12 +1470,12 @@ organic behaviour, and not an unwanted, malicious behaviour?The protocol change introduces flexibility in the governance structure by enabling the referenda
track list to be modified dynamically at runtime. This is achieved by replacing static slices in
TracksInfo with iterators, facilitating storage-based track management. As a result, governance
tracks can be modified or added based on real-time decisions and without requiring runtime upgrades.
Polkadot's governance system is designed to be adaptive and decentralized, but modifying the referenda tracks (which determine decision-making paths for proposals) has historically required runtime upgrades. This poses an operational challenge, delaying governance changes until an upgrade @@ -1549,12 +1483,12 @@ is scheduled and executed. The new system provides the flexibility needed to adj dynamically, reflecting real-time changes in governance needs without the latency and risks associated with runtime upgrades. This reduces governance bottlenecks and allows for quicker adaptation to emergent scenarios.
-The protocol modification replaces the current static slice method used for storing referenda tracks with an iterator-based approach that allows tracks to be managed dynamically using chain storage. Governance participants can define and modify referenda tracks as needed, which are then accessed @@ -1566,12 +1500,12 @@ rather enhances them by increasing adaptability.
to alter track configurations based on storage data rather than static definitions. This opens up possibilities for new track types and governance configurations to be deployed without the need for upgrades that might take up weeks. -The most significant drawback is the increased complexity for developers managing track configurations via storage-based iterators, which require careful handling to avoid misuse or inefficiencies.
Additionally, this flexibility could introduce risks if track configurations are modified improperly during runtime, potentially leading to governance instabilities.
-To ensure security, the change must be tested in testnet environments first (Paseo, Westend), particularly in scenarios where multiple track changes happen concurrently. Potential vulnerabilities in governance adjustments must be addressed to prevent abuse.
@@ -1579,27 +1513,27 @@ vulnerabilities in governance adjustments must be addressed to prevent abuse.Comprehensive tests should be conducted to validate correct track modifications in different governance scenarios.
-The proposal optimizes governance track management by avoiding the overhead of runtime upgrades, reducing downtime, and eliminating the need for full consensus on upgrades. However, there is a slight performance cost related to runtime access to storage-based iterators, though this is mitigated by the overall system efficiency gains.
-Developers and governance actors benefit from simplified governance processes but must account for the technical complexity of managing iterator-based track configurations.
Tools may need to be developed to help streamline track adjustments in runtime.
-The change is backward compatible with existing governance operations, and does not require developers to adjust how they interact with referenda tracks.
A migration is required to convert existing statically-defined tracks to dynamic storage-based configurations without disruption.
-This dynamic governance track approach builds on previous work around Polkadot's on-chain governance and leverages standard iterator patterns in Rust programming to improve runtime flexibility. Comparable solutions in other governance networks were examined, but this proposal uniquely tailors them to Polkadot’s decentralized, runtime-upgradable architecture.
-There are already two proposed solutions for both the implementation and
pallet-referenda's TracksInfo to make tracks
@@ -1615,6 +1549,72 @@ return an iterator.pallet-referenda-tracks, which
stores the configurations, and implements TracksInfo using the iterator approach.Table of Contents
+ +:pending_code as intermediate storage key for the runtime code| Start Date | 14.10.2024 |
|---|---|
| Description | Store a runtime upgrade in :pending_code before moving it to :code. |
| Authors | Bastian Köcher |
The code of a runtime is stored in its own state, and when performing a runtime upgrade, this code is replaced. The new runtime can contain runtime migrations that adapt the state to the state layout as defined by the runtime code. This runtime migration is executed when building the first block with the new runtime code. Anything that interacts with the runtime state uses the state layout as defined by the runtime code. So, when trying to load something from the state in the block that applied the runtime upgrade, it will use the new state layout but will decode the data from the non-migrated state. In the worst case, the data is incorrectly decoded, which may lead to crashes or halting of the chain.
+This RFC proposes to store the new runtime code under a different storage key when applying a runtime upgrade. This way, all the off-chain logic can still load the old runtime code under the default storage key and decode the state correctly. The block producer is then required to use this new runtime code to build the next block. While building the next block, the runtime is executing the migrations and moves the new runtime code to the default runtime code location. So, the runtime code found under the default location is always the correct one to decode the state from which the runtime code was loaded.
+While the issue of having undecodable state only exists for the one block in which the runtime upgrade was applied, it still impacts anything that reads state data, like block explorers, UIs, nodes, etc. For block explorers, the issue mainly results in indexing invalid data and UIs may show invalid data to the user. For nodes, reading incorrect data may lead to a performance degradation of the network. There are also ways to prevent certain decoding issues from happening, but it requires that developers are aware of this issue and also requires introducing extra code, which could introduce further bugs down the line.
+So, this RFC tries to solve these issues by fixing the underlying problem of having temporary undecodable state.
+The runtime code is stored under the special key :code in the state. Nodes and other tooling read the runtime code under this storage key when they want to interact with the runtime for e.g., building/importing blocks or getting the metadata to read the state. To update the runtime code the runtime overwrites the value at :code, and then from the next block on, the new runtime will be loaded.
+This RFC proposes to first store the new runtime code under :pending_code in the state for one block. When the next block is being built, the block builder first needs to check if :pending_code is set, and if so, it needs to load the runtime from this storage key. While building the block the runtime will move :pending_code to :code to have the runtime code at the default location. Nodes importing the block will also need to load :pending_code if it exists to ensure that the correct runtime code is used. By doing it this way, the runtime code found at :code in the state of a block will always be able to decode the state.
+Furthermore, this RFC proposes to introduce system_version: 3. The system_version was introduced in RFC42. Version 3 would then enable the usage of :pending_code when applying a runtime code upgrade. This way, the feature can be introduced first and enabled later when the majority of the nodes have upgraded.
Because the first block built with the new runtime code will move the runtime code from :pending_code to :code, the runtime code will need to be loaded. This means the runtime code will appear in the proof of validity of a parachain for the first block built with the new runtime code. Generally this is not a problem as the runtime code is also loaded by the parachain when setting the new runtime code.
+There is still the possibility of having state that is not migrated even when following the proposal as presented by this RFC. The issue is that if the amount of data to be migrated is too big, not all of it can be migrated in one block, because either it takes more time than there is assigned for a block or parachains for example have a fixed budget for their proof of validity. To solve this issue there already exist multi-block migrations that can chunk the migration across multiple blocks. Consensus-critical data needs to be migrated in the first block to ensure that block production etc., can continue. For the other data being migrated by multi-block migrations the migrations could for example expose to the outside which keys are being migrated and should not be indexed until the migration is finished.
Testing should be straightforward and most of the existing testing should already be good enough. Extending with some checks that :pending_code is moved to :code.
The performance should not be impacted besides requiring loading the runtime code in the first block being built with the new runtime code.
+It only alters the way blocks are produced and imported after applying a runtime upgrade. This means that only nodes need to be adapted to the changes of this RFC.
+The change will require that the nodes are upgraded before the runtime starts using this feature. Otherwise they will fail to import the block build by :pending_code.
+For Polkadot/Kusama this means that also the parachain nodes need to be running with a relay chain node version that supports this new feature. Otherwise the parachains will stop producing/finalizing nodes as they can not sync the relay chain any more.
The issue initially reported a bug that led to this RFC. It also discusses multiple solutions for the problem.
+None
+move function. When using the V1 trie layout every value bigger than 32 bytes is put into the db separately. This means a low level move function would only need to move the hash of the runtime code from :code to :pending_code.Table of Contents
TracksInfo using the iter
-
+
@@ -307,7 +307,7 @@ stores the configurations, and implements TracksInfo using the iter
-
+