Undefined
This book contains the Polkadot Fellowship Requests for Comments (RFCs) detailing proposed changes to the technical implementation of the Polkadot network.
polkadot-fellows/RFCs
(source)
Table of Contents
An off-chain approximation protocol should assign rewards based upon the approvals and availability work done by validators.
All validators track which approval votes they actually use, reporting the aggregate, after which an on-chain median computation gives a good approximation under byzantine assumptions. Approval checkers report aggregate information about which availability chunks they use too, but in availability we need a tit-for-tat game to enforce honesty, because approval committees could often bias results thanks to their small size.
We want all or most polkadot subsystems be profitable for validataors, because otherwise operators might profit from running modified code. In particular, almost all rewards in Kusama/Polkadot should come from work done securing parachains, primarily approval checking, but also backing, availability, and support of XCMP.
Among these task, our highest priorities must be approval checks, which ensure soundness, and sending availability chunks to approval checkers. We prove backers must be paid strictly less than approval checkers.
At present though, validators' rewards have relatively little relationship to validators operating costs, in terms of bandwidth and CPU time. Worse, polkadot's scaling makes us particular vulnerable "no-shows" caused by validators skipping their approval checks.
We're particularly concernned about hardware specks impact upon the number of parachain cores. We've requested relatively low spec machines so far, only four physical CPU cores, although some run even lower specs like only two physical CPU cores. Alone, rewards cannot fix our low speced validator problem, but rewards and outreach together should far more impact than either alone.
In future, we'll further increase validator spec requirements, which directly improve polkadot's throughput, and which repeats this dynamic of purging underspeced nodes, except outreach becomes more important because de facto too many slow validators can "out vote" the faster ones
We alter the validators rewards protocol, but with negligable impact upon rewards for honest validators who comply with hardware and bandwidth recommendations.
We shall still reward participation in relay chain concensus of course, which de facto means block production but not finality, but these current reward levels shall wind up greatly reduced. Any validators who manipulate block rewards now could lose rewards here, simply because of rewards being shifted from block production to availability, but this sounds desirable.
We've discussed roughly this rewards protocol in https://hackmd.io/@rgbPIkIdTwSICPuAq67Jbw/S1fHcvXSF and https://github.com/paritytech/polkadot-sdk/issues/1811 as well as related topics like https://github.com/paritytech/polkadot-sdk/issues/5122
We alter the current rewards scheme by reducing to roughly these proportions of total rewards:
We add roughly these proportions of total rewards covering parachain work:
We track this data for each candidate during the approvals process:
/// Our subjective record of out availability transfers for this candidate. +CandidateRewards { + /// Anyone who backed this parablock + backers: [AuthorityId; NumBackers], + /// Anyone to whome we think no-showed, even only briefly. + noshows: HashSet<AuthorityId>, + /// Anyone who sent us chunks for this candidate + downloaded_from: HashMap<AuthorityId,u16>, + /// Anyone to whome we sent chunks for this candidate + uploaded_to: HashMap<AuthorityId,u16>, +} +
We no longer require this data during disputes.
After we approve a relay chain block, then we collect all its CandidateRewards into an ApprovalsTally, with one ApprovalTallyRecord for each validator. In this, we compute approval_usages from the final run of the approvals loop, plus 0.8 for each backer.
CandidateRewards
ApprovalsTally
ApprovalTallyRecord
approval_usages
0.8
/// Our subjective record of what we used from, and provided to, all other validators on the finalized chain +pub struct ApprovalsTally(Vec<ApprovalTallyLine>); + +/// Our subjective record of what we used from, and provided to, all one other validators on the finalized chain +pub struct ApprovalTallyLine { + /// Approvals by this validator which our approvals gadget used in marking candidates approved. + approval_usages: u32, + /// How many times we think this validator no-showed, even only briefly. + noshows: u32 + /// Availability chunks we downloaded from this validator for our approval checks we used. + used_downloads: u32, + /// Availability chunks we uploaded to this validator which whose approval checks we used. + used_uploads: u32, +} +
At finality we sum these ApprovalsTally for one for the whole epoch so far, into another ApprovalsTally. We can optionally sum them earlier at chain heads, but this requires mutablity.
After the epoch is finalized, we share the first two lines of its ApprovalTally.
ApprovalTally
/// Our subjective record of what we used from some other validator on the finalized chain +pub struct ApprovalTallyMessageLine { + /// Approvals by this validator which our approvals gadget used in marking candidates approved. + approval_usages: u32, + /// How many times we think this validator no-showed, even only briefly. + noshows: u32 + /// Availability chunks we downloaded from this validator for our approval checks we used. + used_downloads: u32, +} + +/// Our subjective record of what we used from all other validators on the finalized chain +pub struct ApprovalsTallyMessage(Vec<ApprovalTallyMessageLine>); +
We compute the approvals rewards for each validator by taking the median of the approval_usages fields for each validator across all validators ApprovalsTallyMessages. We compute some noshows_percentiles for each validator similarly, but using a 2/3 precentile instead of the median.
ApprovalsTallyMessage
noshows_percentiles
let mut approval_usages_medians = Vec::new(); +let mut noshows_percentiles = = Vec::new(); +for i in 0..num_validators { + let mut v: Vec<u32> = approvals_tally_messages.iter().map(|atm| atm.0[i].approval_usages); + v.sort(); + approval_usages_medians.push(v[num_validators/2]); + let mut v: Vec<u32> = approvals_tally_messages.iter().map(|atm| atm.0[i].noshows); + v.sort(); + noshows_percentiles.push(v[num_validators/3]); +} +
Assuming more than 50% honersty, these median tell us how many approval votes form each validator.
We re-weight the used_downloads from the ith validator by their median times their expected f+1 chunks and divided by how many chunks downloads they claimed, and sum them
used_downloads
i
f+1
#[cfg(offchain)] +let mut my_missing_uploads = my_approvals_tally.iter().map(|l| l.used_uploads).collect(); +let mut reweighted_total_used_downloads = vec[0u64; num_validators]; +for (mmu,atm) in my_missing_uploads.iter_mut().zip(approvals_tally_messages) { + let d = atm.0.iter().map(|l| l.used_downloads).sum(); + for i in 0..num_validators { + let atm_from_i = approval_usages_medians[i] * (f+1) / d; + #[cfg(offchain)] + if i == me { mmu -= atm_from_i }; + reweighted_total_used_downloads[i] += atm_from_i; + } +} +
We distribute rewards on-chain using approval_usages_medians and reweighted_total_used_downloads. Approval checkers could later change from who they download chunks using my_missing_uploads.
approval_usages_medians
reweighted_total_used_downloads
my_missing_uploads
We deduct small amount of rewards using noshows_medians too, likely 1% of the rewards for an approval, but excuse some small number of noshows, ala noshows_medians[i].saturating_sub(MAX_NO_PENALTY_NOSHOWS).
noshows_medians
noshows_medians[i].saturating_sub(MAX_NO_PENALTY_NOSHOWS)
In theory, validators could adopt whatever strategy they like to penalize validators who stiff them on availability redistribution rewards, except they should not stiff back, only choose other availability providers. We discuss one good strategy below, but initially this could go unimplemented.
Polkadot's efficency creates subtle liveness concerns: Anytime one node cannot perform one of its approval checks then Polkadot loses in expectation 3.25 approval checks, or 0.10833 parablocks. This makes back pressure essential.
We cannot throttle approval checks securely either, so reactive off-chain back pressure only makes sense during or before the backing phase. In other words, if nodes feel overworked themselves, or perhaps beleive others to be, then they should drop backing checks, never approval checks. It follows backing work must be rewarded less well and less reliably than approvals, as otherwise validators could benefit from behavior that harms the network.
We propose that one backing statement be rewarded at 80% of one approval statement, so backers earn only 80% of what approval checkers earn. We omit rewards for availability distribution, so backers spend more on bandwidth too. Approval checkers always fetch chunks first from backers though, so good backers earn roughly 7% there, meaning backing checks earn roughly 13% less than approval checks. We should lower this 80% if we ever increase availability redistribution rewards.
Although imperfect, we believe this simplifies implementation, and provides robustness against mistakes elsewhere, including by governance mistakes, but incurs minimal risk. In principle, backer might not distribute systemic chunks, but approval checkers fetch systemic chunks from backers first anyways, so likely this yields negligable gains.
As always we require that backers' rewards covers their operational costs plus some profit, but approval checks must be more profitable.
In polkadot, all validators run an approval assignment loop for each candidate, in which the validator listens to other approval checkers assignments and approval statements/votes, with which it marks checkers no-show or done, and marks candidates approved. Also, this loop determines and announces validators' own approval checker assignments.
Any validator should always conclude whatever approval checks it begins, but our approval assignment loop ignore some approval checks, either because they were announced too soon or because an earlier no-show delivered its approval vote before the final approval. We say a validator $u$ uses an approval vote by a validator $v$ on a candidate $c$ if the approval assignments loop by $u$ counted the vote by $v$ towards approving the candidate $c$. We should not rewards votes announced too soon, so we unavoidably omit rewards for some honest no-show replacements too. We expect the 80% discount for backing covers these losses, so approval checks remain more profitable than backing.
We propose a simple approximate solution based upon computing medians across validators for used votes.
In an epoch $e$, each validator $u$ counts of the number $\alpha_{u,v}$ of votes they used from each validator $v$, including themselves. Any time a validator marks a candidate approved, they increment these counts appropriately.
After epoch $e$'s last block gets finalized, all validators of epoch $e$ submit an approvals tally message ApprovalsTallyMessage that reveals their number $\alpha_{u,v}$ of useful approvals they saw from each validator $v$ on candidates that became available in epoch $n$. We do not send $\alpha_{u,u}$ for tit-for-tat reasons discussed below, not for bias concerns. We record these approvals tally messages on-chain.
After some delay, we compute on-chain the median $\alpha_v := \textrm{median} { \alpha_{u,v} : u }$ used approvals statements for each validator $v$.
As discussed in https://hackmd.io/@rgbPIkIdTwSICPuAq67Jbw/S1fHcvXSF we could compute these medians using the on-line algorithm if substrate had a nice priority queue.
We never achieve true consensus on approval checkers and their approval votes. Yet, our approval assignment loop gives a rough concensus, under our Byzantine assumption and some synchrony assumption. It then follows that miss-reporting by malicious validators should not appreciably alter the median $\alpha_v$ and hence rewards.
We never tally used approval assignments to candidate equivocations or other forks. Any validator should always conclude whatever approval checks it begins, even on other forks, but we expect relay chain equivocations should be vanishingly rare, and sassafras should make forks uncommon.
We account for noshows similarly, and deduce a much smaller amount of rewards, but require a 2/3 precentile level, not kjust a median.
As approval checkers could easily perform useless checks, we shall reward availability providers for the availability chunks they provide that resulted in useful approval checks. We enforce honesty using a tit-for-tat mechanism because chunk transfers are inherently subjective.
An approval checker reconstructs the full parachain block by downloading distinct $f+1$ chunks from other validators, where at most $f$ validators are byzantine, out of the $n \ge 3 f + 1$ total validators. In downloading chunks, validators prefer the $f+1$ systemic chunks over the non-systemic chunks, and prefer fetching from validators who already voted valid, like backing checkers. It follows some validators should recieve credit for more than one chunk per candidate.
We expect a validator $v$ has actually performed more approval checks $\omega_v$ than the median $\alpha_v$ for which they actually received credit. In fact, approval checkers even ignore some of their own approval checks, meaning $\alpha_{v,v} \le \omega_v$ too.
Alongside approvals count for epoch $e$, approval checker $v$ computes the counts $\beta_{u,v}$ of the number of chunks they downloaded from each availability provider $u$, excluding themselves, for which they percieve the approval check turned out useful, meaning their own approval counts in $\alpha_{v,v}$. Approval checkers publish $\beta_{u,v}$ alongside $\alpha_{u,v}$ in the approvals tally message ApprovalsTallyMessage. We originally proposed include the self availability usage $\beta_{v,v}$ here, but this should not matter, and excluding simplifies the code.
Symmetrically, availability provider $u$ computes the counts $\gamma_{u,v}$ of the number of chunks they uploaded to each approval checker $v$, again including themselves, again for which they percieve the approval check turned out useful. Availability provider $u$ never reveal its $\gamma_{u,v}$ however.
At this point, $\alpha_v$, $\alpha_{v,v}$, and $\alpha_{u,v}$ all potentially differ. We established consensus upon $\alpha_v$ above however, with which we avoid approval checkers printing unearned availability provider rewards:
After receiving "all" pairs $(\alpha_{u,v},\beta_{u,v})$, validator $w$ re-weights the $\beta_{u,v}$ and their own $\gamma_{w,v}$. +$$ +\begin{aligned} +\beta\prime_{w,v} &= {(f+1) \alpha_v \over \sum_u \beta_{u,v}} \beta_{w,v} \ +\gamma\prime_{w,v} &= {(f+1) \alpha_w \over \sum_v \gamma_{w,v}} \gamma_{w,v} \ +\end{aligned} +$$ +At this point, we compute $\beta\prime_w = \sum_v \beta\prime_{w,v}$ on-chain for each $w$ and reward $w$ proportionally.
We employ a tit-for-tat strategy to punish validators who lie about from whome they obtain availability chunks. We only alter validators future choices in from whom they obtain availability chunks, and never punish by lying ourselves, so nothing here breaks polkadot, but not having roughly this strategy enables cheating.
An availability provider $w$ defines $\delta\prime_{w,v} := \gamma\prime_{w,v} - \beta\prime_{w,v}$ to be the re-weighted number of chunks by which $v$ stiffed $w$. Now $w$ increments their cumulative stiffing perception $\eta_{w,v}$ from $v$ by the value $\delta\prime_{w,v}$, so $\eta_{w,v} \mathrel{+}= \delta\prime_{w,v}$
In future, anytime $w$ seeks chunks in reconstruction $w$ skips $v$ proportional to $\eta_{w,v} / \sum_u \eta_{w,u}$, with each skip reducing $\eta_{w,u}$ by 1. We expect honest accedental availability stiffs have only small $\delta\prime_{w,v}$, so they clear out quickly, but intentional skips add up more quickly.
We keep $\gamma_{w,v}$ and $\alpha_{u,u}$ secret so that approval checkers cannot really know others stiffing perceptions, although $\alpha_{u,v}$ leaks some relevant information. We expect this secrecy keeps skips secret and thus prevents the tit-for-tat escalating beyond one round, which hopefully creates a desirable Nash equilibrium.
We favor skiping systematic chunks to reduce reconstructon costs, so we face costs when skipping them. We could however fetch systematic chunks from availability providers as well as backers, or even other approval checkers, so this might not become problematic in practice.
We do not pay backers individually for availability distribution per se. We could only do so by including this information into the availability bitfields, which complicates on-chain computation. Also, if one of the two backers does not distribute then the availability core should remain occupied longer, meaning the lazy backer loses some rewards too. It's likely future protocol improbvements change this, so we should monitor for lazy backers outside the rewards system.
We discuss approvals being considered by the tit-for-tat in earlier drafts. An adversary who successfuly manipulates the rewards median votes would've alraedy violated polkadot's security assumptions though, which requires a hard fork and correcting the dot allocation. Incorrect report wrong approval_usages remain interesting statistics though.
Adversarial validators could manipulates their availability votes though, even without being a supermajority. If they still download honestly, then this costs them more rewards than they earn. We do not prevent validators from preferentially obtaining their pieces from their friends though. We should analyze, or at least observe, the long-term consequences.
A priori, whale nominator's validators could stiff validators but then rotate their validators quickly enough so that they never suffered being skipped back. We discuss several possible solution, and their difficulties, under "Rob's nominator-wise skipping" in https://hackmd.io/@rgbPIkIdTwSICPuAq67Jbw/S1fHcvXSF but overall less seems like more here. Also frequent validator rotation could be penalized elsewhere.
We operate off-chain except for final rewards votes and median tallies. We expect lower overhead rewards protocols would lack information, thereby admitting easier cheating.
Initially, we designed the ELVES approval gadget to allow on-chain operation, in part for rewards computation, but doing so looks expensive. Also, on-chain rewards computaiton remains only an approximation too, but could even be biased more easily than our off-chain protocol presented here.
We alraedy teach validators about missed parachain blocks, but we'll teach approval checking more going forwards, because current efforts focus more upon backing.
JAM's block exports should not complicate availability rewards, but could impact some alternative schemes.
None
Provide specific questions to discuss and address before the RFC is voted on by the Fellowship. This should include, for example, alternatives to aspects of the proposed design where the appropriate trade-off to make is unclear.
Any rewards protocol could simply be "out voted" by too many slow validators: An increase the number of parachain cores increases more workload, but this creates no-shows if too few validators could handle this workload.
We could add a synthetic parachain flag, only settable by governance, which treats no-shows as positive approval votes for that parachain, but without adding rewards. We should never enable this for real parachains, only for synthetic ones like gluttons. We should not enable the synthetic parachain flag long-term even for gluttonsm, because validators could easily modify their code. Yet, synthetic approval checks might enable pushing the hardware upgrades more agressively over the short-term.
:pending_code
:code
The code of a runtime is stored in its own state, and when performing a runtime upgrade, this code is replaced. The new runtime can contain runtime migrations that adapt the state to the state layout as defined by the runtime code. This runtime migration is executed when building the first block with the new runtime code. Anything that interacts with the runtime state uses the state layout as defined by the runtime code. So, when trying to load something from the state in the block that applied the runtime upgrade, it will use the new state layout but will decode the data from the non-migrated state. In the worst case, the data is incorrectly decoded, which may lead to crashes or halting of the chain.
This RFC proposes to store the new runtime code under a different storage key when applying a runtime upgrade. This way, all the off-chain logic can still load the old runtime code under the default storage key and decode the state correctly. The block producer is then required to use this new runtime code to build the next block. While building the next block, the runtime is executing the migrations and moves the new runtime code to the default runtime code location. So, the runtime code found under the default location is always the correct one to decode the state from which the runtime code was loaded.
While the issue of having undecodable state only exists for the one block in which the runtime upgrade was applied, it still impacts anything that reads state data, like block explorers, UIs, nodes, etc. For block explorers, the issue mainly results in indexing invalid data and UIs may show invalid data to the user. For nodes, reading incorrect data may lead to a performance degradation of the network. There are also ways to prevent certain decoding issues from happening, but it requires that developers are aware of this issue and also requires introducing extra code, which could introduce further bugs down the line.
So, this RFC tries to solve these issues by fixing the underlying problem of having temporary undecodable state.
The runtime code is stored under the special key :code in the state. Nodes and other tooling read the runtime code under this storage key when they want to interact with the runtime for e.g., building/importing blocks or getting the metadata to read the state. To update the runtime code the runtime overwrites the value at :code, and then from the next block on, the new runtime will be loaded. +This RFC proposes to first store the new runtime code under :pending_code in the state for one block. When the next block is being built, the block builder first needs to check if :pending_code is set, and if so, it needs to load the runtime from this storage key. While building the block the runtime will move :pending_code to :code to have the runtime code at the default location. Nodes importing the block will also need to load :pending_code if it exists to ensure that the correct runtime code is used. By doing it this way, the runtime code found at :code in the state of a block will always be able to decode the state. +Furthermore, this RFC proposes to introduce system_version: 3. The system_version was introduced in RFC42. Version 3 would then enable the usage of :pending_code when applying a runtime code upgrade. This way, the feature can be introduced first and enabled later when the majority of the nodes have upgraded.
system_version: 3
system_version
RFC42
3
Because the first block built with the new runtime code will move the runtime code from :pending_code to :code, the runtime code will need to be loaded. This means the runtime code will appear in the proof of validity of a parachain for the first block built with the new runtime code. Generally this is not a problem as the runtime code is also loaded by the parachain when setting the new runtime code. +There is still the possibility of having state that is not migrated even when following the proposal as presented by this RFC. The issue is that if the amount of data to be migrated is too big, not all of it can be migrated in one block, because either it takes more time than there is assigned for a block or parachains for example have a fixed budget for their proof of validity. To solve this issue there already exist multi-block migrations that can chunk the migration across multiple blocks. Consensus-critical data needs to be migrated in the first block to ensure that block production etc., can continue. For the other data being migrated by multi-block migrations the migrations could for example expose to the outside which keys are being migrated and should not be indexed until the migration is finished.
Testing should be straightforward and most of the existing testing should already be good enough. Extending with some checks that :pending_code is moved to :code.
The performance should not be impacted besides requiring loading the runtime code in the first block being built with the new runtime code.
It only alters the way blocks are produced and imported after applying a runtime upgrade. This means that only nodes need to be adapted to the changes of this RFC.
The change will require that the nodes are upgraded before the runtime starts using this feature. Otherwise they will fail to import the block build by :pending_code. +For Polkadot/Kusama this means that also the parachain nodes need to be running with a relay chain node version that supports this new feature. Otherwise the parachains will stop producing/finalizing nodes as they can not sync the relay chain any more.
The issue initially reported a bug that led to this RFC. It also discusses multiple solutions for the problem.
move
V1
This proposal introduces PVQ (PolkaVM Query), a unified query interface that bridges different chain runtime implementations and client tools/UIs. PVQ provides an extension-based system where runtime developers can expose chain-specific functionality through standardized interfaces, while allowing client-side developers to perform custom computations on the data through PolkaVM programs. By abstracting away concrete implementations across chains and supporting both off-chain and cross-chain scenarios, PVQ aims to reduce code duplication and development complexity while maintaining flexibility for custom use cases.
In Substrate, runtime APIs facilitate off-chain clients in reading the state of the consensus system. +However, the APIs defined and implemented by individual chains often fall short of meeting the diverse requirements of client-side developers. +For example, client-side developers may want some aggregated data from multiple pallets, or apply various custom transformations on the raw data. +Additionally, chains often implement different APIs and data types (such as AccountId) for similar functionality, which increases complexity and development effort on the client side. +As a result, client-side developers frequently resort to directly accessing storage (which is susceptible to breaking changes) and reimplementing custom computations on the raw data. This leads to code duplication between the Rust runtime logic and UI JavaScript/TypeScript logic, increasing development effort and introducing potential for bugs.
AccountId
Moreover, the diversity also extends to cross-chain queries.
Therefore, a system that serves as an intermediary layer between runtime implementations and client-side implementations with a unified but flexible interface will be beneficial for both sides. It should be able to:
Use cases that will benefit from such a system:
The core idea of PVQ is to have a unified interface that meets the aforementioned requirements.
On the runtime side, an extension-based system is introduced to serve as a standardization layer across different chains. +Each extension specification defines a set of cohesive APIs. +Runtime developers can freely select which extensions they want to implement, and have full control over how the data is sourced - whether from single or multiple pallet functions, with optional data transformations applied. +The runtime aggregates all implemented extensions into a single unified interface that takes a query program and corresponding arguments, and returns the query result. +This interface will be exposed in two ways: as a Substrate RuntimeAPI for off-chain queries, and as an XCM instruction for cross-chain queries.
On the client side or in XCM use cases, the client can easily detect which extensions are supported by a runtime through the metadata API. This allows for runtime feature discovery and enables clients to adapt their behavior based on available functionality. +Client-side developers can encode their desired custom computation logic into the query program and its arguments, while the actual data access happens through runtime-implemented extensions.
In conclusion, PVQ involves three components:
The PVQ extension system has the following features:
Example Design:
The following code declares two extensions: extension_core and extension_fungibles with some associated types.
extension_core
extension_fungibles
#![allow(unused)] +fn main() { +#[extension_decl] +mod extension_core { + #[extension_decl::config] + pub trait Config { + type ExtensionId: Codec; + } + #[extension_decl::view_fns] + pub trait ExtensionCore<T: Config> { + fn has_extension(id: T::ExtensionId) -> bool; + } +} +#[extension_decl] +mod extension_fungibles { + #[extension_decl::config] + pub trait Config { + type AssetId: Codec; + type AccountId: Codec; + type Balance: Codec; + } + + #[extension_decl::view_fns] + pub trait ExtensionFungibles<T: Config> { + fn total_supply(asset: T::AssetId) -> T::Balance; + fn balance(asset: T::AssetId, who: T::AccountId) -> T::Balance; + } +} +}
The following code implements the extensions, amalgamates them and generates the corresponding metadata.
#![allow(unused)] +fn main() { +#[extensions_impl] +mod extensions_impl { + + #[extensions_impl::extensions_config] + pub struct ExtensionsConfig; + #[extensions_impl::extensions_struct] + pub struct Extensions; + + #[extensions_impl::extensions_config_impl] + impl extension_core::Config for ExtensionsConfig { + type ExtensionId = u64; + } + + #[extensions_impl::extensions_config_impl] + impl extension_fungibles::Config for ExtensionsConfig { + type AssetId = u32; + type AccountId = [u8; 32]; + type Balance = u64; + } + + #[extensions_impl::extension_struct_impl] + impl extension_core::ExtensionCore<ExtensionsConfig> for Extensions { + fn has_extension(id: u64) -> bool { + matches!(id, 0 | 1) + } + } + + #[extensions_impl::extension_struct_impl] + impl extension_fungibles::ExtensionFungibles<ExtensionsConfig> for Extensions { + fn total_supply(asset: u32) -> u64 { + 200 + } + fn balance(asset: u32, who: [u8; 32]) -> u64 { + 100 + } + } +} +}
Extensions are uniquely identified by a hash value computed from their name and method names. This means that modifying either the extension name or its method names results in a new extension. This design allows new functionality to be added independently of the PVQ core version, enabling a permissionless extension system while keeping the core implementation minimal.
The extension ID generation can be expressed mathematically as:
$ExtID = twox64(P \parallel E \parallel M_1 \parallel M_2 \parallel ... \parallel M_n)$
Where:
$P$ is the prefix string constant, pvq-ext
pvq-ext
$E$ is the extension name
$M_1...M_n$ are the method names in lexicographical order
$\parallel$ represents string concatenation with a separator @ to avoid collision
@
$twox64()$ is the 64-bit xxHash function
A permission control system allows filtering extension method invocations based on their origin (Runtime, Extrinsics, RuntimeAPI, or XCM). This enables runtime developers to restrict certain functions from being called through specific interfaces, such as preventing access via XCM when desired.
The PVQ Executor provides a unified interface that only takes query programs with corresponding arguments and returns results. It can be formulated as a PVM program-argument invocation, as detailed in Appendix A.8 in the JAM Gray Paper. Specifically, we call it PVQ invocation.
PVQ program size limit: +While the standard PVM code format contains instructions, jump table, and initial RAM state information, PVQ programs can be significantly trimmed down. This is because PVQ separates computation logic from state access: the computation happens in the program while state access is handled through host functions. This makes the program stateless, allowing us to eliminate the initial read-write (heap) data and stack sections along with their length encodings in the PVQ program binary. The read-only data section can be minimized to only contain essential utility data like host function extension IDs, keeping it within a reasonable size limit.
Entrypoint: +PVQ programs have a single static entrypoint that begins at instruction 0, since all PVQ computation can be expressed through a single entry point.
Argument passing: +Query data is encoded as invocation arguments. Specifically, it includes the view function index and its arguments. As discussed in Equation A.36 in the Gray Paper, arguments start at 0xfeff0000 which is stored in a0(7th register), and the length is specified at a1(8th register).
0xfeff0000
a0
a1
Return results +As discussed in Equation A.39 in the Gray Paper, the invocation returns its output through register a0 (7th register), which contains a pointer to the output buffer. Register a1(8th register) contains the length of the output buffer. The output buffer must be allocated within the program's memory space and contain the SCALE-encoded return value.
The following host functions are available to PVQ invocations. The index numbers shown below correspond to the values used in the ecalli instruction.
ecalli
extension_call
extension_id
u64
query_data
a2
a3
The returned results are stored in registers a0 (pointer) and a1 (length). The output buffer contains the SCALE-encoded return value.
All host functions must properly account for and deduct gas based on their computational costs.
Example Rust Implementation using PolkaVM SDK:
#![allow(unused)] +fn main() { +#[polkavm_derive::polkavm_import] +extern "C" { + fn extension_call(extension_id:u64, call_ptr:u32, call_len: u32) -> (u32, u32); +} +}
Practically, the executor has a core method execute to initialize the program and perform argument invocation, which takes:
execute
program
args
gas_limit
Example Rust Implementation:
#![allow(unused)] +fn main() { +pub fn execute( + &mut self, + program: &[u8], + args: &[u8], + gas_limit: u64, +) -> Result<Vec<u8>, PvqExecutorError> {...} +enum PvqExecutorError { + InvalidProgramFormat, + OutOfGas, + // Implementors can define additional error variants to differentiate specific panic reasons for debugging purposes + Panic, +} +}
Additionally, it provides an initialization method that sets up the PVM execution environment and external interfaces by pre-registering the required host functions:
#![allow(unused)] +fn main() { +pub fn new(context: PvqContext) -> Self +}
The RuntimeAPI for off-chain query usage includes two methods:
execute_query: Executes the query and returns the result. It takes:
execute_query
ref_time_limit
metadata: Returns information about available extensions, including their IDs, supported methods, gas costs, etc. This provides feature discovery capabilities. The metadata is encoded using scale-info, following a similar approach to frame-metadata.
metadata
scale-info
frame-metadata
Example PVQ Runtime API:
#![allow(unused)] +fn main() { +decl_runtime_apis! { + pub trait PvqApi { + fn execute_query(program: Vec<u8>, args: Vec<u8>, ref_time_limit: u64) -> PvqResult; + fn metadata() -> Vec<u8>; + } +} +type PvqResult = Result<PvqResponse, PvqError>; +type PvqResponse = Vec<u8>; +enum PvqError { + InvalidProgramFormat, + Timeout, + Panic(String), +} +}
Example Metadata:
#![allow(unused)] +fn main() { +pub struct Metadata { + pub types: PortableRegistry, + pub extensions: Vec<ExtensionMetadata<PortableForm>>, +} +}
The integration of PVQ into XCM is achieved by adding a new instruction to XCM, as well as a new variant of the Response type in the QueryResponse message.
Response
QueryResponse
ReportQuery
PvqResult
Operands:
query: BoundedVec<u8, MAX_QUERY_SIZE>: Encoded bytes of the tuple (program, args). MAX_QUERY_SIZE is the generic parameter type size limit (i.e. 2MB).
query: BoundedVec<u8, MAX_QUERY_SIZE>
(program, args)
MAX_QUERY_SIZE
max_weight: Weight: Maximum weight that the query should take.
max_weight: Weight
info: QueryResponseInfo: Information for making the response.
info: QueryResponseInfo
#![allow(unused)] +fn main() { +ReportQuery { + query: BoundedVec<u8, MAX_QUERY_SIZE>, + max_weight: Weight, + info: QueryResponseInfo, +} +}
PvqResult = 6 (BoundedVec<u8, MaxPvqResult>)
PvqResult is a variant type:
Ok(Vec<u8>)
Err(PanicReason)
FailedToDecode
MaxWeightInvalid
Overflow
BadOrigin
ReanchorFailed
NotHoldingFees
Unroutable
DestinationUnsupported
ExceedsMaxMessageSize
Transport
Testing:
Security:
Privacy: +N/A
As a newly introduced feature, PVQ operates independently and does not impact or degrade the performance of existing runtime implementations.
From the perspective of off-chain tooling, this proposal streamlines development by unifying multiple chain-specific RuntimeAPIs under a single consistent interface. +This significantly benefits wallet and dApp developers by eliminating the need to handle individual implementations for similar operations across different chains. The proposal also enhances development flexibility by allowing custom computations to be modularly encapsulated as PolkaVM programs that interact with the exposed APIs.
For RuntimeAPI integration, the proposal defines new APIs, which do not break compatibility with existing interfaces. +For XCM Integration, the proposal does not modify the existing XCM message format, which is backwards compatible.
There are several discussions related to the proposal, including:
CustomMetadata
Once PVQ and the aforementioned Facade Project are ready, there are opportunities to consolidate overlapping functionality between the two systems. For example, the metadata APIs could potentially be unified to provide a more cohesive interface for runtime information. This would help reduce duplication and improve maintainability while preserving the distinct benefits of each approach.
This RFC proposes a change that makes it possible to identify types of compressed blobs stored on-chain, as well as used off-chain, without the need for decompression.
Currently, a compressed blob does not give any idea of what's inside because the only thing that can be inside, according to the spec, is Wasm. In reality, other blob types are already being used, and more are to come. Apart from being error-prone by itself, the current approach does not allow to properly route the blob through the execution paths before its decompression, which will result in suboptimal implementations when more blob types are used. Thus, it is necessary to introduce a mechanism allowing to identify the blob type without decompressing it.
This proposal is intended to support future work enabling Polkadot to execute PolkaVM and, more generally, other-than-Wasm parachain runtimes, and allow developers to introduce arbitrary compression methods seamlessly in the future.
Node developers are the main stakeholders for this proposal. It also creates a foundation on which parachain runtime developers will build.
The current approach to compressing binary blobs involves using zstd compression, and the resulting compressed blob is prefixed with a unique 64-bit magic value specified in that subsection. The same procedure is used to compress both Wasm code blobs and proofs-of-validity. Currently, having solely a compressed blob, it's impossible to tell what's inside it without decompression, a Wasm blob, or a PoV. That doesn't cause problems in the current protocol, as Wasm blobs and PoV blobs take completely different execution paths in the code.
zstd
The changes proposed below are intended to define the means for distinguishing compressed blob types in a backward-compatible and future-proof way.
It is proposed to introduce an open list of 64-bit prefixes, each representing a compressed blob of a specific type compressed with a specific compression method. The currently used prefix becomes deprecated and will be removed or reused when it is no longer in use.
The proposed list of prefixes to support the current as well as currently known future work follows:
CBLOB_ZSTD_LEGACY
CBLOB_ZSTD_POV
CBLOB_ZSTD_WASM_CODE
CBLOB_ZSTD_PVM_CODE
No runtime code changes should be needed to imnplement this proposal. Node-side changes are trivial; a PoC already implemented as a part of SDK PR#6704 may be used as an example.
Currently, the only requirement for a compressed blob prefix is not to coincide with Wasm magic bytes (as stated in code comments). Changes proposed here increase prefix collision risk, given that arbitrary data may be compressed in the future. However, it must be taken into account that:
As the change increases granularity, it will positively affect both testing possibilities and security, allowing developers to check what's inside a given compressed blob precisely. Testing the change itself is trivial. Privacy is not affected by this change.
The current implementation's performance is not affected by this change. Future implementations allowing for the execution of other-than-Wasm parachain runtimes will benefit from this change performance-wise.
The end-user ergonomics is not affected. The ergonomics for developers will benefit from this change as it enables exact checks and less guessing.
The change is designed to be backward-compatible.
SDK PR#6704 (WIP) introduces a mechanism similar to that described in this proposal and proves the necessity of such a change.
This proposal creates a foundation for two future work directions:
The current election mechanism for permissionless collators on system chains was introduced in RFC-7. This RFC proposes a mechanism to facilitate replacements in the invulnerable sets of system chains by breaking down barriers that exist today.
Following RFC-7 and the introduction of the collator election mechanism, anyone can now collate on a system chain on the permissionless slots, but the invulnerable set has been a contentious issue among @@ -258,12 +1059,12 @@ invulnerable set de facto immutable.
This RFC proposes a periodic, mandatory, round-robin, two-round election mechanism for invulnerables.
The first major drawback of this proposal is that it would put more responsibility on governance by having people vote regularly in order to maintain the invulnerable collator set on each chain. Today the collator-selection pallet employs a fire-and-forget system where the invulnerables are chosen @@ -339,24 +1140,24 @@ possibly communicate with all system chains via XCM. While this voting system co other contexts as well, I don't think it's worth conditioning the invulnerable collator redesign on a separate implementation of the multiple choice voting system when the Two-Round proposed achieves the objectives of this RFC.
collator-selection
All election mechanisms as well as corner cases can be covered with unit tests.
The chain will have to run extrinsics to start and end elections periodically, but the impact in terms of weight and PoV size is negligible.
The invulnerables will be the most affected group, as they will have to now compete in elections periodically to secure their spots. Permissionless candidates will now have a clear, though not guaranteed, path towards becoming an invulnerable, at least for a period of time.
Any changes to the election mechanism of invulnerables should be compatible with the current invulnerable set interaction with the collator set chosen at the session boundary. The current invulnerable set for each chain can be grandfathered in when upgrading the collator-selection pallet version.
This RFC builds on RFC-7, which introduced the election mechanism for system chain collators.
The main spinoff of this RFC might be a multiple choice poll implementation in a separate pallet to hold a First Past the Post election instead of the Two-Round System proposed, which would prompt a migration to the new voting system within the collator-selection pallet. Additionally, a more complex solution where the voting for all system chains happens in a single place which then sends XCM responses with election results back to system chains can be implemented in the next iteration of this RFC.
Update the runtime-host interface to no longer make use of a host-side allocator.
The heap allocation of the runtime is currently controlled by the host using a memory allocator on the host side.
The API of many host functions consists in allocating a buffer. For example, when calling ext_hashing_twox_256_version_1, the host allocates a 32 bytes buffer using the host allocator, and returns a pointer to this buffer to the runtime. The runtime later has to call ext_allocator_free_version_1 on this pointer in order to free the buffer.
ext_hashing_twox_256_version_1
ext_allocator_free_version_1
Even though no benchmark has been done, it is pretty obvious that this design is very inefficient. To continue with the example of ext_hashing_twox_256_version_1, it would be more efficient to instead write the output hash to a buffer that was allocated by the runtime on its stack and passed by pointer to the function. Allocating a buffer on the stack in the worst case scenario simply consists in decreasing a number, and in the best case scenario is free. Doing so would save many Wasm memory reads and writes by the allocator, and would save a function call to ext_allocator_free_version_1.
Furthermore, the existence of the host-side allocator has become questionable over time. It is implemented in a very naive way, and for determinism and backwards compatibility reasons it needs to be implemented exactly identically in every client implementation. Runtimes make substantial use of heap memory allocations, and each allocation needs to go twice through the runtime <-> host boundary (once for allocating and once for freeing). Moving the allocator to the runtime side, while it would increase the size of the runtime, would be a good idea. But before the host-side allocator can be deprecated, all the host functions that make use of it need to be updated to not use it.
No attempt was made at convincing stakeholders.
This section contains a list of new host functions to introduce.
(func $ext_storage_read_version_2 - (param $key i64) (param $value_out i64) (param $offset i32) (result i64)) -(func $ext_default_child_storage_read_version_2 - (param $child_storage_key i64) (param $key i64) (param $value_out i64) - (param $offset i32) (result i64)) -
The signature and behaviour of ext_storage_read_version_2 and ext_default_child_storage_read_version_2 is identical to their version 1 counterparts, but the return value has a different meaning. -The new functions directly return the number of bytes that were written in the value_out buffer. If the entry doesn't exist, a value of -1 is returned. Given that the host must never write more bytes than the size of the buffer in value_out, and that the size of this buffer is expressed as a 32 bits number, a 64bits value of -1 is not ambiguous.
ext_storage_read_version_2
ext_default_child_storage_read_version_2
value_out
-1
The runtime execution stops with an error if value_out is outside of the range of the memory of the virtual machine, even if the size of the buffer is 0 or if the amount of data to write would be 0 bytes.
(func $ext_storage_next_key_version_2 - (param $key i64) (param $out i64) (return i32)) -(func $ext_default_child_storage_next_key_version_2 - (param $child_storage_key i64) (param $key i64) (param $out i64) (return i32)) -
The behaviour of these functions is identical to their version 1 counterparts. -Instead of allocating a buffer, writing the next key to it, and returning a pointer to it, the new version of these functions accepts an out parameter containing a pointer-size to the memory location where the host writes the output. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine, even if the function wouldn't write anything to out. -These functions return the size, in bytes, of the next key, or 0 if there is no next key. If the size of the next key is larger than the buffer in out, the bytes of the key that fit the buffer are written to out and any extra byte that doesn't fit is discarded.
out
0
Some notes:
ext_storage_next_key_version_2
ext_default_child_storage_next_key_version_2
(func $ext_hashing_keccak_256_version_2 - (param $data i64) (param $out i32)) -(func $ext_hashing_keccak_512_version_2 - (param $data i64) (param $out i32)) -(func $ext_hashing_sha2_256_version_2 - (param $data i64) (param $out i32)) -(func $ext_hashing_blake2_128_version_2 - (param $data i64) (param $out i32)) -(func $ext_hashing_blake2_256_version_2 - (param $data i64) (param $out i32)) -(func $ext_hashing_twox_64_version_2 - (param $data i64) (param $out i32)) -(func $ext_hashing_twox_128_version_2 - (param $data i64) (param $out i32)) -(func $ext_hashing_twox_256_version_2 - (param $data i64) (param $out i32)) -(func $ext_trie_blake2_256_root_version_3 - (param $data i64) (param $version i32) (param $out i32)) -(func $ext_trie_blake2_256_ordered_root_version_3 - (param $data i64) (param $version i32) (param $out i32)) -(func $ext_trie_keccak_256_root_version_3 - (param $data i64) (param $version i32) (param $out i32)) -(func $ext_trie_keccak_256_ordered_root_version_3 - (param $data i64) (param $version i32) (param $out i32)) -(func $ext_default_child_storage_root_version_3 - (param $child_storage_key i64) (param $out i32)) -(func $ext_crypto_ed25519_generate_version_2 - (param $key_type_id i32) (param $seed i64) (param $out i32)) -(func $ext_crypto_sr25519_generate_version_2 - (param $key_type_id i32) (param $seed i64) (param $out i32) (return i32)) -(func $ext_crypto_ecdsa_generate_version_2 - (param $key_type_id i32) (param $seed i64) (param $out i32) (return i32)) -
The behaviour of these functions is identical to their version 1 or version 2 counterparts. Instead of allocating a buffer, writing the output to it, and returning a pointer to it, the new version of these functions accepts an out parameter containing the memory location where the host writes the output. The output is always of a size known at compilation time. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine.
(func $ext_default_child_storage_root_version_3 - (param $child_storage_key i64) (param $out i32)) -(func $ext_storage_root_version_3 - (param $out i32)) -
The behaviour of these functions is identical to their version 1 and version 2 counterparts. Instead of allocating a buffer, writing the output to it, and returning a pointer to it, the new versions of these functions accepts an out parameter containing the memory location where the host writes the output. The output is always of a size known at compilation time. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine.
I have taken the liberty to take the version 1 of these functions as a base rather than the version 2, as a PPP deprecating the version 2 of these functions has previously been accepted: https://github.com/w3f/PPPs/pull/6.
(func $ext_storage_clear_prefix_version_3 - (param $prefix i64) (param $limit i64) (param $removed_count_out i32) - (return i32)) -(func $ext_default_child_storage_clear_prefix_version_3 - (param $child_storage_key i64) (param $prefix i64) - (param $limit i64) (param $removed_count_out i32) (return i32)) -(func $ext_default_child_storage_kill_version_4 - (param $child_storage_key i64) (param $limit i64) - (param $removed_count_out i32) (return i32)) -
The behaviour of these functions is identical to their version 2 and 3 counterparts. Instead of allocating a buffer, writing the output to it, and returning a pointer to it, the version 3 and 4 of these functions accepts a removed_count_out parameter containing the memory location to a 8 bytes buffer where the host writes the number of keys that were removed in little endian. The runtime execution stops with an error if removed_count_out is outside of the range of the memory of the virtual machine. The functions return 1 to indicate that there are keys remaining, and 0 to indicate that all keys have been removed.
removed_count_out
Note that there is an alternative proposal to add new host functions with the same names: https://github.com/w3f/PPPs/pull/7. This alternative doesn't conflict with this one except for the version number. One proposal or the other will have to use versions 4 and 5 rather than 3 and 4.
(func $ext_crypto_ed25519_sign_version_2 - (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i32)) -(func $ext_crypto_sr25519_sign_version_2 - (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i32)) -func $ext_crypto_ecdsa_sign_version_2 - (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i32)) -(func $ext_crypto_ecdsa_sign_prehashed_version_2 - (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i64)) -
The behaviour of these functions is identical to their version 1 counterparts. The new versions of these functions accept an out parameter containing the memory location where the host writes the signature. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine, even if the function wouldn't write anything to out. The signatures are always of a size known at compilation time. On success, these functions return 0. If the public key can't be found in the keystore, these functions return 1 and do not write anything to out.
1
Note that the return value is 0 on success and 1 on failure, while the previous version of these functions write 1 on success (as it represents a SCALE-encoded Some) and 0 on failure (as it represents a SCALE-encoded None). Returning 0 on success and non-zero on failure is consistent with common practices in the C programming language and is less surprising than the opposite.
Some
(func $ext_crypto_secp256k1_ecdsa_recover_version_3 - (param $sig i32) (param $msg i32) (param $out i32) (return i64)) -(func $ext_crypto_secp256k1_ecdsa_recover_compressed_version_3 - (param $sig i32) (param $msg i32) (param $out i32) (return i64)) -
The behaviour of these functions is identical to their version 2 counterparts. The new versions of these functions accept an out parameter containing the memory location where the host writes the signature. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine, even if the function wouldn't write anything to out. The signatures are always of a size known at compilation time. On success, these functions return 0. On failure, these functions return a non-zero value and do not write anything to out.
The non-zero value written on failure is:
These values are equal to the values returned on error by the version 2 (see https://spec.polkadot.network/chap-host-api#defn-ecdsa-verify-error), but incremented by 1 in order to reserve 0 for success.
(func $ext_crypto_ed25519_num_public_keys_version_1 - (param $key_type_id i32) (return i32)) -(func $ext_crypto_ed25519_public_key_version_2 - (param $key_type_id i32) (param $key_index i32) (param $out i32)) -(func $ext_crypto_sr25519_num_public_keys_version_1 - (param $key_type_id i32) (return i32)) -(func $ext_crypto_sr25519_public_key_version_2 - (param $key_type_id i32) (param $key_index i32) (param $out i32)) -(func $ext_crypto_ecdsa_num_public_keys_version_1 - (param $key_type_id i32) (return i32)) -(func $ext_crypto_ecdsa_public_key_version_2 - (param $key_type_id i32) (param $key_index i32) (param $out i32)) -
The functions superceded the ext_crypto_ed25519_public_key_version_1, ext_crypto_sr25519_public_key_version_1, and ext_crypto_ecdsa_public_key_version_1 host functions.
ext_crypto_ed25519_public_key_version_1
ext_crypto_sr25519_public_key_version_1
ext_crypto_ecdsa_public_key_version_1
Instead of calling ext_crypto_ed25519_public_key_version_1 in order to obtain the list of all keys at once, the runtime should instead call ext_crypto_ed25519_num_public_keys_version_1 in order to obtain the number of public keys available, then ext_crypto_ed25519_public_key_version_2 repeatedly. -The ext_crypto_ed25519_public_key_version_2 function writes the public key of the given key_index to the memory location designated by out. The key_index must be between 0 (included) and n (excluded), where n is the value returned by ext_crypto_ed25519_num_public_keys_version_1. Execution must trap if n is out of range.
ext_crypto_ed25519_num_public_keys_version_1
ext_crypto_ed25519_public_key_version_2
key_index
n
The same explanations apply for ext_crypto_sr25519_public_key_version_1 and ext_crypto_ecdsa_public_key_version_1.
Host implementers should be aware that the list of public keys (including their ordering) must not change while the runtime is running. This is most likely done by copying the list of all available keys either at the start of the execution or the first time the list is accessed.
(func $ext_offchain_http_request_start_version_2 - (param $method i64) (param $uri i64) (param $meta i64) (result i32)) -
The behaviour of this function is identical to its version 1 counterpart. Instead of allocating a buffer, writing the request identifier in it, and returning a pointer to it, the version 2 of this function simply returns the newly-assigned identifier to the HTTP request. On failure, this function returns -1. An identifier of -1 is invalid and is reserved to indicate failure.
(func $ext_offchain_http_request_write_body_version_2 - (param $method i64) (param $uri i64) (param $meta i64) (result i32)) -(func $ext_offchain_http_response_read_body_version_2 - (param $request_id i32) (param $buffer i64) (param $deadline i64) (result i64)) -
The behaviour of these functions is identical to their version 1 counterpart. Instead of allocating a buffer, writing two bytes in it, and returning a pointer to it, the new version of these functions simply indicates what happened:
ext_offchain_http_request_write_body_version_2
ext_offchain_http_response_read_body_version_2
These values are equal to the values returned on error by the version 1 (see https://spec.polkadot.network/chap-host-api#defn-http-error), but tweaked in order to reserve positive numbers for success.
When it comes to ext_offchain_http_response_read_body_version_2, the host implementers must not read too much data at once in order to not create ambiguity in the returned value. Given that the size of the buffer is always inferior or equal to 4 GiB, this is not a problem.
buffer
(func $ext_offchain_http_response_wait_version_2 - (param $ids i64) (param $deadline i64) (param $out i32)) -
The behaviour of this function is identical to its version 1 counterpart. Instead of allocating a buffer, writing the output to it, and returning a pointer to it, the new version of this function accepts an out parameter containing the memory location where the host writes the output. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine.
The encoding of the response code is also modified compared to its version 1 counterpart and each response code now encodes to 4 little endian bytes as described below:
The buffer passed to out must always have a size of 4 * n where n is the number of elements in the ids.
4 * n
ids
(func $ext_offchain_http_response_header_name_version_1 - (param $request_id i32) (param $header_index i32) (param $out i64) (result i64)) -(func $ext_offchain_http_response_header_value_version_1 - (param $request_id i32) (param $header_index i32) (param $out i64) (result i64)) -
These functions supercede the ext_offchain_http_response_headers_version_1 host function.
ext_offchain_http_response_headers_version_1
Contrary to ext_offchain_http_response_headers_version_1, only one header indicated by header_index can be read at a time. Instead of calling ext_offchain_http_response_headers_version_1 once, the runtime should call ext_offchain_http_response_header_name_version_1 and ext_offchain_http_response_header_value_version_1 multiple times with an increasing header_index, until a value of -1 is returned.
header_index
ext_offchain_http_response_header_name_version_1
ext_offchain_http_response_header_value_version_1
These functions accept an out parameter containing a pointer-size to the memory location where the header name or value should be written. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine, even if the function wouldn't write anything to out.
These functions return the size, in bytes, of the header name or header value. If request doesn't exist or is in an invalid state (as documented for ext_offchain_http_response_headers_version_1) or the header_index is out of range, a value of -1 is returned. Given that the host must never write more bytes than the size of the buffer in out, and that the size of this buffer is expressed as a 32 bits number, a 64bits value of -1 is not ambiguous.
If the buffer in out is too small to fit the entire header name of value, only the bytes that fit are written and the rest are discarded.
(func $ext_offchain_submit_transaction_version_2 - (param $data i64) (return i32)) -(func $ext_offchain_http_request_add_header_version_2 - (param $request_id i32) (param $name i64) (param $value i64) (result i32)) -
Instead of allocating a buffer, writing 1 or 0 in it, and returning a pointer to it, the version 2 of these functions return 0 or 1, where 0 indicates success and 1 indicates failure. The runtime must interpret any non-0 value as failure, but the client must always return 1 in case of failure.
(func $ext_offchain_local_storage_read_version_1 - (param $kind i32) (param $key i64) (param $value_out i64) (param $offset i32) (result i64)) -
This function supercedes the ext_offchain_local_storage_get_version_1 host function, and uses an API and logic similar to ext_storage_read_version_2.
ext_offchain_local_storage_get_version_1
It reads the offchain local storage key indicated by kind and key starting at the byte indicated by offset, and writes the value to the pointer-size indicated by value_out.
kind
key
offset
The function returns the number of bytes that were written in the value_out buffer. If the entry doesn't exist, a value of -1 is returned. Given that the host must never write more bytes than the size of the buffer in value_out, and that the size of this buffer is expressed as a 32 bits number, a 64bits value of -1 is not ambiguous.
(func $ext_offchain_network_peer_id_version_1 - (param $out i64)) -
This function writes the PeerId of the local node to the memory location indicated by out. A PeerId is always 38 bytes long. -The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine.
PeerId
(func $ext_input_size_version_1 - (return i64)) -(func $ext_input_read_version_1 - (param $offset i64) (param $out i64)) -
When a runtime function is called, the host uses the allocator to allocate memory within the runtime where to write some input data. These two new host functions provide an alternative way to access the input that doesn't make use of the allocator.
The ext_input_size_version_1 host function returns the size in bytes of the input data.
ext_input_size_version_1
The ext_input_read_version_1 host function copies some data from the input data to the memory of the runtime. The offset parameter indicates the offset within the input data where to start copying, and must be inferior or equal to the value returned by ext_input_size_version_1. The out parameter is a pointer-size containing the buffer where to write to. -The runtime execution stops with an error if offset is strictly superior to the size of the input data, or if out is outside of the range of the memory of the virtual machine, even if the amount of data to copy would be 0 bytes.
ext_input_read_version_1
In addition to the new host functions, this RFC proposes two changes to the runtime-host interface:
(func (result i64))
__heap_base
All the host functions that are being superceded by new host functions are now considered deprecated and should no longer be used. -The following other host functions are similarly also considered deprecated:
ext_storage_get_version_1
ext_default_child_storage_get_version_1
ext_allocator_malloc_version_1
ext_offchain_network_state_version_1
This RFC might be difficult to implement in Substrate due to the internal code design. It is not clear to the author of this RFC how difficult it would be.
The API of these new functions was heavily inspired by API used by the C programming language.
The changes in this RFC would need to be benchmarked. This involves implementing the RFC and measuring the speed difference.
It is expected that most host functions are faster or equal speed to their deprecated counterparts, with the following exceptions:
ext_input_size_version_1/ext_input_read_version_1 is inherently slower than obtaining a buffer with the entire data due to the two extra function calls and the extra copying. However, given that this only happens once per runtime call, the cost is expected to be negligible.
The ext_crypto_*_public_keys, ext_offchain_network_state, and ext_offchain_http_* host functions are likely slightly slower than their deprecated counterparts, but given that they are used only in offchain workers this is acceptable.
ext_crypto_*_public_keys
ext_offchain_network_state
ext_offchain_http_*
It is unclear how replacing ext_storage_get with ext_storage_read and ext_default_child_storage_get with ext_default_child_storage_read will impact performances.
ext_storage_get
ext_storage_read
ext_default_child_storage_get
ext_default_child_storage_read
It is unclear how the changes to ext_storage_next_key and ext_default_child_storage_next_key will impact performances.
ext_storage_next_key
ext_default_child_storage_next_key
After this RFC, we can remove from the source code of the host the allocator altogether in a future version, by removing support for all the deprecated host functions. -This would remove the possibility to synchronize older blocks, which is probably controversial and requires a some preparations that are out of scope of this RFC.
This RFC proposes a solution to replicate an existing pure proxy from one chain to others. The aim is to address the current limitations where pure proxy accounts, which are keyless, cannot have their proxy relationships recreated on different chains. This leads to issues where funds or permissions transferred to the same keyless account address on chains other than its origin chain become inaccessible.
A pure proxy is a new account created by a primary account. The primary account is set as a proxy for the pure proxy account, managing it. Pure proxies are keyless and non-reproducible, meaning they lack a private key and have an address derived from a preimage determined by on-chain logic. More on pure proxies can be found here.
For the purpose of this document, we define a keyless account as a "pure account", the controlling account as a "proxy account", and the entire relationship as a "pure proxy".
The relationship between a pure account (e.g., account ID: pure1) and its proxy (e.g., account ID: alice) is stored on-chain (e.g., parachain A) and currently cannot be replicated to another chain (e.g., parachain B). Because the account pure1 is keyless and its proxy relationship with alice is not replicable from the parachain A to the parachain B, alice does not control the pure1 account on the parachain B.
pure1
alice
A
B
Although this behaviour is not promised, users and clients often mistakenly expect alice to control the same pure1 account on the parachain B. As a result, assets transferred to the account or permissions granted for it are inaccessible. Several factors contribute to this misuse:
Given that these mistakes are likely, it is necessary to provide a solution to either prevent them or enable access to a pure account on a target chain.
Runtime Users, Runtime Devs, wallets, cross-chain dApps.
One possible solution is to allow a proxy to create or replicate a pure proxy relationship for the same pure account on a target chain. For example, Alice, as the proxy of the pure1 pure account on parachain A, should be able to set a proxy for the same pure1 account on parachain B.
To minimise security risks, the parachain B should grant the parachain A the least amount of permission necessary for the replication. First, Parachain A claims to Parachain B that the operation is commanded by the pure account, and thus by its proxy, and second, provides proof that the account is keyless.
The replication process will be facilitated by XCM, with the first claim made using the DescendOrigin instruction. The replication call on parachain A would require a signed origin by the pure account and construct an XCM program for parachain B, where it first descends the origin, resulting in the ParachainA/AccountId32(pure1) origin location on the receiving side.
DescendOrigin
ParachainA/AccountId32(pure1)
To prove that the pure account is keyless, the client must provide the initial preimage used by the chain to derive the pure account. Parachain A verifies it and sends it to parachain B with the replication request.
We can draft a pallet extension for the proxy pallet, which needs to be initialised on both sides to enable replication:
#![allow(unused)] -fn main() { -// Simplified version to illustrate the concept. -mod pallet_proxy_replica { - /// The part of the pure account preimage that has to be provided by a client. - struct Witness { - /// Pure proxy swapner - spawner: AccountId, - /// Disambiguation index - index: u16, - /// The block height and extrinsic index of when the pure account was created. - block_number: BlockNumber, - /// The extrinsic index. - ext_index: u32, - // Part of the preimage, but constant. - // proxy_type: ProxyType::Any, - } - // ... - - /// The replication call to be initiated on the source chain. - // Simplified version, the XCM part will be abstracted by the `Config` trait. - fn replicate(origin: SignedOrigin, witness: Witness, proxy: xcm::Location) -> ... { - let pure = ensure_signed(origin); - ensure!(pure == proxy_pallet::derive_pure_account(witness), Error::NotPureAccount); - let xcm = vec![ - DescendOrigin(who), - Transact( - // … - origin_kind: OriginKind::Xcm, - call: pallet_proxy_replica::create(witness, proxy).encode(), - ) - ]; - xcmTransport::send(xcm)?; - } - // … - - /// The call initiated by the source chain on the receiving chain. - // `Config::CreateOrigin` - generally open for whitelisted parachain IDs and - // converts `Origin::Xcm(ParachainA/AccountId32(pure1))` to `AccountID(pure1)`. - fn create(origin: Config::CreateOrigin, witness: Witness, proxy: xcm::Location) -> ... { - let pure = T::CreateOrigin::ensure_origin(origin); - ensure!(pure == proxy_pallet::derive_pure_account(witness), Error::NotPureAccount); - proxy_pallet::create_pure_proxy(pure, proxy); - } -} - -}
There are two disadvantages to this approach:
We could eliminate the first disadvantage by allowing only the spawner of the pure proxy to recreate the pure proxies, if they sign the transaction on a remote chain and supply the witness/preimage. Since the preimage of a pure account includes the account ID of the spawner, we can verify that the account signing the transaction is indeed the spawner of the given pure account. However, this approach would grant exclusive rights to the spawner over the pure account, which is not a property of pure proxies at present. This is why it's not an option for us.
As an alternative to requiring clients to provide a witness data, we could label pure accounts on the source chain and trust it on the receiving chain. However, this would require the receiving chain to place greater trust in the source chain. If the source chain is compromised, any type of account on the trusting chain could also be compromised.
A conceptually different solution would be to not implement replication of pure proxies and instead inform users that ownership of a pure proxy on one chain does not imply ownership of the same account on another chain. This solution seems complex, as it would require UIs and clients to adapt to this understanding. Moreover, mistakes would likely remain unavoidable.
Each chain expressly authorizes another chain to replicate its pure proxies, accepting the inherent risk of that chain potentially being compromised. This authorization allows a malicious actor from the compromised chain to take control of any pure proxy account on the chain that granted the authorization. However, this is limited to pure proxies that originated from the compromised chain if they have a chain-specific seed within the preimage.
There is a security issue, not introduced by the proposed solution but worth mentioning. The same spawner can create the pure accounts on different chains controlled by the different accounts. This is possible because the current preimage version of the proxy pallet does not include any non-reproducible, chain-specific data, and elements like block numbers and extrinsic indexes can be reproduced with some effort. This issue could be addressed by adding a chain-specific seed into the preimages of pure accounts.
The replication is facilitated by XCM, which adds some additional load to the communication channel. However, since the number of replications is not expected to be large, the impact is minimal.
The proposed solution does not alter any existing interfaces. It does require clients to obtain the witness data which should not be an issue with support of an indexer.
None.
This proposal introduces XCQ (Cross Consensus Query), which aims to serve as an intermediary layer between different chain runtime implementations and tools/UIs, to provide a unified interface for cross-chain queries. -XCQ abstracts away concrete implementations across chains and supports custom query computations.
XCQ
Use cases benefiting from XCQ include:
In Substrate, runtime APIs facilitate off-chain clients in reading the state of the consensus system. -However, different chains may expose different APIs for a similar query or have varying data types, such as doing custom transformations on direct data, or differing AccountId types. -This diversity also extends to client-side, which may require custom computations over runtime APIs in various use cases. -Therefore, tools and UI developers often access storage directly and reimplement custom computations to convert data into user-friendly representations, leading to duplicated code between Rust runtime logic and UI JS/TS logic. -This duplication increases workload and potential for bugs.
Therefore, a system is needed to serve as an intermediary layer between concrete chain runtime implementations and tools/UIs, to provide a unified interface for cross-chain queries.
The overall query pattern of XCQ consists of three components:
The runtime API for off-chain query usage includes two methods:
#![allow(unused)] -fn main() { -decl_runtime_apis! { - pub trait XcqApi { - fn execute_query(query: Vec<u8>, input: Vec<u8>, ref_time_limit: u64) -> XcqResult; - fn metadata() -> Vec<u8>; - } -} -type XcqResult = Result<XcqResponse, XcqError>; -type XcqResponse = Vec<u8>; -enum XcqError { - Custom(String), -} -}
#![allow(unused)] -fn main() { -pub struct Metadata { - pub types: PortableRegistry, - pub extensions: Vec<ExtensionMetadata<PortableForm>>, -} -}
An XCQ executor is a runtime module that executes XCQ queries. -It has a core method execute that takes a PolkaVM program binary, -method name of the exported functions in the PolkaVM program, input arguments, and weight limit that the PolkaVM program can consume.
#![allow(unused)] -fn main() { -pub fn execute( - &mut self, - raw_blob: &[u8], - method: &str, - input: &[u8], - ref_time_limit: u64, -) -> Result<Vec<u8>, XcqExecutorError> {...} -}
An extension-based design is essential for several reasons:
Essential components of an XCQ extension system include:
A hash-based extension id generation mechanism for addressing and versioning . The hash value derives from the extension name and its method sets. Any update to an extension is treated as a new extension.
decl_extension macro: Defines an extension as a Rust trait with optional associated types.
decl_extension
Example usage:
#![allow(unused)] -fn main() { -use xcq_extension::decl_extension; - -pub trait Config { - type AssetId: Codec; - type AccountId: Codec; - type Balance: Codec; -} -decl_extension! { - pub trait ExtensionFungibles { - type Config: Config; - fn total_supply(asset: <Self::Config as Config>::AssetId) -> <Self::Config as Config>::Balance; - fn balance(asset: <Self::Config as Config>::AssetId, who: <Self::Config as Config>::AccountId) -> <Self::Config as Config>::Balance; - } -} -}
impl_extensions
Example Usage:
#![allow(unused)] -fn main() { -// ExtensionImpl is an aggregate struct to impl different extensions -struct ExtensionImpl; -impl extension_fungibles::Config for ExtensionImpl { - type AssetId = u32; - type AccountId = [u8; 32]; - type Balance = u64; -} -impl_extensions! { - impl extension_core::ExtensionCore for ExtensionImpl { - type Config = ExtensionImpl; - fn has_extension(id: <Self::Config as extension_core::Config>::ExtensionId) -> bool { - matches!(id, 0 | 1) - } - } - - impl extension_fungibles::ExtensionFungibles for ExtensionImpl { - type Config = ExtensionImpl; - #[allow(unused_variables)] - fn total_supply(asset: <Self::Config as extension_fungibles::Config>::AssetId) -> <Self::Config as extension_fungibles::Config>::Balance { - 200 - } - #[allow(unused_variables)] - fn balance(asset: <Self::Config as extension_fungibles::Config>::AssetId, who: <Self::Config as extension_fungibles::Config>::AccountId) -> <Self::Config as extension_fungibles::Config>::Balance { - 100 - } - } -} -}
ExtensionExecutor
xcq-executor
host_call
#![allow(unused)] -fn main() { -linker - .define_typed( - "host_call", - move |caller: Caller<'_, Self::UserData>, - extension_id: u64, - call_ptr: u32, - call_len: u32| - -> Result<u64, ExtensionError> { - ... - }); -}
PermissionController
#![allow(unused)] -fn main() { -pub trait PermissionController { - fn is_allowed(extension_id: ExtensionIdTy, call: &[u8], source: InvokeSource) -> bool; -} -#[derive(Copy, Clone)] -pub enum InvokeSource { - RuntimeAPI, - XCM, - Extrinsic, - Runtime, -} -}
An XCQ program is structured as a PolkaVM program with the following key components:
Imported Functions:
host_call: Dispatches call requests to the XCQ Extension Executor.
#![allow(unused)] -fn main() { -#[polkavm_derive::polkavm_import] -extern "C" { - fn host_call(extension_id: u64, call_ptr: u32, call_len: u32) -> u64; -} -}
Results are SCALE-encoded bytes, with the pointer address (lower 32 bits) and length (higher 32 bits) packed into a u64.
return_ty: Returns the type of the function call result.
return_ty
#![allow(unused)] -fn main() { -#[polkavm_derive::polkavm_import] -extern "C" { - fn return_ty(extension_id: u64, call_index: u32) -> u64; -} -}
Results are SCALE-encoded bytes, with the pointer address and length packed similarly to host_call.
Exported Functions:
main
The interaction between an XCQ program and the XCQ Extension Executor follows these steps:
Program Loading: The Executor loads the PolkaVM program binary.
Environment Setup: The Executor configures the PolkaVM environment, registering host functions like host_call and return_ty.
Main Function Execution: The Executor calls the program's main function, passing serialized query arguments.
Program Execution:
Result Retrieval: The Executor reads the result from shared memory and returns it to the caller.
The integration of XCQ into XCM is achieved by adding a new instruction to XCM, as well as a new variant of the Response type in QueryResponse message.:
#![allow(unused)] -fn main() { -ReportQuery { - query: BoundedVec<u8, SIZE_LIMIT>, - max_weight: Weight, - info: QueryResponseInfo, -} -}
Report to a given destination the results of an XCQ query. After query, a QueryResponse message of type XcqResult will be sent to the described destination.
XcqResult
query: BoundedVec<u8, SIZE_LIMIT>: which is the encoded bytes of the tuple (program, input):
query: BoundedVec<u8, SIZE_LIMIT>
(program, input)
program: Vec<u8>
input: Vec<u8>
SIZE_LIMIT
max_weight: Weight: The maximum weight that the query should take.
Add a new variant to the Response type in QueryResponse
XcqResult = 6 (Vec<u8>) -The containing bytes is the SCALE-encoded XcqResult
XcqResult = 6 (Vec<u8>)
frame_support::storage::with_transaction
It's a new functionality, which doesn't modify the existing implementations.
The proposal facilitate the wallets and dApps developers. Developers no longer need to examine every concrete implementation to support conceptually similar operations across different chains. Additionally, they gain a more modular development experience through encapsulating custom computations over the exposed APIs in PolkaVM programs.
The proposal defines new apis, which doesn't break compatibility with existing interfaces.
view functions
view function
No specific considerations.
Parachains already deployed into the Polkadot UC must have a clear plan of action to migrate to an agile Coretime market.
While this proposal does not introduce documentable features per se, adequate documentation must be provided to potential purchasers of Polkadot Coretime. This SHOULD include any alterations to the Polkadot-SDK software collection.
A regular security review SHOULD be conducted prior to deployment through a review by the Web3 Foundation economic research group.
Any final implementation MUST pass a professional external security audit.
The proposal introduces no new privacy concerns.
RFC-3 proposes a means of implementing the high-level allocations within the Relay-chain.
RFC-5 proposes the API for interacting with Relay-chain.
Additional work should specify the interface for the instantaneous market revenue so that the Coretime-chain can ensure Bulk Coretime placed in the instantaneous market is properly compensated.
Robert Habermeier initially wrote on the subject of Polkadot blockspace-centric in the article Polkadot Blockspace over Blockchains. While not going into details, the article served as an early reframing piece for moving beyond one-slot-per-chain models and building out secondary market infrastructure for resource allocation.
For request_revenue_info, a successful request should be possible if when is no less than the Relay-chain block number on arrival of the message less 100,000.
request_revenue_info
when
For assign_core, a successful request should be possible if begin is no less than the Relay-chain block number on arrival of the message plus 10 and workload contains no more than 100 items.
assign_core
begin
workload
Standard Polkadot testing and security auditing applies.
RFC-1 proposes a means of determining allocation of Coretime using this interface.
None at present.
The primary drawback is a reliance on governance for continued treasury funding of infrastructure costs for Invulnerable collators.
The vast majority of cases can be covered by unit testing. Integration test should ensure that the Collator Selection UpdateOrigin, which has permission to modify the Invulnerables and desired number of Candidates, can handle updates over XCM from the system's governance location.
UpdateOrigin
This proposal has very little impact on most users of Polkadot, and should improve the performance of system chains by reducing the number of missed blocks.
This RFC is compatible with the existing implementation and can be handled via upgrades and migration.
None at this time.
There may exist in the future system chains for which this model of collator selection is not appropriate. These chains should be evaluated on a case-by-case basis.
The maximum size of a response is set to an arbitrary 16kiB. The responding side should make sure to conform to this limit. Given that fork_id is typically very small and that the only variable-length field is addrs, this is easily achieved by limiting the number of addresses.
fork_id
addrs
Implementers should be aware that addrs might be very large, and are encouraged to limit the number of addrs to an implementation-defined value.
The peer_id and addrs fields are in theory not strictly needed, as the PeerId and addresses could be always equal to the PeerId and addresses of the node being registered as the provider and serving the response. However, the Cumulus implementation currently uses two different networking stacks, one of the parachain and one for the relay chain, using two separate PeerIds and addresses, and as such the PeerId and addresses of the other networking stack must be indicated. Asking them to use only one networking stack wouldn't feasible in a realistic time frame.
peer_id
The values of the genesis_hash and fork_id fields cannot be verified by the requester and are expected to be unused at the moment. Instead, a client that desires connecting to a parachain is expected to obtain the genesis hash and fork ID of the parachain from the parachain chain specification. These fields are included in the networking protocol nonetheless in case an acceptable solution is found in the future, and in order to allow use cases such as discovering parachains in a not-strictly-trusted way.
genesis_hash
For this reason, an attacker can abuse this mechanism by randomly generating libp2p PeerIds until they find the 20 entries closest to the key representing the target parachain. They are then in control of the parachain bootnodes. Because the key changes periodically and isn't predictable, and assuming that the Polkadot DHT is sufficiently large, it is not realistic for an attack like this to be maintained in the long term.
Furthermore, parachain clients are expected to cache a list of known good nodes on their disk. If the mechanism described in this RFC went down, it would only prevent new nodes from accessing the parachain, while clients that have connected before would not be affected.
The DHT mechanism generally has a low overhead, especially given that publishing providers is done only every 24 hours.
Doing a Kademlia iterative query then sending a provider record shouldn't take more than around 50 kiB in total of bandwidth for the parachain bootnode.
Irrelevant.
While it fundamentally doesn't change much to this RFC, using BabeApi_currentEpoch and BabeApi_nextEpoch might be inappropriate. I'm not familiar enough with good practices within the runtime to have an opinion here. Should it be an entirely new pallet?
BabeApi_currentEpoch
BabeApi_nextEpoch
It is possible that in the future a client could connect to a parachain without having to rely on a trusted parachain specification.
child_trie_info
includeDescendants
This protocol keeps the same maximum response size limit as currently exists (16 MiB). It is not possible for the querier to know in advance whether its query will lead to a reply that exceeds the maximum size. If the reply is too large, the replier should send back only a limited number (but at least one) of requested items in the proof. The querier should then send additional requests for the rest of the items. A response containing none of the requested items is invalid.
The server is allowed to silently discard some keys of the request if it judges that the number of requested keys is too high. This is in line with the fact that the server might truncate the response.
This proposal doesn't handle one specific situation: what if a proof containing a single specific item would exceed the response size limit? For example, if the response size limit was 1 MiB, querying the runtime code (which is typically 1.0 to 1.5 MiB) would be impossible as it's impossible to generate a proof less than 1 MiB. The response size limit is currently 16 MiB, meaning that no single storage item must exceed 16 MiB.
Unfortunately, because it's impossible to verify a Merkle proof before having received it entirely, parsing the proof in a streaming way is also not possible.
A way to solve this issue would be to Merkle-ize large storage items, so that a proof could include only a portion of a large storage item. Since this would require a change to the trie format, it is not realistically feasible in a short time frame.
The main security consideration concerns the size of replies and the resources necessary to generate them. It is for example easily possible to ask for all keys and values of the chain, which would take a very long time to generate. Since responses to this networking protocol have a maximum size, the replier should truncate proofs that would lead to the response being too large. Note that it is already possible to send a query that would lead to a very large reply with the existing network protocol. The only thing that this proposal changes is that it would make it less complicated to perform such an attack.
Implementers of the replier side should be careful to detect early on when a reply would exceed the maximum reply size, rather than inconditionally generate a reply, as this could take a very large amount of CPU, disk I/O, and memory. Existing implementations might currently be accidentally protected from such an attack thanks to the fact that requests have a maximum size, and thus that the list of keys in the query was bounded. After this proposal, this accidental protection would no longer exist.
Malicious server nodes might truncate Merkle proofs even when they don't strictly need to, and it is not possible for the client to (easily) detect this situation. However, malicious server nodes can already do undesirable things such as throttle down their upload bandwidth or simply not respond. There is no need to handle unnecessarily truncated Merkle proofs any differently than a server simply not answering the request.
It is unclear to the author of the RFC what the performance implications are. Servers are supposed to have limits to the amount of resources they use to respond to requests, and as such the worst that can happen is that light client requests become a bit slower than they currently are.
The prior networking protocol is maintained for now. The older version of this protocol could get removed in a long time.
None. This RFC is a clean-up of an existing mechanism.
The current networking protocol could be deprecated in a long time. Additionally, the current "state requests" protocol (used for warp syncing) could also be deprecated in favor of this one.
Collective removal may also come with other governance calls, for example voiding any scheduled Treasury spends that would fund the given collective.
Passing a Root origin referendum is slow. However, given the network's investment (in terms of code maintenance and salaries) in a new collective, this is an appropriate step.
No impacts.
Generally all new collectives will be in the Collectives parachain. Thus, performance impacts should strictly be limited to this parachain and not affect others. As the majority of logic for collectives is generalized and reusable, we expect most collectives to be instances of similar subsets of modules. That is, new collectives should generally be compatible with UIs and other services that provide collective-related functionality, with little modifications to support new ones.
The launch of the Technical Fellowship, see the initial forum post.
1. Multi-Block-Migrations: The runtime is being put into lock-down mode for the duration of the migration process by returning OnlyInherents from initialize_block. This ensures that no user provided transaction can interfere with the migration process. It is absolutely necessary to ensure this, otherwise a transaction could call into un-migrated storage and violate storage invariants.
OnlyInherents
initialize_block
2. poll is possible by using apply_extrinsic as entry-point and not hindered by this approach. It would not be possible to use a pallet inherent like System::last_inherent to achieve this for two reasons: First is that pallets do not have access to AllPalletsWithSystem which is required to invoke the poll hook on all pallets. Second is that the runtime does currently not enforce an order of inherents.
poll
apply_extrinsic
System::last_inherent
AllPalletsWithSystem
3. System::PostInherents can be done in the same manner as poll.
System::PostInherents
The previous drawback of cementing the order of inherents has been addressed and removed by redesigning the approach. No further drawbacks have been identified thus far.
The new logic of initialize_block can be tested by checking that the block-builder will skip transactions when OnlyInherents is returned.
Security: n/a
Privacy: n/a
The performance overhead is minimal in the sense that no clutter was added after fulfilling the requirements. The only performance difference is that initialize_block also returns an enum that needs to be passed through the WASM boundary. This should be negligible.
The advice here is OPTIONAL and outside of the RFC. To not degrade user experience, it is recommended to ensure that an updated node can still import historic blocks.
The RFC is currently being implemented in polkadot-sdk#1781 (formerly substrate#14275). Related issues and merge requests:
ExtrinsicInclusionMode
Is post_inherents more consistent instead of last_inherent? Then we should change it. => renamed to last_inherent
post_inherents
last_inherent
The long-term future here is to move the block building logic into the runtime. Currently there is a tight dance between the block author and the runtime; the author has to call into different runtime functions in quick succession and exact order. Any misstep causes the block to be invalid. This can be unified and simplified by moving both parts into the runtime.
Parachain locks are designed in such way to ensure the decentralization of parachains. If parachains are not locked when it should be, it could introduce centralization risk for new parachains.
For example, one possible scenario is that a collective may decide to launch a parachain fully decentralized. However, if the parachain is unable to produce block, the parachain manager will be able to replace the wasm and genesis without the consent of the collective.
It is considered this risk is tolerable as it requires the wasm/genesis to be invalid at first place. It is not yet practically possible to develop a parachain without any centralized risk currently.
This RFC should improve the developer experiences for new and existing parachain teams
This RFC is fully compatibility with existing interfaces.
None at this stage.
This RFC is only intended to be a short term solution. Slots will be removed in future and lock mechanism is likely going to be replaced with a more generalized parachain manage & recovery system in future. Therefore long term impacts of this RFC are not considered.
https://github.com/paritytech/cumulus/issues/377 @@ -2787,17 +2695,17 @@ This can be unified and simplified by moving both parts into the runtime.
Other than all other system chains, development and maintenance of the Encointer Network is mainly financed by the KSM Treasury and possibly the DOT Treasury in the future. Encointer is dedicated to maintaining its network and runtime code for as long as possible, but there is a dependency on funding which is not in the hands of the fellowship. The only risk in the context of funding, however, is that the Encointer runtime will see less frequent updates if there's less funding.
No changes to the existing system are proposed. Only changes to how maintenance is organized.
No changes
Existing Encointer runtime repo
None identified
More info on Encointer: encointer.org
These subsystems will have reduced resources in cores than on the Relay Chain. Staking in particular may require some optimizations to deal with constraints.
Standard audit/review requirements apply. More powerful multi-chain integration test tools would be useful in developement.
Describe the impact of the proposal on the exposed functionality of Polkadot.
This is an optimization. The removal of public/user transactions on the Relay Chain ensures that its @@ -3905,7 +3813,7 @@ runtimes to recognize the new locations in the network.
Implementing this proposal will require some changes to pallet APIs and/or a pub-sub protocol. Application developers will need to interact with multiple chains in the network.
There remain some implementation questions, like how to use balances for both Staking and Governance. See, for example, Moving Staking off the Relay Chain.
Ideally the Relay Chain becomes transactionless, such that not even balances are represented there. With Staking and Governance off the Relay Chain, this is not an unreasonable next step.
With Identity on Polkadot, Kusama may opt to drop its People Chain.
There should be no drawbacks as it would replace state_version with same behavior but documentation should be updated so that chains know which system_version to use.
state_version
AFAIK, should not have any impact on the security or privacy.
These changes should be compatible for existing chains if they use state_version value for system_verision.
system_verision
I do not believe there is any performance hit with this change.
This does not break any exposed Apis.
This change should not break any compatibility.
We proposed introducing a similar change by introducing a parameter to frame_system::Config but did not feel that is the correct way of introducing this change.
frame_system::Config
I do not have any specific questions about this change at the moment.
IMO, this change is pretty self-contained and there won't be any future work necessary.
The host function MUST return an unsigned 64-bit integer value representing the current proof size. In block-execution and block-import contexts, this function MUST return the current size of the proof. To achieve this, parachain node implementors need to enable proof recording for block imports. In other contexts, this function MUST return 18446744073709551615 (u64::MAX), which represents disabled proof recording.
Parachain nodes need to enable proof recording during block import to correctly implement the proposed host function. Benchmarking conducted with balance transfers has shown a performance reduction of around 0.6% when proof recording is enabled.
The host function proposed in this RFC allows parachain runtime developers to keep track of the proof size. Typical usage patterns would be to keep track of the overall proof size or the difference between subsequent calls to the host function.
Parachain teams will need to include this host function to upgrade.
where the constant a moves the inflection to lower or higher x values, the constant b adjusts the rate of the deposit increase, and the independent variable x is the number of collections or items, depending on application.
a
x
b
Modifying deposit requirements necessitates a balanced assessment of the potential drawbacks. Highlighted below are cogent points extracted from the discourse on the Polkadot Forum conversation, @@ -4270,7 +4178,7 @@ Polkadot Asset Hub and 191 on Kusama Asset Hub with a relatively low volume.
As noted above, state bloat is a security concern. In the case of abuse, governance could adapt by increasing deposit rates and/or using forceDestroy on collections agreed to be spam.
forceDestroy
The primary performance consideration stems from the potential for state bloat due to increased activity from lower deposit requirements. It's vital to monitor and manage this to avoid any @@ -4537,7 +4445,7 @@ struct (added in https://github.com/paritytech/polkadot-sdk/pull/2177Configuration::set_node_feature extrinsic. Once the feature is enabled and new configuration is live, the validator->chunk mapping ceases to be a 1:1 mapping and systematic recovery may begin.
https://github.com/paritytech/polkadot-sdk/pull/2177Configuration::set_node_feature
core_index
Extensive testing will be conducted - both automated and manual. This proposal doesn't affect security or privacy.
This is a necessary data availability optimisation, as reed-solomon erasure coding has proven to be a top consumer of CPU time in polkadot as we scale up the parachain block size and number of availability cores.
This is a breaking change. See upgrade path section above. All validators and collators need to have upgraded their node versions before the feature will be enabled via a governance call.
See comments on the tracking issue and the in-progress PR
Not applicable.
This enables future optimisations for the performance of availability recovery, such as retrieving batched systematic chunks from backers/approval-checkers.
proof
Vec<u8>
Proof
account_id
SessionKeys
Validator operators need to pass the their account id when rotating their session keys in a node. This will require updating some high level docs and making users familiar with the slightly changed ergonomics.
Testing of the new changes only requires passing an appropriate owner for the current testing context. The changes to the proof generation and verification got audited to ensure they are correct.
owner
The session key generation is an offchain process and thus, doesn't influence the performance of the chain. Verifying the proof is done on chain as part of the transaction logic for setting the session keys. @@ -4719,11 +4627,11 @@ a runtime is enacted that contains these changes otherwise they will fail to gen The RPC that exists around this runtime api needs to be updated to support passing the account id and for returning the ownership proof alongside the public session keys.
UIs would need to be updated to support the new RPC and the changed on chain logic.
Substrate implementation of the RFC.
Updates to these levels, whether relative ratios, the asset used, or the amount, shall be done via RFC.
By not using DOT for payment, the protocol relies on the stability of other assets and the ability to acquire them. However, the asset of choice can be changed in the future.
N/A.
N/A
4
Vec<Transaction>
for
Vec
As explained in the motivation section, this allows extracting scale(transaction) items without having to know how to decode them.
scale(transaction)
By "flattening" the two-steps hierarchy, an implementation only needs to back-pressure individual notifications rather than back-pressure notifications and transactions within notifications.
This RFC chooses to maintain backwards compatibility at the cost of introducing a very small wart (the Compact(1)).
Compact(1)
An alternative could be to introduce a new version of the transactions notifications protocol that sends one Transaction per notification, but this is significantly more complicated to implement and can always be done later in case the Compact(1) is bothersome.
Transaction
The change is backwards compatible if done in two steps: modify the sender to always send one transaction per notification, then, after a while, modify the receiver to enforce the new format.
None. This is a simple isolated change.
Implementations that have the head of the chain provider capability do not register themselves as providers, but instead are the nodes that participate in the main DHT. In other words, they are the nodes that serve requests of the /<genesis_hash>/kad protocol.
/<genesis_hash>/kad
Any implementation that isn't a head of the chain provider (read: light clients) must not participate in the main DHT. This is already presently the case.
Implementations must not participate in the main DHT if they don't fulfill the capability yet. For example, a node that is still in the process of warp syncing must not participate in the main DHT. However, assuming that warp syncing doesn't last more than a few seconds, it is acceptable to ignore this requirement in order to avoid complicating implementations too much.
None that I can see.
The content of this section is basically the same as the one in RFC 8.
For this reason, an attacker can abuse this mechanism by randomly generating libp2p PeerIds until they find the 20 entries closest to the key representing the target capability. They are then in control of the list of nodes with that capability. While doing this can in no way be actually harmful, it could lead to eclipse attacks.
Because the key changes periodically and isn't predictable, and assuming that the Polkadot DHT is sufficiently large, it is not realistic for an attack like this to be maintained in the long term.
Unknown.
This RFC would make it possible to reliably discover archive nodes, which would make it possible to reliably send archive node requests, something that isn't currently possible. This could solve the problem of finding archive RPC node providers by migrating archive-related request to using the native peer-to-peer protocol rather than JSON-RPC.
If we ever decide to break backwards compatibility, we could divide the "history" and "archive" capabilities in two, between nodes capable of serving older blocks and nodes capable of serving newer blocks. We could even add to the peer-to-peer network nodes that are only capable of serving older blocks (by reading from a database) but do not participate in the head of the chain, and that just exist for historical purposes.
u8
Option<[u8; 32]>
Some(metadata_hash)
The chunking may not be the optimal case for every kind of offline wallet.
All implementations are required to strictly follow the RFC to generate the metadata hash. This includes which hash function to use and how to construct the metadata types tree. So, all implementations are following the same security criteria. As the chains will calculate the metadata hash at compile time, the build process needs to be trusted. However, this is already a solved problem in the Polkadot ecosystem by using reproducible builds. So, anyone can rebuild a chain runtime to ensure that a proposal is actually containing the changes as advertised.
Implementations can also be tested easily against each other by taking some metadata and ensuring that they all come to the same metadata hash.
Privacy of users should also not be impacted. This assumes that wallets will generate the metadata hash locally and don't leak any information to third party services about which chunks a user will send to their offline wallet. Besides that, there is no leak of private information as getting the raw metadata from the chain is an operation that is done by almost everyone.
There should be no measurable impact on performance to Polkadot or any other chain using this feature. The metadata root hash is calculated at compile time and at runtime it is optionally used when checking the signature of a transaction. This means that at runtime no performance heavy operations are done.
The proposal alters the way a transaction is built, signed, and verified. So, this imposes some required changes to any kind of developer who wants to construct transactions for Polkadot or any chain using this feature. As the developer can pass 0 for disabling the verification of the metadata root hash, it can be easily ignored.
RFC 46 produced by the Alzymologist team is a previous work reference that goes in this direction as well.
On other ecosystems, there are other solutions to the problem of trusted signing. Cosmos for example has a standardized way of transforming a transaction into some textual representation and this textual representation is included in the signed data. Basically achieving the same as what the RFC proposes, but it requires that for every transaction applied in a block, every node in the network always has to generate this textual representation to ensure the transaction signature is valid.
This change would reduce the maximum possible transaction version from the current 127 to 63. In order to bypass the new, lower limit, the extrinsic format would have to change again.
127
63
There is no impact on testing, security or privacy.
This change would allow Polkadot to support new types of transactions, with the specific "general" transaction type in mind at the time of writing this proposal.
There is no performance impact.
The impact to developers and end-users is minimal as it would just be a bitmask update on their part for parsing the extrinsic type along with the version.
This change breaks backwards compatiblity because any transaction that is neither signed nor unsigned, but a new transaction type, would be interpreted as having a future extrinsic format version.
The original design was originally proposed in the TransactionExtension PR, which is also the motivation behind this effort.
TransactionExtension
Following this change, the "general" transaction type will be introduced as part of the Extrinsic Horizon effort, which will shape future work.
In theory the new protocol creates a bit more traffic on the DHT network, because it waits for DHT records to be received from more than one node, while in the current implementation we just take the first record that we receive and cancel all in-flight requests to other peers. However, because the redundancy factor will be relatively small and this operation happens rarerly, every 10min, this cost is negligible.
This RFC's implementation https://github.com/paritytech/polkadot-sdk/pull/3786 had been tested on various local test networks and versi.
With regard to security the creation time is wrapped inside SignedAuthorityRecord wo it will be signed with the authority id key, so there is no way for other malicious nodes to manipulate this field without the received node observing.
The changes are backwards compatible with the existing protocol, so nodes with both the old protocol and newer protocol can exist in the network, this is achieved by the fact that we use protobuf for serializing and deserializing the records, so new fields will be ignore when deserializing with the older protocol and vice-versa when deserializing an old record with the new protocol the new field will be None and the new code accepts this record as being valid.
The enhancements have been inspired by the algorithm specified in here
In addition to a simple queue, we could add a market component that lets users always unbond from staking at the minimum possible waiting time)(== LOWER_BOUND, e.g., 2 days), by paying a variable fee. To achieve this, it is reasonable to split the total unbonding capacity into two chunks, with the first capacity for the simple queue and the remaining capacity for the fee-based unbonding. By doing so, we allow users to choose whether they want the quickest unbond and paying a dynamic fee or join the simple queue. Setting a capacity restriction for both queues enables us to guarantee a predictable unbonding time in the simple queue, while allowing users with the respective willingness to pay to get out even earlier. The fees are dynamically adjusted and are proportional to the unbonding stake (and thereby expressed in a percentage of the requested unbonding stake). In contrast to a unified queue, this prevents the issue that users paying a fee jump in front of other users not paying a fee, pushing their unbonding time back (which would be bad for UX). The revenue generated could be burned.
LOWER_BOUND
This extension and further specifications are left out of this RFC, because it adds further complexity and the empirical analysis above suggests that average unbonding times will already be close the LOWER_BOUND, making a more complex design unnecessary. We advise to first implement the discussed mechanism and assess after some experience whether an extension is desirable.
UPPER_BOUND
NA
The authors cannot see any potential impact on performance.
The authors cannot see any potential impact on compatibility. This should be assessed by the technical fellows.
6
5
The Version being a SCALE encoded u8 representing the version of the transaction extensions.
Version
In the chain runtime the version can be used to determine which set of transaction extensions should be used to decode and to validate the transaction.
This adds one byte more to each signed transaction.
This will ensure that changes to the transactions extensions can be done in a backwards compatible way.
When introduced together with extrinsic format version 5 from RFC84, it can be implemented in a backwards compatible way. So, transactions can still be send using the old extrinsic format and decoded by the runtime.
No drawbacks identified.
There should be no security risks related to the new instruction from the XCVM perspective. It follows the same @@ -6096,7 +6004,7 @@ which minimizes the potential free/unpaid work that a receiving chain has to do. required execution fee payment, part of the instruction logic through the remote_fees: Option<AssetTransferFilter> parameter, which will make sure the remote XCM starts with a single-asset-holding-loading-instruction, immediately followed by a BuyExecution using said asset.
remote_fees: Option<AssetTransferFilter>
BuyExecution
This brings no impact to the rest of the XCM spec. It is a new, independent instruction, no changes to existing instructions.
Enhances the exposed functionality of Polkadot. Will allow multi-chain transfers that are currently forced to happen in multiple programs per asset per "hop", to be possible in a single XCM program.
The XCVM implementation shall no longer use require_weight_at_most for weighing. Instead, it shall weigh the Transact instruction by decoding and weighing the inner call.
require_weight_at_most
call
No drawbacks, existing scenarios work as before, while this also allows new/easier flows.
Currently, an XCVM implementation can weigh a message just by looking at the decoded instructions without decoding the Transact's call, but assuming require_weight_at_most weight for it. With the new version it has to decode the inner call to know its actual weight.
The security considerations around how much can someone execute for free are the same for both this new version and the old. In both cases, an "attacker" can do the XCM decoding (including Transact inner calls) for free by adding a large enough BuyExecution without actually having the funds available.
In both cases, decoding is done for free, but in both cases execution fails early on BuyExecution.
No performance change.
Ergonomics are slightly improved by simplifying Transact API.
Transact
Compatible with previous XCM programs.
The only drawback is that further additions to the descriptor are limited to the amount of remaining unused space.
Any tooling that decodes UMP XCM messages needs an update to support or ignore the new UMP messages, but they should be fine to decode the regular XCM messages that come before the separator.
Forum discussion about a new CandidateReceipt format: https://forum.polkadot.network/t/pre-rfc-discussion-candidate-receipt-format-v2/3738
CandidateReceipt
The implementation is extensible and future-proof to some extent. With minimal or no breaking changes, additional fields can be added in the candidate descriptor until the reserved space is exhausted
There needs to be an explicit change from BuyExecution to PayFees, most often accompanied by a reduction in the assets passed in.
PayFees
It might become a security concern if leftover fees are trapped, since a lot of them are expected.
There should be no performance downsides to this approach. The fees register is a simplification that may actually result in better performance, in the case an implementation is doing a workaround to achieve what this RFC proposes.
fees
This RFC can't just change the semantics of the BuyExecution instruction since that instruction accepts any funds, uses what it needs and returns the rest immediately. The new proposed instruction, PayFees, doesn't return the leftover immediately, it keeps it in the fees register. In practice, the deprecated BuyExecution needs to be slowly rolled out in favour of PayFees.
The closed RFC PR on the xcm-format repository, before XCM RFCs got moved to fellowship RFCs: https://github.com/polkadot-fellows/xcm-format/pull/53.
This proposal would greatly benefit from an improved asset trapping system.
CustomAssetClaimer is also related, as it directly improves the ergonomics of this proposal.
LeftoverAssetsDestination execution hint would also similarly improve the ergonomics.
The SetHints instruction might be hard to benchmark, since we should look into the actual hints being set to know how much weight to attribute to it.
SetHints
Hints are specified on a per-message basis, so they have to be specified at the beginning of a message. If they were to be specified at the end, hints like AssetClaimer would be useless if an error occurs beforehand and assets get trapped before ever reaching the hint.
Hint
AssetClaimer
The instruction takes a bounded vector of hints so as to not force barriers to allow an arbitrary number of SetHint instructions.
SetHint
The previous RFC PR in the xcm-format repository before XCM RFCs moved to fellowship RFCs: https://github.com/polkadot-fellows/xcm-format/pull/59.
NetworkId::ByGenesis
Remove Westend and Rococo from the included NetworkIds in the language.
Westend
Rococo
NetworkId
This RFC will make it less convenient to specify a testnet, but not by a large amount.
It will very slightly reduce the ergonomics of testnet developers but improve the stability of the language.
NetworkId::Rococo and NetworkId::Westend can just use NetworkId::ByGenesis, as can other testnets.
NetworkId::Rococo
NetworkId::Westend
A previous attempt to add NetworkId::Paseo: https://github.com/polkadot-fellows/xcm-format/pull/58.
NetworkId::Paseo
In terms of ergonomics and user experience, this support for combining an asset transfer with a subsequent action (like Transact) is a net positive.
In terms of performance, and privacy, this is neutral with no changes.
In terms of security, the feature by itself is also neutral because it allows preserve_origin: false usage for operating with no extra trust assumptions. When wanting to support preserving origin, chains need to configure secure origin aliasing filters. The one suggested in this RFC should be the right choice for the majority of chains, but each chain will ultimately choose depending on their business model and logic (e.g. chain does not plan to integrate with Asset Hub). It is up to the individual chains to configure accordingly.
preserve_origin: false
Normally, XCM program builders should audit their programs and eliminate assumptions of "no origin" on remote side of this instruction. In this case, the InitiateAssetsTransfer has not been released yet, it will be part of XCMv5, and we can make this change part of the same XCMv5 so that there isn't even the possibility of someone in the wild having built XCM programs using this instruction on those wrong assumptions.
InitiateAssetsTransfer
The working assumption going forward is that the origin on the remote side can either be cleared or it can be the local origin's reanchored location. This assumption is in line with the current behavior of remote XCM programs sent over using pallet_xcm::send.
pallet_xcm::send
The existing DepositReserveAsset, InitiateReserveWithdraw and InitiateTeleport cross chain asset transfer instructions will not attempt to do origin aliasing and will always clear origin same as before for compatibility reasons.
DepositReserveAsset
InitiateReserveWithdraw
InitiateTeleport
No impact.
For compatibility reasons, this RFC proposes this mechanism be added as an enhancement to the yet unreleased InitiateAssetsTransfer instruction, thus eliminating possibilities of XCM logic breakages in the wild. Following the same logic, the existing DepositReserveAsset, InitiateReserveWithdraw and InitiateTeleport cross chain asset transfer instructions will not attempt to do origin aliasing and will always clear the origin same as before for compatibility reasons.
Any one of DepositReserveAsset, InitiateReserveWithdraw and InitiateTeleport instructions can be replaced with a InitiateAssetsTransfer instruction with or without origin aliasing, thus providing a clean and clear upgrade path for opting-in this new feature.
AssetId
Asset { id: <AssetId>, fun: NonFungible(AssetInstance::Undefined) }
As a singleton non-fungible instance is barely distinguishable from its collection, this convention shouldn't cause any problems.
Thus, the AssetInstance docs must be updated accordingly in the implementations.
AssetInstance
Regarding ergonomics, no drawbacks were noticed.
As for the user experience, it could discover new cross-chain use cases involving asset collections and NFTs, indicating a positive impact.
There are no security concerns except for the ReportMetadata instruction, which implies that the source of the information must be trusted.
ReportMetadata
In terms of performance and privacy, there will be no changes.
The implementations must honor the contract for the new instructions. Namely, if the instance field has the value of AssetInstance::Undefined, the metadata must relate to the asset collection but not to a non-fungible token inside it.
instance
AssetInstance::Undefined
No significant impact.
Introducing a standard metadata format and a way of communicating it is a valuable addition to the XCM format that potentially increases cross-chain interoperability without the need to form ad-hoc chain-to-chain integrations via Transact.
This RFC proposes new functionality, so there are no compatibility issues.
RFC: XCM Asset Metadata
The original RFC draft contained additional metadata instructions. Though they could be useful, they're clearly outside the basic logic. So, this RFC version omits them to make the metadata discussion more focused on the core things. Nonetheless, there is hope that metadata approval instructions might be useful in the future, so they are mentioned here.
You can read about the details in the original draft.
We want all polkadot subsystems be profitable for validataors, because otherwise operators might profit from running modified code. In particular, almost all rewards in Kusama/Polkadot should come from work done securing parachains, primarily approval checking, but also backing, availability, and support of XCMP.
/// Our subjective record of out availability transfers for this candidate. -CandidateRewards { - /// Anyone who backed this parablock - backers: [AuthorityId; NumBackers], - /// Anyone who sent us chunks for this candidate - downloaded_from: HashMap<AuthorityId,u16>, - /// Anyone to whome we sent chunks for this candidate - uploaded_to: HashMap<AuthorityId,u16>, -} -
/// Our subjective record of what we used from, and provided to, all other validators on the finalized chain -pub struct ApprovalsTally(Vec<ApprovalTallyLine>); - -/// Our subjective record of what we used from, and provided to, all one other validators on the finalized chain -pub struct ApprovalTallyLine { - /// Approvals by this validator which our approvals gadget used in marking candidates approved. - approval_usages: u32, - /// Availability chunks we downloaded from this validator for our approval checks we used. - used_downloads: u32, - /// Availability chunks we uploaded to this validator which whose approval checks we used. - used_uploads: u32, -} -
/// Our subjective record of what we used from some other validator on the finalized chain -pub struct ApprovalTallyMessageLine { - /// Approvals by this validator which our approvals gadget used in marking candidates approved. - approval_usages: u32, - /// Availability chunks we downloaded from this validator for our approval checks we used. - used_downloads: u32, -} - -/// Our subjective record of what we used from all other validators on the finalized chain -pub struct ApprovalsTallyMessage(Vec<ApprovalTallyMessageLine>); -
We compute the approvals rewards by taking the median of the approval_usages fields for each validator across all validators ApprovalsTallyMessages.
let mut approval_usages_medians = Vec::new(); -for i in 0..num_validators { - let mut v: Vec<u32> = approvals_tally_messages.iter().map(|atm| atm.0[i].approval_usages); - v.sort(); - approval_usages_medians.push(v[num_validators/2]); -} -
Assuming more than 50% honersty, these median tell us how many approval votes form each validator
#[cfg(offchain)] -let mut my_missing_uploads = my_approvals_tally.iter().map(|l| l.used_uploads).collect(); -let mut reweighted_total_used_downloads = vec[0u64; num_validators]; -for (mmu,atm) in my_missing_uploads.iter_mut().zip(approvals_tally_messages) { - let d = atm.0.iter().map(|l| l.used_downloads).sum(); - for i in 0..num_validators { - let atm_from_i = approval_usages_medians[i] * (f+1) / d; - #[cfg(offchain)] - if i == me { mmu -= atm_from_i }; - reweighted_total_used_downloads[i] += atm_from_i; - } -} -
After receiving "all" pairs $(\alpha_{u,v},\beta_{u,v})$, validator $w$ re-weights the $\beta_{u,v}$ and their own $\gamma_{w,v}$. -$$ -\begin{aligned} -\beta\prime_{w,v} &= {(f+1) \alpha_v \over \sum_u \beta_{u,v}} \beta_{w,v} \ -\gamma\prime_{w,v} &= {(f+1) \alpha_w \over \sum_v \gamma_{w,v}} \gamma_{w,v} \ -\end{aligned} -$$ -At this point, we compute $\beta\prime_w = \sum_v \beta\prime_{w,v}$ on-chain for each $w$ and reward $w$ proportionally.
(func $ext_storage_read_version_2 + (param $key i64) (param $value_out i64) (param $offset i32) (result i64)) +(func $ext_default_child_storage_read_version_2 + (param $child_storage_key i64) (param $key i64) (param $value_out i64) + (param $offset i32) (result i64)) +
The signature and behaviour of ext_storage_read_version_2 and ext_default_child_storage_read_version_2 is identical to their version 1 counterparts, but the return value has a different meaning. +The new functions directly return the number of bytes that were written in the value_out buffer. If the entry doesn't exist, a value of -1 is returned. Given that the host must never write more bytes than the size of the buffer in value_out, and that the size of this buffer is expressed as a 32 bits number, a 64bits value of -1 is not ambiguous.
(func $ext_storage_next_key_version_2 + (param $key i64) (param $out i64) (return i32)) +(func $ext_default_child_storage_next_key_version_2 + (param $child_storage_key i64) (param $key i64) (param $out i64) (return i32)) +
The behaviour of these functions is identical to their version 1 counterparts. +Instead of allocating a buffer, writing the next key to it, and returning a pointer to it, the new version of these functions accepts an out parameter containing a pointer-size to the memory location where the host writes the output. The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine, even if the function wouldn't write anything to out. +These functions return the size, in bytes, of the next key, or 0 if there is no next key. If the size of the next key is larger than the buffer in out, the bytes of the key that fit the buffer are written to out and any extra byte that doesn't fit is discarded.
(func $ext_hashing_keccak_256_version_2 + (param $data i64) (param $out i32)) +(func $ext_hashing_keccak_512_version_2 + (param $data i64) (param $out i32)) +(func $ext_hashing_sha2_256_version_2 + (param $data i64) (param $out i32)) +(func $ext_hashing_blake2_128_version_2 + (param $data i64) (param $out i32)) +(func $ext_hashing_blake2_256_version_2 + (param $data i64) (param $out i32)) +(func $ext_hashing_twox_64_version_2 + (param $data i64) (param $out i32)) +(func $ext_hashing_twox_128_version_2 + (param $data i64) (param $out i32)) +(func $ext_hashing_twox_256_version_2 + (param $data i64) (param $out i32)) +(func $ext_trie_blake2_256_root_version_3 + (param $data i64) (param $version i32) (param $out i32)) +(func $ext_trie_blake2_256_ordered_root_version_3 + (param $data i64) (param $version i32) (param $out i32)) +(func $ext_trie_keccak_256_root_version_3 + (param $data i64) (param $version i32) (param $out i32)) +(func $ext_trie_keccak_256_ordered_root_version_3 + (param $data i64) (param $version i32) (param $out i32)) +(func $ext_default_child_storage_root_version_3 + (param $child_storage_key i64) (param $out i32)) +(func $ext_crypto_ed25519_generate_version_2 + (param $key_type_id i32) (param $seed i64) (param $out i32)) +(func $ext_crypto_sr25519_generate_version_2 + (param $key_type_id i32) (param $seed i64) (param $out i32) (return i32)) +(func $ext_crypto_ecdsa_generate_version_2 + (param $key_type_id i32) (param $seed i64) (param $out i32) (return i32)) +
(func $ext_default_child_storage_root_version_3 + (param $child_storage_key i64) (param $out i32)) +(func $ext_storage_root_version_3 + (param $out i32)) +
(func $ext_storage_clear_prefix_version_3 + (param $prefix i64) (param $limit i64) (param $removed_count_out i32) + (return i32)) +(func $ext_default_child_storage_clear_prefix_version_3 + (param $child_storage_key i64) (param $prefix i64) + (param $limit i64) (param $removed_count_out i32) (return i32)) +(func $ext_default_child_storage_kill_version_4 + (param $child_storage_key i64) (param $limit i64) + (param $removed_count_out i32) (return i32)) +
(func $ext_crypto_ed25519_sign_version_2 + (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i32)) +(func $ext_crypto_sr25519_sign_version_2 + (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i32)) +func $ext_crypto_ecdsa_sign_version_2 + (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i32)) +(func $ext_crypto_ecdsa_sign_prehashed_version_2 + (param $key_type_id i32) (param $key i32) (param $msg i64) (param $out i32) (return i64)) +
(func $ext_crypto_secp256k1_ecdsa_recover_version_3 + (param $sig i32) (param $msg i32) (param $out i32) (return i64)) +(func $ext_crypto_secp256k1_ecdsa_recover_compressed_version_3 + (param $sig i32) (param $msg i32) (param $out i32) (return i64)) +
(func $ext_crypto_ed25519_num_public_keys_version_1 + (param $key_type_id i32) (return i32)) +(func $ext_crypto_ed25519_public_key_version_2 + (param $key_type_id i32) (param $key_index i32) (param $out i32)) +(func $ext_crypto_sr25519_num_public_keys_version_1 + (param $key_type_id i32) (return i32)) +(func $ext_crypto_sr25519_public_key_version_2 + (param $key_type_id i32) (param $key_index i32) (param $out i32)) +(func $ext_crypto_ecdsa_num_public_keys_version_1 + (param $key_type_id i32) (return i32)) +(func $ext_crypto_ecdsa_public_key_version_2 + (param $key_type_id i32) (param $key_index i32) (param $out i32)) +
Instead of calling ext_crypto_ed25519_public_key_version_1 in order to obtain the list of all keys at once, the runtime should instead call ext_crypto_ed25519_num_public_keys_version_1 in order to obtain the number of public keys available, then ext_crypto_ed25519_public_key_version_2 repeatedly. +The ext_crypto_ed25519_public_key_version_2 function writes the public key of the given key_index to the memory location designated by out. The key_index must be between 0 (included) and n (excluded), where n is the value returned by ext_crypto_ed25519_num_public_keys_version_1. Execution must trap if n is out of range.
(func $ext_offchain_http_request_start_version_2 + (param $method i64) (param $uri i64) (param $meta i64) (result i32)) +
(func $ext_offchain_http_request_write_body_version_2 + (param $method i64) (param $uri i64) (param $meta i64) (result i32)) +(func $ext_offchain_http_response_read_body_version_2 + (param $request_id i32) (param $buffer i64) (param $deadline i64) (result i64)) +
(func $ext_offchain_http_response_wait_version_2 + (param $ids i64) (param $deadline i64) (param $out i32)) +
(func $ext_offchain_http_response_header_name_version_1 + (param $request_id i32) (param $header_index i32) (param $out i64) (result i64)) +(func $ext_offchain_http_response_header_value_version_1 + (param $request_id i32) (param $header_index i32) (param $out i64) (result i64)) +
(func $ext_offchain_submit_transaction_version_2 + (param $data i64) (return i32)) +(func $ext_offchain_http_request_add_header_version_2 + (param $request_id i32) (param $name i64) (param $value i64) (result i32)) +
(func $ext_offchain_local_storage_read_version_1 + (param $kind i32) (param $key i64) (param $value_out i64) (param $offset i32) (result i64)) +
(func $ext_offchain_network_peer_id_version_1 + (param $out i64)) +
This function writes the PeerId of the local node to the memory location indicated by out. A PeerId is always 38 bytes long. +The runtime execution stops with an error if out is outside of the range of the memory of the virtual machine.
(func $ext_input_size_version_1 + (return i64)) +(func $ext_input_read_version_1 + (param $offset i64) (param $out i64)) +
The ext_input_read_version_1 host function copies some data from the input data to the memory of the runtime. The offset parameter indicates the offset within the input data where to start copying, and must be inferior or equal to the value returned by ext_input_size_version_1. The out parameter is a pointer-size containing the buffer where to write to. +The runtime execution stops with an error if offset is strictly superior to the size of the input data, or if out is outside of the range of the memory of the virtual machine, even if the amount of data to copy would be 0 bytes.
All the host functions that are being superceded by new host functions are now considered deprecated and should no longer be used. +The following other host functions are similarly also considered deprecated:
After this RFC, we can remove from the source code of the host the allocator altogether in a future version, by removing support for all the deprecated host functions. +This would remove the possibility to synchronize older blocks, which is probably controversial and requires a some preparations that are out of scope of this RFC.
The new generic fees mechanism is related to this proposal and further stimulates it as the JIT withdraw fees mechanism will become useless anyway.
#![allow(unused)] +fn main() { +// Simplified version to illustrate the concept. +mod pallet_proxy_replica { + /// The part of the pure account preimage that has to be provided by a client. + struct Witness { + /// Pure proxy swapner + spawner: AccountId, + /// Disambiguation index + index: u16, + /// The block height and extrinsic index of when the pure account was created. + block_number: BlockNumber, + /// The extrinsic index. + ext_index: u32, + // Part of the preimage, but constant. + // proxy_type: ProxyType::Any, + } + // ... + + /// The replication call to be initiated on the source chain. + // Simplified version, the XCM part will be abstracted by the `Config` trait. + fn replicate(origin: SignedOrigin, witness: Witness, proxy: xcm::Location) -> ... { + let pure = ensure_signed(origin); + ensure!(pure == proxy_pallet::derive_pure_account(witness), Error::NotPureAccount); + let xcm = vec![ + DescendOrigin(who), + Transact( + // … + origin_kind: OriginKind::Xcm, + call: pallet_proxy_replica::create(witness, proxy).encode(), + ) + ]; + xcmTransport::send(xcm)?; + } + // … + + /// The call initiated by the source chain on the receiving chain. + // `Config::CreateOrigin` - generally open for whitelisted parachain IDs and + // converts `Origin::Xcm(ParachainA/AccountId32(pure1))` to `AccountID(pure1)`. + fn create(origin: Config::CreateOrigin, witness: Witness, proxy: xcm::Location) -> ... { + let pure = T::CreateOrigin::ensure_origin(origin); + ensure!(pure == proxy_pallet::derive_pure_account(witness), Error::NotPureAccount); + proxy_pallet::create_pure_proxy(pure, proxy); + } +} + +}
This RFC proposes compressing the state response message during the state syncing process to reduce the amount of data transferred.
State syncing can require downloading several gigabytes of data, particularly for blockchains with large state sizes, such as Astar, which has a state size exceeding 5 GiB (https://github.com/AstarNetwork/Astar/issues/1110). This presents a significant challenge for nodes with slower network connections. Additionally, the current state sync implementation lacks a persistence feature (https://github.com/paritytech/polkadot-sdk/issues/4), meaning any network disruption forces the node to re-download the entire state, making the process even more difficult.
This RFC benefits all projects utilizing the Substrate framework, specifically in improving the efficiency of state syncing.
The largest portion of the state response message consists of either CompactProof or Vec<KeyValueStateEntry>, depending on whether a proof is requested (source):
CompactProof
Vec<KeyValueStateEntry>
None identified.
The code changes required for this RFC are straightforward: compress the state response on the sender side and decompress it on the receiver side. Existing sync tests should ensure functionality remains intact.
This RFC optimizes network bandwidth usage during state syncing, particularly for blockchains with gigabyte-sized states, while introducing negligible CPU overhead for compression and decompression. For example, compressing the state response during a recent Polkadot warp sync (around height #22076653) reduces the data transferred from 530,310,121 bytes to 352,583,455 bytes — a 33% reduction, saving approximately 169 MiB of data.
Performance data is based on this patch, with logs available here.
No compatibility issues identified.
This RFC proposes a new host function, secp256r1_ecdsa_verify_prehashed, for verifying NIST-P256 signatures. The function takes as input the message hash, r and s components of the signature, and the x and y coordinates of the public key. By providing this function, runtime authors can leverage a more efficient verification mechanism for "secp256r1" elliptic curve signatures, reducing computational costs and improving overall performance.
secp256r1_ecdsa_verify_prehashed
NIST-P256
r
s
y
“secp256r1” elliptic curve is a standardized curve by NIST which has the same calculations by different input parameters with “secp256k1” elliptic curve. The cost of combined attacks and the security conditions are almost the same for both curves. Adding a host function can provide signature verifications using the “secp256r1” elliptic curve in the runtime and multi-faceted benefits can occur. One important factor is that this curve is widely used and supported in many modern devices such as Apple’s Secure Enclave, Webauthn, Android Keychain which proves the user adoption. Additionally, the introduction of this host function could enable valuable features in the account abstraction which allows more efficient and flexible management of accounts by transaction signs in mobile devices. Most of the modern devices and applications rely on the “secp256r1” elliptic curve. The addition of this host function enables a more efficient verification of device native transaction signing mechanisms. For example:
This RFC proposes a new host function for runtime authors to leverage a more efficient verification mechanism for "secp256r1" elliptic curve signatures.
Proposed host function signature:
#![allow(unused)] @@ -9922,19 +10015,19 @@ Most of the modern devices and applications rely on the “secp256r1” elliptic ) -> bool; }
The host function MUST return true if the signature is valid or false otherwise.
true
false
The changes are not directly affecting the protocol security, parachains are not enforced to use the host function.
The host function proposed in this RFC allows parachain runtime developers to use a more efficient verification mechanism for "secp256r1" elliptic curve signatures.
A followup of the RFC-0014. This RFC proposes adding a new collective to the Polkadot Collectives Chain: The Unbrick Collective, as well as improvements in the mechanisms that will allow teams operating paras that had stopped producing blocks to be assisted, in order to restore the production of blocks of these paras.
Since the initial launch of Polkadot parachains, there has been many incidients causing parachains to stop producing new blocks (therefore, being bricked) and many occurrences that required Polkadot governance to update the parachain head state/wasm. This can be due to many reasons range @@ -9994,14 +10087,14 @@ damage to the parachain and users.
In consequence, the idea of a Unbrick Collective that can provide assistance to para teams when they brick and further protection against future halts is reasonable enough.
The Unbrick Collective is defined as an unranked collective of members, not paid by the Polkadot Treasury. Its main goal is to serve as a point of contact and assistance for enacting the actions @@ -10069,31 +10162,31 @@ of the new PVF being set). Therefore, they must have the technical capacity to p
The ability to modify the Head State and/or the PVF of a para means a possibility to perform arbitrary modifications of it (i.e. take control the native parachain token or any bridged assets in the para).
This could introduce a new attack vector, and therefore, such great power needs to be handled carefully.
The implementation of this RFC will be tested on testnets (Rococo and Westend) first.
An audit will be required to ensure the implementation doesn't introduce unwanted side effects.
There are no privacy related concerns.
This RFC should not introduce any performance impact.
This RFC should improve the experience for new and existing parachain teams, lowering the barrier to unbrick a stalled para.
This RFC is fully compatible with existing interfaces.
WhitelistedUnbrickCaller
Unbrick
In an attempt to mitigate risks derived from unwanted behaviours around long decision periods on referenda, this proposal describes how to finalize and decide a result of a poll via a mechanism similar to candle auctions.
Referenda protocol provide permissionless and efficient mechanisms to enable governance actors to decide the future of the blockchains around Polkadot network. However, they pose a series of risks derived from the game theory perspective around these mechanisms. One of them being where an actor @@ -10166,7 +10259,7 @@ on a poll as early as possible. This proposal's approach suggests using a Candle be determined right after the confirm period finishes, thus decreasing the chances of actors to alter the results of a poll on confirming state, and instead incentivizing them to cast their votes earlier, on deciding state.
Finalizing
Currently, the process of a referendum/poll is defined as an sequence between an ongoing state (where accounts can vote), comprised by a with a preparation period, a decision period, and a confirm period. If the poll is passing before the decision period ends, it's possible to push @@ -10257,43 +10350,43 @@ candle was "lit-off".
This approach doesn't include a mechanism to determine whether a change of the poll status in the confirming period is due to a legitimate change of mind of the voters, or an exploitation of its aforementioned vulnerabilities (like a sniping attack), instead treating all of them as potential attacks.
This is an issue that can be addressed by additional mechanisms, and heuristics that can help determine the probability of a change of poll status to happen as a result of a legitimate behaviour.
The implementation of this RFC will be tested on testnets (Paseo and Westend) first. Furthermore, it should be enabled in a canary network (like Kusama) to ensure the behaviours it is trying to address is indeed avoided.
The added steps imply pessimization, necessary to meet the expected changes. An implementation MUST exit from the Finalization period as early as possible to minimize this impact.
This proposal does not alter the already exposed interfaces or developers or end users. However, they must be aware of the changes in the additional overhead the new period might incur (these depend on the implemented VRF).
This proposal does not break compatibility with existing interfaces, older versions, but it alters the previous implementation of the referendum processing algorithm.
An acceptable upgrade strategy that can be applied is defining a point in time (block number, poll index) from which to start applying the new mechanism, thus, not affecting the already ongoing referenda.
polkadot-runtime-commont
A proposed implementation of this change can be seen on this Pull Request.
The protocol change introduces flexibility in the governance structure by enabling the referenda track list to be modified dynamically at runtime. This is achieved by replacing static slices in TracksInfo with iterators, facilitating storage-based track management. As a result, governance tracks can be modified or added based on real-time decisions and without requiring runtime upgrades.
TracksInfo
Polkadot's governance system is designed to be adaptive and decentralized, but modifying the referenda tracks (which determine decision-making paths for proposals) has historically required runtime upgrades. This poses an operational challenge, delaying governance changes until an upgrade @@ -10339,12 +10432,12 @@ is scheduled and executed. The new system provides the flexibility needed to adj dynamically, reflecting real-time changes in governance needs without the latency and risks associated with runtime upgrades. This reduces governance bottlenecks and allows for quicker adaptation to emergent scenarios.
The protocol modification replaces the current static slice method used for storing referenda tracks with an iterator-based approach that allows tracks to be managed dynamically using chain storage. Governance participants can define and modify referenda tracks as needed, which are then accessed @@ -10356,12 +10449,12 @@ rather enhances them by increasing adaptability.
The most significant drawback is the increased complexity for developers managing track configurations via storage-based iterators, which require careful handling to avoid misuse or inefficiencies.
Additionally, this flexibility could introduce risks if track configurations are modified improperly during runtime, potentially leading to governance instabilities.
To ensure security, the change must be tested in testnet environments first (Paseo, Westend), particularly in scenarios where multiple track changes happen concurrently. Potential vulnerabilities in governance adjustments must be addressed to prevent abuse.
Comprehensive tests should be conducted to validate correct track modifications in different governance scenarios.
The proposal optimizes governance track management by avoiding the overhead of runtime upgrades, reducing downtime, and eliminating the need for full consensus on upgrades. However, there is a slight performance cost related to runtime access to storage-based iterators, though this is mitigated by the overall system efficiency gains.
Developers and governance actors benefit from simplified governance processes but must account for the technical complexity of managing iterator-based track configurations.
Tools may need to be developed to help streamline track adjustments in runtime.
The change is backward compatible with existing governance operations, and does not require developers to adjust how they interact with referenda tracks.
A migration is required to convert existing statically-defined tracks to dynamic storage-based configurations without disruption.
This dynamic governance track approach builds on previous work around Polkadot's on-chain governance and leverages standard iterator patterns in Rust programming to improve runtime flexibility. Comparable solutions in other governance networks were examined, but this proposal uniquely tailors them to Polkadot’s decentralized, runtime-upgradable architecture.
There are already two proposed solutions for both the implementation and
pallet-referenda
tracks
pallet-referenda-tracks
The runtime code is stored under the special key :code in the state. Nodes and other tooling read the runtime code under this storage key when they want to interact with the runtime for e.g., building/importing blocks or getting the metadata to read the state. To update the runtime code the runtime overwrites the value at :code, and then from the next block on, the new runtime will be loaded. -This RFC proposes to first store the new runtime code under :pending_code in the state for one block. When the next block is being built, the block builder first needs to check if :pending_code is set, and if so, it needs to load the runtime from this storage key. While building the block the runtime will move :pending_code to :code to have the runtime code at the default location. Nodes importing the block will also need to load :pending_code if it exists to ensure that the correct runtime code is used. By doing it this way, the runtime code found at :code in the state of a block will always be able to decode the state. -Furthermore, this RFC proposes to introduce system_version: 3. The system_version was introduced in RFC42. Version 3 would then enable the usage of :pending_code when applying a runtime code upgrade. This way, the feature can be introduced first and enabled later when the majority of the nodes have upgraded.
Because the first block built with the new runtime code will move the runtime code from :pending_code to :code, the runtime code will need to be loaded. This means the runtime code will appear in the proof of validity of a parachain for the first block built with the new runtime code. Generally this is not a problem as the runtime code is also loaded by the parachain when setting the new runtime code. -There is still the possibility of having state that is not migrated even when following the proposal as presented by this RFC. The issue is that if the amount of data to be migrated is too big, not all of it can be migrated in one block, because either it takes more time than there is assigned for a block or parachains for example have a fixed budget for their proof of validity. To solve this issue there already exist multi-block migrations that can chunk the migration across multiple blocks. Consensus-critical data needs to be migrated in the first block to ensure that block production etc., can continue. For the other data being migrated by multi-block migrations the migrations could for example expose to the outside which keys are being migrated and should not be indexed until the migration is finished.
The change will require that the nodes are upgraded before the runtime starts using this feature. Otherwise they will fail to import the block build by :pending_code. -For Polkadot/Kusama this means that also the parachain nodes need to be running with a relay chain node version that supports this new feature. Otherwise the parachains will stop producing/finalizing nodes as they can not sync the relay chain any more.
let mut approval_usages_medians = Vec::new(); +let mut noshows_percentiles = = Vec::new(); for i in 0..num_validators { let mut v: Vec<u32> = approvals_tally_messages.iter().map(|atm| atm.0[i].approval_usages); v.sort(); approval_usages_medians.push(v[num_validators/2]); + let mut v: Vec<u32> = approvals_tally_messages.iter().map(|atm| atm.0[i].noshows); + v.sort(); + noshows_percentiles.push(v[num_validators/3]); }
#[cfg(offchain)] let mut my_missing_uploads = my_approvals_tally.iter().map(|l| l.used_uploads).collect(); @@ -311,6 +321,7 @@ for (mmu,atm) in my_missing_uploads.iter_mut().zip(approvals_tally_messages) { }