diff --git a/404.html b/404.html index 2559b89..00a3876 100644 --- a/404.html +++ b/404.html @@ -91,7 +91,7 @@ diff --git a/approved/0001-agile-coretime.html b/approved/0001-agile-coretime.html index 53d0960..0694994 100644 --- a/approved/0001-agile-coretime.html +++ b/approved/0001-agile-coretime.html @@ -90,7 +90,7 @@ diff --git a/approved/0005-coretime-interface.html b/approved/0005-coretime-interface.html index e7ff33a..454b587 100644 --- a/approved/0005-coretime-interface.html +++ b/approved/0005-coretime-interface.html @@ -90,7 +90,7 @@ diff --git a/approved/0007-system-collator-selection.html b/approved/0007-system-collator-selection.html index 76ead4e..b37c27c 100644 --- a/approved/0007-system-collator-selection.html +++ b/approved/0007-system-collator-selection.html @@ -90,7 +90,7 @@ diff --git a/approved/0008-parachain-bootnodes-dht.html b/approved/0008-parachain-bootnodes-dht.html index ba32935..cc5598e 100644 --- a/approved/0008-parachain-bootnodes-dht.html +++ b/approved/0008-parachain-bootnodes-dht.html @@ -90,7 +90,7 @@ diff --git a/approved/0012-process-for-adding-new-collectives.html b/approved/0012-process-for-adding-new-collectives.html index 7dd3043..2db674a 100644 --- a/approved/0012-process-for-adding-new-collectives.html +++ b/approved/0012-process-for-adding-new-collectives.html @@ -90,7 +90,7 @@ diff --git a/approved/0014-improve-locking-mechanism-for-parachains.html b/approved/0014-improve-locking-mechanism-for-parachains.html index 87f44a8..a3ffa2f 100644 --- a/approved/0014-improve-locking-mechanism-for-parachains.html +++ b/approved/0014-improve-locking-mechanism-for-parachains.html @@ -90,7 +90,7 @@ diff --git a/approved/0022-adopt-encointer-runtime.html b/approved/0022-adopt-encointer-runtime.html index 66fed49..1065380 100644 --- a/approved/0022-adopt-encointer-runtime.html +++ b/approved/0022-adopt-encointer-runtime.html @@ -90,7 +90,7 @@ diff --git a/approved/0032-minimal-relay.html b/approved/0032-minimal-relay.html index 57b4558..81203fd 100644 --- a/approved/0032-minimal-relay.html +++ b/approved/0032-minimal-relay.html @@ -90,7 +90,7 @@ diff --git a/approved/0050-fellowship-salaries.html b/approved/0050-fellowship-salaries.html index b6ec1fe..8598317 100644 --- a/approved/0050-fellowship-salaries.html +++ b/approved/0050-fellowship-salaries.html @@ -90,7 +90,7 @@ diff --git a/approved/0056-one-transaction-per-notification.html b/approved/0056-one-transaction-per-notification.html index d44bbb6..c23e96a 100644 --- a/approved/0056-one-transaction-per-notification.html +++ b/approved/0056-one-transaction-per-notification.html @@ -90,7 +90,7 @@ diff --git a/index.html b/index.html index dd1acaa..b2ab91e 100644 --- a/index.html +++ b/index.html @@ -90,7 +90,7 @@ diff --git a/introduction.html b/introduction.html index dd1acaa..b2ab91e 100644 --- a/introduction.html +++ b/introduction.html @@ -90,7 +90,7 @@ diff --git a/print.html b/print.html index d26bb77..d4292a9 100644 --- a/print.html +++ b/print.html @@ -91,7 +91,7 @@ @@ -1600,9 +1600,6 @@ is the correct way of introducing this change.

  • Merging protocol
  • Complete Binary Merkle Tree construction protocol
  • Digest
  • -
  • Shortening
  • -
  • Transmission
  • -
  • Offline verification
  • Chain verification
  • @@ -1733,11 +1730,11 @@ Node 0 is root
    1. The metadata is converted into lean modular form (vector of chunks)
    2. A Merkle tree is constructed from the metadata chunks
    3. -
    4. A root of tree (as a left element) is merged with Metadata Descriptor (as a right element)
    5. +
    6. A root of tree is merged with the hash of the MetadataDescriptor
    7. Resulting value is a constant to be included in additionalSigned to prove that the metadata seen by cold device is genuine

    Metadata modularization

    -

    Structure of types in shortened metadata exactly matches structure of types in scale-info, but doc field is always empty

    +

    Structure of types in shortened metadata exactly matches structure of types in scale-info at MetadataV14 state, but doc field is always empty

    struct Type {
       path: Path, // vector of strings
       type_params: Vec<TypeParams>,
    @@ -1791,6 +1788,26 @@ modularized_registry.sort(|a, b| {
     
  • Right node and then left node is popped from the front of the nodes queue and merged; the result is sent to the end of the queue.
  • Step (4) is repeated until only one node remains; this is tree root.
  • +
    queue = empty_queue
    +
    +while (leaves.length>1) {
    +  right = leaves.pop_last
    +  left = leaves.pop_last
    +  queue.push_back(merge(left, right))
    +}
    +
    +if leaves.length == 1 {
    +  queue.push_front(leaves.last)
    +}
    +
    +while queue.len() > 1 {
    +  right = queue.pop_front
    +  left = queue.pop_front
    +  queue.push_back(merge(left, right))
    +}
    +
    +return queue.pop
    +
    Resulting tree for metadata consisting of 5 nodes (numbered from 0 to 4):
     
            root
    @@ -1808,14 +1825,8 @@ modularized_registry.sort(|a, b| {
     
  • Root hash of this tree (left) is merged with metadata descriptor blake3 hash (right); this is metadata digest.
  • Version number and corresponding resulting metadata digest MUST be included into Signed Extensions as specified in Chain Verification section below.

    -

    Shortening

    -

    For shortening, an attempt to decode transaction completely using provided metadata is performed with the same algorithm that would be used on the cold side. All chunks are associated with their leaf indices. An example of this protocol is proposed in metadata-shortener that is based on substrate-parser decoding protocol; any decoding protocol could be used here as long as cold signer's design finds it appropriate for given security model.

    -

    Transmission

    -

    Shortened metadata chunks MAY be trasmitted into cold device together with Merkle proof in its entirety or in parts, depending on memory capabilities of the cold device and it ability to reconstruct larger fraction of tree. This document does not specify the manner of transmission. The order of metadata chunks MAY be arbitrary, the only requirement is that indices of leaf nodes in Merkle tree corresponding to chunks MUST be communicated. Community MAY handle proof format standartization independently.

    -

    Offline verification

    -

    The transmitted metadata chunks are hashed together with proof lemmas to obtain root that MAY be transmitted along with the rest of payload. Verification that the root transmitted with message matches with calculated root is optional; the transmitted root SHOULD NOT be used in signature, calculated root MUST be used; however, there is no mechanism to enforce this - it should be done during cold signers code audit.

    Chain verification

    -

    The root of metadata computed by cold device MAY be included into Signed Extensions; this way the transaction will pass as valid iff hash of metadata as seen by cold storage device is identical to consensus hash of metadata, ensuring fair signing protocol.

    +

    The root of metadata computed by cold device MAY be included into Signed Extensions; if it is included, the transaction will pass as valid iff hash of metadata as seen by cold storage device is identical to consensus hash of metadata, ensuring fair signing protocol.

    The Signed Extension representing metadata digest is a single byte representing both digest vaule inclusion and shortening protocol version; this MUST be included in Signed Extensions set. Depending on its value, a digest value is included as additionalSigned to signature computation according to following specification:

    @@ -2131,6 +2142,119 @@ This information would be used in order to query the runtime API and retrieve th occupying. However, considering it's part of an unimported fork, the validator cannot call a runtime API on that block.

    Adding the core_index to the CandidateReceipt would solve this problem and would enable systematic recovery for all dispute scenarios.

    +

    (source)

    +

    Table of Contents

    + +

    RFC-0059: Add a discovery mechanism for nodes based on their capabilities

    +
    signed extension valuedigest valuecomment
    0x00digest is not included
    + + + +
    Start Date2023-12-18
    DescriptionNodes having certain capabilities register themselves in the DHT to be discoverable
    AuthorsPierre Krieger
    +
    +

    Summary

    +

    This RFC proposes to make the mechanism of RFC #8 more generic by introducing the concept of "capabilities".

    +

    Implementations can implement certain "capabilities", such as serving old block headers or being a parachain bootnode.

    +

    The discovery mechanism of RFC #8 is extended to be able to discover nodes of specific capabilities.

    +

    Motivation

    +

    The Polkadot peer-to-peer network is made of nodes. Not all these nodes are equal. Some nodes store only the headers of recently blocks, some nodes store all the block headers and bodies since the genesis, some nodes store the storage of all blocks since the genesis, and so on.

    +

    It is currently not possible to know ahead of time (without connecting to it and asking) which nodes have which data available, and it is not easily possible to build a list of nodes that have a specific piece of data available.

    +

    If you want to download for example the header of block 500, you have to connect to a randomly-chosen node, ask it for block 500, and if it says that it doesn't have the block, disconnect and try another randomly-chosen node. +In certain situations such as downloading the storage of old blocks, nodes that have the information are relatively rare, and finding through trial and error a node that has the data can take a long time.

    +

    This RFC attempts to solve this problem by giving the possibility to build a list of nodes that are capable of serving specific data.

    +

    Stakeholders

    +

    Low-level client developers. +People interested in accessing the archive of the chain.

    +

    Explanation

    +

    Reading RFC #8 first might help with comprehension, as this RFC is very similar.

    +

    Please keep in mind while reading that everything below applies for both relay chains and parachains, except mentioned otherwise.

    +

    Capabilities

    +

    This RFC defines a list of so-called capabilities:

    +
      +
    • Head of chain provider. An implementation with this capability must be able to serve to other nodes block headers, block bodies, justifications, calls proofs, and storage proofs of "recent" (see below) blocks, and, for relay chains, to serve to other nodes warp sync proofs where the starting block is a session change block and must participate in Grandpa and Beefy gossip.
    • +
    • History provider. An implementation with this capability must be able to serve to other nodes block headers and block bodies of any block since the genesis, and must be able to serve to other nodes justifications of any session change block since the genesis up until and including their currently finalized block.
    • +
    • Archive provider. This capability is a superset of History provider. In addition to the requirements of History provider, an implementation with this capability must be able to serve call proofs and storage proof requests of any block since the genesis up until and including their currently finalized block.
    • +
    • Parachain bootnode (only for relay chains). An implementation with this capability must be able to serve the network request described in RFC 8.
    • +
    +

    In the context of the head of chain provider, the word "recent" means: any not-finalized-yet block that is equal to or an ancestor of a block that it has announced through a block announce, and any finalized block whose height is superior to its current finalized block minus 16. +This does not include blocks that have been pruned because they're not a descendant of its current finalized block. In other words, blocks that aren't a descendant of the current finalized block can be thrown away. +A gap of blocks is required due to race conditions: when a node finalizes a block, it takes some time for its peers to be made aware of this, during which they might send requests concerning older blocks. The exact gap is arbitrary.

    +

    Substrate is currently by default a head of chain provider provider. After it has finished warp syncing, it downloads the list of old blocks, after which it becomes a history provider. +If Substrate is instead configured as an archive node, then it downloads the state of all blocks since the genesis, after which it becomes an archive provider, history provider, and head of chain provider. +If blocks pruning is enabled and the chain is a relay chain, then Substrate unfortunately doesn't implement any of these capabilities, not even head of chain provider. This is considered as a bug that should be fixed, see https://github.com/paritytech/polkadot-sdk/issues/2733.

    +

    DHT provider registration

    +

    This RFC heavily relies on the functionalities of the Kademlia DHT already in use by Polkadot. You can find a link to the specification here.

    +

    Implementations that have the history provider capability should register themselves as providers under the key sha256(concat("history", randomness)).

    +

    Implementations that have the archive provider capability should register themselves as providers under the key sha256(concat("archive", randomness)).

    +

    Implementations that have the parachain bootnode capability should register themselves as provider under the key sha256(concat(scale_compact(para_id), randomness)), as described in RFC 8.

    +

    "Register themselves as providers" consists in sending ADD_PROVIDER requests to nodes close to the key, as described in the Content provider advertisement section of the specification.

    +

    The value of randomness can be found in the randomness field when calling the BabeApi_currentEpoch function.

    +

    In order to avoid downtimes when the key changes, nodes should also register themselves as a secondary key that uses a value of randomness equal to the randomness field when calling BabeApi_nextEpoch.

    +

    Implementers should be aware that their implementation of Kademlia might already hash the key before XOR'ing it. The key is not meant to be hashed twice.

    +

    Implementations must not register themselves if they don't fulfill the capability yet. For example, a node configured to be an archive node but that is still building its archive state in the background must register itself only after it has finished building its archive.

    +

    Secondary DHTs

    +

    Implementations that have the history provider capability must also participate in a secondary DHT that comprises only of nodes with that capability. The protocol name of that secondary DHT must be /<genesis-hash>/kad/history.

    +

    Similarly, implementations that have the archive provider capability must also participate in a secondary DHT that comprises only of nodes with that capability and whose protocol name is /<genesis-hash>/kad/archive.

    +

    Just like implementations must not register themselves if they don't fulfill their capability yet, they must also not participate in the secondary DHT if they don't fulfill their capability yet.

    +

    Head of the chain providers

    +

    Implementations that have the head of the chain provider capability do not register themselves as providers, but instead are the nodes that participate in the main DHT. In other words, they are the nodes that serve requests of the /<genesis_hash>/kad protocol.

    +

    Any implementation that isn't a head of the chain provider (read: light clients) must not participate in the main DHT. This is already presently the case.

    +

    Implementations must not participate in the main DHT if they don't fulfill the capability yet. For example, a node that is still in the process of warp syncing must not participate in the main DHT. However, assuming that warp syncing doesn't last more than a few seconds, it is acceptable to ignore this requirement in order to avoid complicating implementations too much.

    +

    Drawbacks

    +

    None that I can see.

    +

    Testing, Security, and Privacy

    +

    The content of this section is basically the same as the one in RFC 8.

    +

    This mechanism doesn't add or remove any security by itself, as it relies on existing mechanisms.

    +

    Due to the way Kademlia works, it would become the responsibility of the 20 Polkadot nodes whose sha256(peer_id) is closest to the key (described in the explanations section) to store the list of nodes that have specific capabilities. +Furthermore, when a large number of providers are registered, only the providers closest to the key are kept, up to a certain implementation-defined limit.

    +

    For this reason, an attacker can abuse this mechanism by randomly generating libp2p PeerIds until they find the 20 entries closest to the key representing the target capability. They are then in control of the list of nodes with that capability. While doing this can in no way be actually harmful, it could lead to eclipse attacks.

    +

    Because the key changes periodically and isn't predictable, and assuming that the Polkadot DHT is sufficiently large, it is not realistic for an attack like this to be maintained in the long term.

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    +

    The DHT mechanism generally has a low overhead, especially given that publishing providers is done only every 24 hours.

    +

    Doing a Kademlia iterative query then sending a provider record shouldn't take more than around 50 kiB in total of bandwidth for the parachain bootnode.

    +

    Assuming 1000 nodes with a specific capability, the 20 Polkadot full nodes corresponding to that capability will each receive a sudden spike of a few megabytes of networking traffic when the key rotates. Again, this is relatively negligible. If this becomes a problem, one can add a random delay before a node registers itself to be the provider of the key corresponding to BabeApi_next_epoch.

    +

    Maybe the biggest uncertainty is the traffic that the 20 Polkadot full nodes will receive from light clients that desire knowing the nodes with a capability. If this every becomes a problem, this value of 20 is an arbitrary constant that can be increased for more redundancy.

    +

    Ergonomics

    +

    Irrelevant.

    +

    Compatibility

    +

    Irrelevant.

    +

    Prior Art and References

    +

    Unknown.

    +

    Unresolved Questions

    +

    While it fundamentally doesn't change much to this RFC, using BabeApi_currentEpoch and BabeApi_nextEpoch might be inappropriate. I'm not familiar enough with good practices within the runtime to have an opinion here. Should it be an entirely new pallet?

    + +

    This RFC would make it possible to reliably discover archive nodes, which would make it possible to reliably send archive node requests, something that isn't currently possible. This could solve the problem of finding archive RPC node providers by migrating archive-related request to using the native peer-to-peer protocol rather than JSON-RPC.

    +

    If we ever decide to break backwards compatibility, we could divide the "history" and "archive" capabilities in two, between nodes capable of serving older blocks and nodes capable of serving newer blocks. +We could even add to the peer-to-peer network nodes that are only capable of serving older blocks (by reading from a database) but do not participate in the head of the chain, and that just exist for historical purposes.

    (source)

    Table of Contents

      @@ -2167,19 +2291,19 @@ dispute scenarios.

      AuthorsJiahao Ye -

      Summary

      +

      Summary

      Currently, substrate runtime use an simple allocator defined by host side. Every runtime MUST import these allocator functions for normal execution. This situation make runtime code not versatile enough.

      So this RFC proposes to define a new spec for allocator part to make substrate runtime more generic.

      -

      Motivation

      +

      Motivation

      Since this RFC define a new way for allocator, we now regard the old one as legacy allocator. As we all know, since the allocator implementation details are defined by the substrate client, parachain/parathread cannot customize memory allocator algorithm, so the new specification allows the runtime to customize memory allocation, and then export the allocator function according to the specification for the client side to use. Another benefit is that some new host functions can be designed without allocating memory on the client, which may have potential performance improvements. Also it will help provide a unified and clean specification if substrate runtime support multi-targets(e.g. RISC-V). There is also a potential benefit. Many programming languages that support compilation to wasm may not be friendly to supporting external allocator. This is beneficial for other programming languages ​​to enter the substrate runtime ecosystem. The last and most important benefit is that for offchain context execution, the runtime can fully support pure wasm. What this means here is that all imported host functions could not actually be called (as stub functions), then the various verification logic of the runtime can be converted into pure wasm, which provides the possibility for the substrate runtime to run block verification in other environments (such as in browsers and other non-substrate environments).

      -

      Stakeholders

      +

      Stakeholders

      No attempt was made at convincing stakeholders.

      -

      Explanation

      +

      Explanation

      Runtime side spec

      This section contains a list of functions should be exported by substrate runtime.

      We define the spec as version 1, so the following dummy function v1 MUST be exported to hint @@ -2218,32 +2342,32 @@ Check if parsed wasm module contains a exported v1 function:

      allocator.

    Detail-heavy explanation of the RFC, suitable for explanation to an implementer of the changeset. This should address corner cases in detail and provide justification behind decisions, and provide rationale for how the design meets the solution requirements.

    -

    Drawbacks

    +

    Drawbacks

    The allocator inside of the runtime will make code size bigger, but it's not obvious. The allocator inside of the runtime maybe slow down(or speed up) the runtime, still not obvious.

    We could ignore these drawbacks since they are not prominent. And the execution efficiency is highly decided by runtime developer. We could not prevent a poor efficiency if developer want to do it.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    Keep the legacy allocator runtime test cases, and add new feature to compile test cases for v1 allocator spec. And then update the test asserts.

    Update template runtime to enable v1 spec. Once the dev network runs well, it seems that the spec is implmented correctly.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    As the above says, not obvious impact about performance. And polkadot-sdk could offer the best practice allocator for all chains. Third party also could customized by theirself. So the performance could be improved over time.

    -

    Ergonomics

    +

    Ergonomics

    Only for runtime developer, Just need to import a new crate and enable a new feature. Maybe it's convienient for other wasm-target language to implment.

    -

    Compatibility

    +

    Compatibility

    It's 100% compatible. Only Some runtime configs and executor configs need to be depreacted.

    For support new runtime spec, we MUST upgrade the client binary to support new spec of client part firstly.

    We SHALL add an optional primtive crate to enable the version 1 spec and disable the legacy allocator by cargo feature. For the first year, we SHALL disable the v1 by default, and enable it by default start in the next year.

    -

    Prior Art and References

    +

    Prior Art and References

    -

    Unresolved Questions

    +

    Unresolved Questions

    None at this time.

    - +

    The content discussed with RFC-0004 is basically orthogonal, but it could still be considered together, and it is preferred that this rfc be implmentented first.

    This feature could make substrate runtime be easier supported by other languages and integreted into other ecosystem.

    (source)

    @@ -2293,7 +2417,7 @@ For the first year, we SHALL disable the v1 by default, and enable it by default AuthorsSourabh Niyogi -

    Summary

    +

    Summary

    This RFC proposes to add the two dominant smart contract programming languages in the Polkadot ecosystem to AssetHub: EVM + ink!/Coreplay. The objective is to increase DOT Revenue by making AssetHub accessible to (1) Polkadot Rollups; @@ -2302,7 +2426,7 @@ EVM + ink!/Coreplay. The objective is to increase DOT Revenue by making AssetHu These changes in AssetHub are enabled by key Polkadot 2.0 technologies: PolkaVM supporting Coreplay, and hyper data availability in Blobs Chain.

    -

    Motivation

    +

    Motivation

    EVM Contracts are pervasive in the Web3 blockchain ecosystem, while Polkadot 2.0's Coreplay aims to surpass EVM Contracts in ease-of-use using PolkaVM's RISC architecture.

    Asset Hub for Polkadot does not have smart contract capabilities, @@ -2362,7 +2486,7 @@ This may be due to missing tooling, slow compile times, and/or simply because in This has the key new promise of making smart contract languages easier to use -- instead of smart contract developers worrying about what can be done within the gas limits of a specific block or a specific transaction, Coreplay smart contracts can be much easier to program on (see here).

    We believe AssetHub should support ink! as a precursor to support CorePlay's capabilities as soon as possible.
    To the best of our knowledge, release times of this are unknown but having ink! inside AssetHub would be natural for Polkadot 2.0.

    -

    Stakeholders

    +

    Stakeholders

    • Asset Hub Users: Those who call any extrinsic on Asset Hub for Polkadot.
    • DOT Token Holders: Those who hold DOT on any chain in the Polkadot ecosystem.
    • @@ -2370,7 +2494,7 @@ To the best of our knowledge, release times of this are Explanation +

      Explanation

      Limit Smart Contract Weight allocation

      AssetHub is a major component of the Polkadot 2.0 Minimal Relay Chain architecture. It is critical that smart contract developers not be able to clog AssetHub's blockspace for other mission critical applications, such as Staking and Governance.

      As such, it is proposed that at most 50% of the available weight in AssetHub for Polkadot blocks be allocated to smart contracts pallets (EVM, ink! and/or Coreplay). While to date AssetHub has limited usage, it is believed (see here) that imposing this limit on smart contracts pallet would limit the effect on non-smart contract usage. A excessively small weight limit like 10% or 20% may limit the attractiveness of Polkadot as a platform for Polkadot rollups and EVM Contracts. A excessively large weight like 90% or 100% may threaten AssetHub usage.

      @@ -2423,25 +2547,25 @@ We believe the lack of growth of parachains in the last 12-18 months, and the hi

      Maintaining EVM Contracts on AssetHub may be seen as difficult and may require Substrate engineers to maintain EVM Pallets and manage the xcContracts.
      We believe this cost will be relatively small based on the proven deployment of Astar and Moonbeam.
      The cost will be justified compared to the potential upside of new DOT revenue from defi/NFT applications on AssetHub and the potential for utilizing Polkadot DA for Polkadot rollups.

      -

      Testing, Security, and Privacy

      +

      Testing, Security, and Privacy

      Testing the mapping between assetIDs and EVM Contracts thoroughly will be critical.

      Having a complete working OP Stack chain using AssetHub for Kusama (1000) and Blobs on Kusama (3338) would be highly desirable, but is unlikely to be required.

      -

      Performance, Ergonomics, and Compatibility

      -

      Performance

      +

      Performance, Ergonomics, and Compatibility

      +

      Performance

      The weight limit of 50% is expected to be adequate to limit excess smart contract usage at this time.

      Storage bloat is expected to kept to a minimum with the nominal 0.01 Existential Deposit.

      -

      Ergonomics

      +

      Ergonomics

      Note that the existential deposit is not 0 DOT but being lowered from 0.1 DOT to 0.01 DOT, which may pose problems for some developers.
      Many developers routinely deploy their EVM contracts on many different EVM Chains in parallel. This non-zero ED may pose problems for some developers

      The 0.01 DOT (worth $0.075 USD) is unlikely to pose significant issue.

      -

      Compatibility

      +

      Compatibility

      It is believed that EVM pallet (as deployed on Moonbeam + Astar) is sufficiently compatible with Ethereum, and that the ED of 0.01 DOT pose negligible issues.

      The messaging architecture for rollups are not compatible with Polkadot XCM.
      It is not clear if leading rollup platforms (OP Stack, Arbitrum Orbit, Polygon zkEVM) could be made compatible with XCM.

      -

      Unresolved Questions

      +

      Unresolved Questions

      It is highly desirable to know the throughput of Polkadot DA with popular rollup architectures OP Stack and Arbitrum Orbit.
      This would enable CEXs and EVM L2 builders to choose Polkadot over Ethereum.

      - +

      If accepted, this RFC could pave the way for CorePlay on Asset Hub for Polkadot/Kusama, a major component of Polkadot 2.0's smart contract future.

      The importance of precompiles should

      (source)

      @@ -2484,9 +2608,9 @@ This would enable CEXs and EVM L2 builders to choose Polkadot over Ethereum.

      AuthorsGavin Wood -

      Summary

      +

      Summary

      This proposes a periodic, sale-based method for assigning Polkadot Coretime, the analogue of "block space" within the Polkadot Network. The method takes into account the need for long-term capital expenditure planning for teams building on Polkadot, yet also provides a means to allow Polkadot to capture long-term value in the resource which it sells. It supports the possibility of building rich and dynamic secondary markets to optimize resource allocation and largely avoids the need for parameterization.

      -

      Motivation

      +

      Motivation

      Present System

      The Polkadot Ubiquitous Computer, or just Polkadot UC, represents the public service provided by the Polkadot Network. It is a trust-free, WebAssembly-based, multicore, internet-native omnipresent virtual machine which is highly resilient to interference and corruption.

      The present system of allocating the limited resources of the Polkadot Ubiquitous Computer is through a process known as parachain slot auctions. This is a parachain-centric paradigm whereby a single core is long-term allocated to a single parachain which itself implies a Substrate/Cumulus-based chain secured and connected via the Relay-chain. Slot auctions are on-chain candle auctions which proceed for several days and result in the core being assigned to the parachain for six months at a time up to 24 months in advance. Practically speaking, we only see two year periods being bid upon and leased.

      @@ -2507,7 +2631,7 @@ This would enable CEXs and EVM L2 builders to choose Polkadot over Ethereum.

    • The solution SHOULD avoid creating additional dependencies on functionality which the Relay-chain need not strictly provide for the delivery of the Polkadot UC.
    • Furthermore, the design SHOULD be implementable and deployable in a timely fashion; three months from the acceptance of this RFC should not be unreasonable.

      -

      Stakeholders

      +

      Stakeholders

      Primary stakeholder sets are:

      • Protocol researchers and developers, largely represented by the Polkadot Fellowship and Parity Technologies' Engineering division.
      • @@ -2516,7 +2640,7 @@ This would enable CEXs and EVM L2 builders to choose Polkadot over Ethereum.

      Socialization:

      The essensials of this proposal were presented at Polkadot Decoded 2023 Copenhagen on the Main Stage. A small amount of socialization at the Parachain Summit preceeded it and some substantial discussion followed it. Parity Ecosystem team is currently soliciting views from ecosystem teams who would be key stakeholders.

      -

      Explanation

      +

      Explanation

      Overview

      Upon implementation of this proposal, the parachain-centric slot auctions and associated crowdloans cease. Instead, Coretime on the Polkadot UC is sold by the Polkadot System in two separate formats: Bulk Coretime and Instantaneous Coretime.

      When a Polkadot Core is utilized, we say it is dedicated to a Task rather than a "parachain". The Task to which a Core is dedicated may change at every Relay-chain block and while one predominant type of Task is to secure a Cumulus-based blockchain (i.e. a parachain), other types of Tasks are envisioned.

      @@ -2948,16 +3072,16 @@ InstaPoolHistory: (empty)
    • Governance upgrade proposal(s).
    • Monitoring of the upgrade process.
    • -

      Performance, Ergonomics and Compatibility

      +

      Performance, Ergonomics and Compatibility

      No specific considerations.

      Parachains already deployed into the Polkadot UC must have a clear plan of action to migrate to an agile Coretime market.

      While this proposal does not introduce documentable features per se, adequate documentation must be provided to potential purchasers of Polkadot Coretime. This SHOULD include any alterations to the Polkadot-SDK software collection.

      -

      Testing, Security and Privacy

      +

      Testing, Security and Privacy

      Regular testing through unit tests, integration tests, manual testnet tests, zombie-net tests and fuzzing SHOULD be conducted.

      A regular security review SHOULD be conducted prior to deployment through a review by the Web3 Foundation economic research group.

      Any final implementation MUST pass a professional external security audit.

      The proposal introduces no new privacy concerns.

      - +

      RFC-3 proposes a means of implementing the high-level allocations within the Relay-chain.

      RFC-5 proposes the API for interacting with Relay-chain.

      Additional work should specify the interface for the instantaneous market revenue so that the Coretime-chain can ensure Bulk Coretime placed in the instantaneous market is properly compensated.

      @@ -2973,7 +3097,7 @@ InstaPoolHistory: (empty)
    • The percentage of cores to be sold as Bulk Coretime.
    • The fate of revenue collected.
    -

    Prior Art and References

    +

    Prior Art and References

    Robert Habermeier initially wrote on the subject of Polkadot blockspace-centric in the article Polkadot Blockspace over Blockchains. While not going into details, the article served as an early reframing piece for moving beyond one-slot-per-chain models and building out secondary market infrastructure for resource allocation.

    (source)

    Table of Contents

    @@ -3006,10 +3130,10 @@ InstaPoolHistory: (empty) AuthorsGavin Wood, Robert Habermeier -

    Summary

    +

    Summary

    In the Agile Coretime model of the Polkadot Ubiquitous Computer, as proposed in RFC-1 and RFC-3, it is necessary for the allocating parachain (envisioned to be one or more pallets on a specialised Brokerage System Chain) to communicate the core assignments to the Relay-chain, which is responsible for ensuring those assignments are properly enacted.

    This is a proposal for the interface which will exist around the Relay-chain in order to communicate this information and instructions.

    -

    Motivation

    +

    Motivation

    The background motivation for this interface is splitting out coretime allocation functions and secondary markets from the Relay-chain onto System parachains. A well-understood and general interface is necessary for ensuring the Relay-chain receives coretime allocation instructions from one or more System chains without introducing dependencies on the implementation details of either side.

    Requirements

      @@ -3021,7 +3145,7 @@ InstaPoolHistory: (empty)
    • The interface MUST allow for the allocating chain to instruct changes to the number of cores which it is able to allocate.
    • The interface MUST allow for the allocating chain to be notified of changes to the number of cores which are able to be allocated by the allocating chain.
    -

    Stakeholders

    +

    Stakeholders

    Primary stakeholder sets are:

    • Developers of the Relay-chain core-management logic.
    • @@ -3029,7 +3153,7 @@ InstaPoolHistory: (empty)

    Socialization:

    This content of this RFC was discussed in the Polkdot Fellows channel.

    -

    Explanation

    +

    Explanation

    The interface has two sections: The messages which the Relay-chain is able to receive from the allocating parachain (the UMP message types), and messages which the Relay-chain is able to send to the allocating parachain (the DMP message types). These messages are expected to be able to be implemented in a well-known pallet and called with the XCM Transact instruction.

    Future work may include these messages being introduced into the XCM standard.

    UMP Message Types

    @@ -3104,17 +3228,17 @@ assert_eq!(targets.iter().map(|x| x.1).sum(), 57600);

    Realistic Limits of the Usage

    For request_revenue_info, a successful request should be possible if when is no less than the Relay-chain block number on arrival of the message less 100,000.

    For assign_core, a successful request should be possible if begin is no less than the Relay-chain block number on arrival of the message plus 10 and workload contains no more than 100 items.

    -

    Performance, Ergonomics and Compatibility

    +

    Performance, Ergonomics and Compatibility

    No specific considerations.

    -

    Testing, Security and Privacy

    +

    Testing, Security and Privacy

    Standard Polkadot testing and security auditing applies.

    The proposal introduces no new privacy concerns.

    - +

    RFC-1 proposes a means of determining allocation of Coretime using this interface.

    RFC-3 proposes a means of implementing the high-level allocations within the Relay-chain.

    Drawbacks, Alternatives and Unknowns

    None at present.

    -

    Prior Art and References

    +

    Prior Art and References

    None.

    (source)

    Table of Contents

    @@ -3160,13 +3284,13 @@ assert_eq!(targets.iter().map(|x| x.1).sum(), 57600); AuthorsJoe Petrowski -

    Summary

    +

    Summary

    As core functionality moves from the Relay Chain into system chains, so increases the reliance on the liveness of these chains for the use of the network. It is not economically scalable, nor necessary from a game-theoretic perspective, to pay collators large rewards. This RFC proposes a mechanism -- part technical and part social -- for ensuring reliable collator sets that are resilient to attemps to stop any subsytem of the Polkadot protocol.

    -

    Motivation

    +

    Motivation

    In order to guarantee access to Polkadot's system, the collators on its system chains must propose blocks (provide liveness) and allow all transactions to eventually be included. That is, some collators may censor transactions, but there must exist one collator in the set who will include a @@ -3202,12 +3326,12 @@ to censor any subset of transactions.

  • Collators selected by governance SHOULD have a reasonable expectation that the Treasury will reimburse their operating costs.
  • -

    Stakeholders

    +

    Stakeholders

    • Infrastructure providers (people who run validator/collator nodes)
    • Polkadot Treasury
    -

    Explanation

    +

    Explanation

    This protocol builds on the existing Collator Selection pallet and its notion of Invulnerables. Invulnerables are collators (identified by their AccountIds) who @@ -3243,27 +3367,27 @@ approximately:

  • of which 15 are Invulnerable, and
  • five are elected by bond.
  • -

    Drawbacks

    +

    Drawbacks

    The primary drawback is a reliance on governance for continued treasury funding of infrastructure costs for Invulnerable collators.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    The vast majority of cases can be covered by unit testing. Integration test should ensure that the Collator Selection UpdateOrigin, which has permission to modify the Invulnerables and desired number of Candidates, can handle updates over XCM from the system's governance location.

    -

    Performance, Ergonomics, and Compatibility

    +

    Performance, Ergonomics, and Compatibility

    This proposal has very little impact on most users of Polkadot, and should improve the performance of system chains by reducing the number of missed blocks.

    -

    Performance

    +

    Performance

    As chains have strict PoV size limits, care must be taken in the PoV impact of the session manager. Appropriate benchmarking and tests should ensure that conservative limits are placed on the number of Invulnerables and Candidates.

    -

    Ergonomics

    +

    Ergonomics

    The primary group affected is Candidate collators, who, after implementation of this RFC, will need to compete in a bond-based election rather than a race to claim a Candidate spot.

    -

    Compatibility

    +

    Compatibility

    This RFC is compatible with the existing implementation and can be handled via upgrades and migration.

    -

    Prior Art and References

    +

    Prior Art and References

    Written Discussions

    • GitHub: Collator Selection Roadmap
    • @@ -3278,9 +3402,9 @@ migration.

    • SR Labs Auditors
    • Current collators including Paranodes, Stake Plus, Turboflakes, Peter Mensik, SIK, and many more.
    -

    Unresolved Questions

    +

    Unresolved Questions

    None at this time.

    - +

    There may exist in the future system chains for which this model of collator selection is not appropriate. These chains should be evaluated on a case-by-case basis.

    (source)

    @@ -3319,10 +3443,10 @@ appropriate. These chains should be evaluated on a case-by-case basis.

    AuthorsPierre Krieger -

    Summary

    +

    Summary

    The full nodes of the Polkadot peer-to-peer network maintain a distributed hash table (DHT), which is currently used for full nodes discovery and validators discovery purposes.

    This RFC proposes to extend this DHT to be used to discover full nodes of the parachains of Polkadot.

    -

    Motivation

    +

    Motivation

    The maintenance of bootnodes has long been an annoyance for everyone.

    When a bootnode is newly-deployed or removed, every chain specification must be updated in order to take the update into account. This has lead to various non-optimal solutions, such as pulling chain specifications from GitHub repositories. When it comes to RPC nodes, UX developers often have trouble finding up-to-date addresses of parachain RPC nodes. With the ongoing migration from RPC nodes to light clients, similar problems would happen with chain specifications as well.

    @@ -3331,14 +3455,14 @@ When it comes to RPC nodes, UX developers often have trouble finding up-to-date

    Because the list of bootnodes in chain specifications is so annoying to modify, the consequence is that the number of bootnodes is rather low (typically between 2 and 15). In order to better resist downtimes and DoS attacks, a better solution would be to use every node of a certain chain as potential bootnode, rather than special-casing some specific nodes.

    While this RFC doesn't solve these problems for relay chains, it aims at solving it for parachains by storing the list of all the full nodes of a parachain on the relay chain DHT.

    Assuming that this RFC is implemented, and that light clients are used, deploying a parachain wouldn't require more work than registering it onto the relay chain and starting the collators. There wouldn't be any need for special infrastructure nodes anymore.

    -

    Stakeholders

    +

    Stakeholders

    This RFC has been opened on my own initiative because I think that this is a good technical solution to a usability problem that many people are encountering and that they don't realize can be solved.

    -

    Explanation

    +

    Explanation

    The content of this RFC only applies for parachains and parachain nodes that are "Substrate-compatible". It is in no way mandatory for parachains to comply to this RFC.

    Note that "Substrate-compatible" is very loosely defined as "implements the same mechanisms and networking protocols as Substrate". The author of this RFC believes that "Substrate-compatible" should be very precisely specified, but there is controversy on this topic.

    While a lot of this RFC concerns the implementation of parachain nodes, it makes use of the resources of the Polkadot chain, and as such it is important to describe them in the Polkadot specification.

    This RFC adds two mechanisms: a registration in the DHT, and a new networking protocol.

    -

    DHT provider registration

    +

    DHT provider registration

    This RFC heavily relies on the functionalities of the Kademlia DHT already in use by Polkadot. You can find a link to the specification here.

    Full nodes of a parachain registered on Polkadot should register themselves onto the Polkadot DHT as the providers of a key corresponding to the parachain that they are serving, as described in the Content provider advertisement section of the specification. This uses the ADD_PROVIDER system of libp2p-kademlia.

    @@ -3370,10 +3494,10 @@ message Response {

    The maximum size of a response is set to an arbitrary 16kiB. The responding side should make sure to conform to this limit. Given that fork_id is typically very small and that the only variable-length field is addrs, this is easily achieved by limiting the number of addresses.

    Implementers should be aware that addrs might be very large, and are encouraged to limit the number of addrs to an implementation-defined value.

    -

    Drawbacks

    +

    Drawbacks

    The peer_id and addrs fields are in theory not strictly needed, as the PeerId and addresses could be always equal to the PeerId and addresses of the node being registered as the provider and serving the response. However, the Cumulus implementation currently uses two different networking stacks, one of the parachain and one for the relay chain, using two separate PeerIds and addresses, and as such the PeerId and addresses of the other networking stack must be indicated. Asking them to use only one networking stack wouldn't feasible in a realistic time frame.

    The values of the genesis_hash and fork_id fields cannot be verified by the requester and are expected to be unused at the moment. Instead, a client that desires connecting to a parachain is expected to obtain the genesis hash and fork ID of the parachain from the parachain chain specification. These fields are included in the networking protocol nonetheless in case an acceptable solution is found in the future, and in order to allow use cases such as discovering parachains in a not-strictly-trusted way.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    Because not all nodes want to be used as bootnodes, implementers are encouraged to provide a way to disable this mechanism. However, it is very much encouraged to leave this mechanism on by default for all parachain nodes.

    This mechanism doesn't add or remove any security by itself, as it relies on existing mechanisms. However, if the principle of chain specification bootnodes is entirely replaced with the mechanism described in this RFC (which is the objective), then it becomes important whether the mechanism in this RFC can be abused in order to make a parachain unreachable.

    @@ -3382,22 +3506,22 @@ Furthermore, when a large number of providers (here, a provider is a bootnode) a

    For this reason, an attacker can abuse this mechanism by randomly generating libp2p PeerIds until they find the 20 entries closest to the key representing the target parachain. They are then in control of the parachain bootnodes. Because the key changes periodically and isn't predictable, and assuming that the Polkadot DHT is sufficiently large, it is not realistic for an attack like this to be maintained in the long term.

    Furthermore, parachain clients are expected to cache a list of known good nodes on their disk. If the mechanism described in this RFC went down, it would only prevent new nodes from accessing the parachain, while clients that have connected before would not be affected.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    The DHT mechanism generally has a low overhead, especially given that publishing providers is done only every 24 hours.

    Doing a Kademlia iterative query then sending a provider record shouldn't take more than around 50 kiB in total of bandwidth for the parachain bootnode.

    Assuming 1000 parachain full nodes, the 20 Polkadot full nodes corresponding to a specific parachain will each receive a sudden spike of a few megabytes of networking traffic when the key rotates. Again, this is relatively negligible. If this becomes a problem, one can add a random delay before a parachain full node registers itself to be the provider of the key corresponding to BabeApi_next_epoch.

    Maybe the biggest uncertainty is the traffic that the 20 Polkadot full nodes will receive from light clients that desire knowing the bootnodes of a parachain. Light clients are generally encouraged to cache the peers that they use between restarts, so they should only query these 20 Polkadot full nodes at their first initialization. If this every becomes a problem, this value of 20 is an arbitrary constant that can be increased for more redundancy.

    -

    Ergonomics

    +

    Ergonomics

    Irrelevant.

    -

    Compatibility

    +

    Compatibility

    Irrelevant.

    -

    Prior Art and References

    +

    Prior Art and References

    None.

    -

    Unresolved Questions

    +

    Unresolved Questions

    While it fundamentally doesn't change much to this RFC, using BabeApi_currentEpoch and BabeApi_nextEpoch might be inappropriate. I'm not familiar enough with good practices within the runtime to have an opinion here. Should it be an entirely new pallet?

    - +

    It is possible that in the future a client could connect to a parachain without having to rely on a trusted parachain specification.

    (source)

    Table of Contents

    @@ -3427,13 +3551,13 @@ If this every becomes a problem, this value of 20 is an arbitrary constant that AuthorsJoe Petrowski -

    Summary

    +

    Summary

    Since the introduction of the Collectives parachain, many groups have expressed interest in forming new -- or migrating existing groups into -- on-chain collectives. While adding a new collective is relatively simple from a technical standpoint, the Fellowship will need to merge new pallets into the Collectives parachain for each new collective. This RFC proposes a means for the network to ratify a new collective, thus instructing the Fellowship to instate it in the runtime.

    -

    Motivation

    +

    Motivation

    Many groups have expressed interest in representing collectives on-chain. Some of these include:

    -

    Drawbacks

    +

    Drawbacks

    Other than all other system chains, development and maintenance of the Encointer Network is mainly financed by the KSM Treasury and possibly the DOT Treasury in the future. Encointer is dedicated to maintaining its network and runtime code for as long as possible, but there is a dependency on funding which is not in the hands of the fellowship. The only risk in the context of funding, however, is that the Encointer runtime will see less frequent updates if there's less funding.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    No changes to the existing system are proposed. Only changes to how maintenance is organized.

    -

    Performance, Ergonomics, and Compatibility

    +

    Performance, Ergonomics, and Compatibility

    No changes

    -

    Prior Art and References

    +

    Prior Art and References

    Existing Encointer runtime repo

    -

    Unresolved Questions

    +

    Unresolved Questions

    None identified

    - +

    More info on Encointer: encointer.org

    (source)

    Table of Contents

    @@ -3741,11 +3865,11 @@ ones.

    AuthorsJoe Petrowski, Gavin Wood -

    Summary

    +

    Summary

    The Relay Chain contains most of the core logic for the Polkadot network. While this was necessary prior to the launch of parachains and development of XCM, most of this logic can exist in parachains. This is a proposal to migrate several subsystems into system parachains.

    -

    Motivation

    +

    Motivation

    Polkadot's scaling approach allows many distinct state machines (known generally as parachains) to operate with common guarantees about the validity and security of their state transitions. Polkadot provides these common guarantees by executing the state transitions on a strict subset (a backing @@ -3757,13 +3881,13 @@ blockspace) to the network.

    By minimising state transition logic on the Relay Chain by migrating it into "system chains" -- a set of parachains that, with the Relay Chain, make up the Polkadot protocol -- the Polkadot Ubiquitous Computer can maximise its primary offering: secure blockspace.

    -

    Stakeholders

    +

    Stakeholders

    -

    Explanation

    +

    Explanation

    The following pallets and subsystems are good candidates to migrate from the Relay Chain:

    -

    Drawbacks

    +

    Drawbacks

    This RFC might be difficult to implement in Substrate due to the internal code design. It is not clear to the author of this RFC how difficult it would be.

    Prior Art

    The API of these new functions was heavily inspired by API used by the C programming language.

    -

    Unresolved Questions

    +

    Unresolved Questions

    The changes in this RFC would need to be benchmarked. This involves implementing the RFC and measuring the speed difference.

    It is expected that most host functions are faster or equal speed to their deprecated counterparts, with the following exceptions:

      @@ -4467,10 +4591,10 @@ This would remove the possibility to synchronize older blocks, which is probably LicenseMIT -

      Summary

      +

      Summary

      This RFC proposes a dynamic pricing model for the sale of Bulk Coretime on the Polkadot UC. The proposed model updates the regular price of cores for each sale period, by taking into account the number of cores sold in the previous sale, as well as a limit of cores and a target number of cores sold. It ensures a minimum price and limits price growth to a maximum price increase factor, while also giving govenance control over the steepness of the price change curve. It allows governance to address challenges arising from changing market conditions and should offer predictable and controlled price adjustments.

      Accompanying visualizations are provided at [1].

      -

      Motivation

      +

      Motivation

      RFC-1 proposes periodic Bulk Coretime Sales as a mechanism to sell continouos regions of blockspace (suggested to be 4 weeks in length). A number of Blockspace Regions (compare RFC-1 & RFC-3) are provided for sale to the Broker-Chain each period and shall be sold in a way that provides value-capture for the Polkadot network. The exact pricing mechanism is out of scope for RFC-1 and shall be provided by this RFC.

      A dynamic pricing model is needed. A limited number of Regions are offered for sale each period. The model needs to find the price for a period based on supply and demand of the previous period.

      The model shall give Coretime consumers predictability about upcoming price developments and confidence that Polkadot governance can adapt the pricing model to changing market conditions.

      @@ -4482,7 +4606,7 @@ This would remove the possibility to synchronize older blocks, which is probably
    • The solution SHOULD provide a maximum factor of price increase should the limit of Regions sold per period be reached.
    • The solution should allow governance to control the steepness of the price function
    • -

      Stakeholders

      +

      Stakeholders

      The primary stakeholders of this RFC are:

      • Protocol researchers and evelopers
      • @@ -4490,7 +4614,7 @@ This would remove the possibility to synchronize older blocks, which is probably
      • Polkadot parachains teams
      • Brokers involved in the trade of Bulk Coretime
      -

      Explanation

      +

      Explanation

      Overview

      The dynamic pricing model sets the new price based on supply and demand in the previous period. The model is a function of the number of Regions sold, piecewise-defined by two power functions.

        @@ -4589,9 +4713,9 @@ SCALE_DOWN = 1 SCALE_UP = 1 OLD_PRICE = 1000
    -

    Drawbacks

    +

    Drawbacks

    None at present.

    -

    Prior Art and References

    +

    Prior Art and References

    This pricing model is based on the requirements from the basic linear solution proposed in RFC-1, which is a simple dynamic pricing model and only used as proof. The present model adds additional considerations to make the model more adaptable under real conditions.

    Future Possibilities

    This RFC, if accepted, shall be implemented in conjunction with RFC-1.

    @@ -4630,9 +4754,9 @@ OLD_PRICE = 1000 AuthorsPierre Krieger -

    Summary

    +

    Summary

    Improve the networking messages that query storage items from the remote, in order to reduce the bandwidth usage and number of round trips of light clients.

    -

    Motivation

    +

    Motivation

    Clients on the Polkadot peer-to-peer network can be divided into two categories: full nodes and light clients. So-called full nodes are nodes that store the content of the chain locally on their disk, while light clients are nodes that don't. In order to access for example the balance of an account, a full node can do a disk read, while a light client needs to send a network message to a full node and wait for the full node to reply with the desired value. This reply is in the form of a Merkle proof, which makes it possible for the light client to verify the exactness of the value.

    Unfortunately, this network protocol is suffering from some issues:

    Once Polkadot and Kusama will have transitioned to state_version = 1, which modifies the format of the trie entries, it will be possible to generate Merkle proofs that contain only the hashes of values in the storage. Thanks to this, it is already possible to prove the existence of a key without sending its entire value (only its hash), or to prove that a value has changed or not between two blocks (by sending just their hashes). Thus, the only reason why aforementioned issues exist is because the existing networking messages don't give the possibility for the querier to query this. This is what this proposal aims at fixing.

    -

    Stakeholders

    +

    Stakeholders

    This is the continuation of https://github.com/w3f/PPPs/pull/10, which itself is the continuation of https://github.com/w3f/PPPs/pull/5.

    -

    Explanation

    +

    Explanation

    The protobuf schema of the networking protocol can be found here: https://github.com/paritytech/substrate/blob/5b6519a7ff4a2d3cc424d78bc4830688f3b184c0/client/network/light/src/schema/light.v1.proto

    The proposal is to modify this protocol in this way:

    @@ -11,6 +11,7 @@ message Request {
    @@ -4702,26 +4826,26 @@ An alternative could have been to specify the child_trie_info for e
     Also note that child tries aren't considered as descendants of the main trie when it comes to the includeDescendants flag. In other words, if the request concerns the main trie, no content coming from child tries is ever sent back.

    This protocol keeps the same maximum response size limit as currently exists (16 MiB). It is not possible for the querier to know in advance whether its query will lead to a reply that exceeds the maximum size. If the reply is too large, the replier should send back only a limited number (but at least one) of requested items in the proof. The querier should then send additional requests for the rest of the items. A response containing none of the requested items is invalid.

    The server is allowed to silently discard some keys of the request if it judges that the number of requested keys is too high. This is in line with the fact that the server might truncate the response.

    -

    Drawbacks

    +

    Drawbacks

    This proposal doesn't handle one specific situation: what if a proof containing a single specific item would exceed the response size limit? For example, if the response size limit was 1 MiB, querying the runtime code (which is typically 1.0 to 1.5 MiB) would be impossible as it's impossible to generate a proof less than 1 MiB. The response size limit is currently 16 MiB, meaning that no single storage item must exceed 16 MiB.

    Unfortunately, because it's impossible to verify a Merkle proof before having received it entirely, parsing the proof in a streaming way is also not possible.

    A way to solve this issue would be to Merkle-ize large storage items, so that a proof could include only a portion of a large storage item. Since this would require a change to the trie format, it is not realistically feasible in a short time frame.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    The main security consideration concerns the size of replies and the resources necessary to generate them. It is for example easily possible to ask for all keys and values of the chain, which would take a very long time to generate. Since responses to this networking protocol have a maximum size, the replier should truncate proofs that would lead to the response being too large. Note that it is already possible to send a query that would lead to a very large reply with the existing network protocol. The only thing that this proposal changes is that it would make it less complicated to perform such an attack.

    Implementers of the replier side should be careful to detect early on when a reply would exceed the maximum reply size, rather than inconditionally generate a reply, as this could take a very large amount of CPU, disk I/O, and memory. Existing implementations might currently be accidentally protected from such an attack thanks to the fact that requests have a maximum size, and thus that the list of keys in the query was bounded. After this proposal, this accidental protection would no longer exist.

    Malicious server nodes might truncate Merkle proofs even when they don't strictly need to, and it is not possible for the client to (easily) detect this situation. However, malicious server nodes can already do undesirable things such as throttle down their upload bandwidth or simply not respond. There is no need to handle unnecessarily truncated Merkle proofs any differently than a server simply not answering the request.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    It is unclear to the author of the RFC what the performance implications are. Servers are supposed to have limits to the amount of resources they use to respond to requests, and as such the worst that can happen is that light client requests become a bit slower than they currently are.

    -

    Ergonomics

    +

    Ergonomics

    Irrelevant.

    -

    Compatibility

    +

    Compatibility

    The prior networking protocol is maintained for now. The older version of this protocol could get removed in a long time.

    -

    Prior Art and References

    +

    Prior Art and References

    None. This RFC is a clean-up of an existing mechanism.

    -

    Unresolved Questions

    +

    Unresolved Questions

    None

    - +

    The current networking protocol could be deprecated in a long time. Additionally, the current "state requests" protocol (used for warp syncing) could also be deprecated in favor of this one.

    (source)

    Table of Contents

    @@ -4742,13 +4866,13 @@ Also note that child tries aren't considered as descendants of the main trie whe AuthorsJonas Gehrlein -

    Summary

    +

    Summary

    The Polkadot UC will generate revenue from the sale of available Coretime. The question then arises: how should we handle these revenues? Broadly, there are two reasonable paths – burning the revenue and thereby removing it from total issuance or divert it to the Treasury. This Request for Comment (RFC) presents arguments favoring burning as the preferred mechanism for handling revenues from Coretime sales.

    -

    Motivation

    +

    Motivation

    How to handle the revenue accrued from Coretime sales is an important economic question that influences the value of DOT and should be properly discussed before deciding for either of the options. Now is the best time to start this discussion.

    -

    Stakeholders

    +

    Stakeholders

    Polkadot DOT token holders.

    -

    Explanation

    +

    Explanation

    This RFC discusses potential benefits of burning the revenue accrued from Coretime sales instead of diverting them to Treasury. Here are the following arguments for it.

    It's in the interest of the Polkadot community to have a consistent and predictable Treasury income, because volatility in the inflow can be damaging, especially in situations when it is insufficient. As such, this RFC operates under the presumption of a steady and sustainable Treasury income flow, which is crucial for the Polkadot community's stability. The assurance of a predictable Treasury income, as outlined in a prior discussion here, or through other equally effective measures, serves as a baseline assumption for this argument.

    Consequently, we need not concern ourselves with this particular issue here. This naturally begs the question - why should we introduce additional volatility to the Treasury by aligning it with the variable Coretime sales? It's worth noting that Coretime revenues often exhibit an inverse relationship with periods when Treasury spending should ideally be ramped up. During periods of low Coretime utilization (indicated by lower revenue), Treasury should spend more on projects and endeavours to increase the demand for Coretime. This pattern underscores that Coretime sales, by their very nature, are an inconsistent and unpredictable source of funding for the Treasury. Given the importance of maintaining a steady and predictable inflow, it's unnecessary to rely on another volatile mechanism. Some might argue that we could have both: a steady inflow (from inflation) and some added bonus from Coretime sales, but burning the revenue would offer further benefits as described below.

    @@ -4800,11 +4924,11 @@ Also note that child tries aren't considered as descendants of the main trie whe AuthorsOliver Tale-Yazdi -

    Summary

    +

    Summary

    Introduces breaking changes to the BlockBuilder and Core runtime APIs.
    A new function BlockBuilder::last_inherent is introduced and the return value of Core::initialize_block is changed to an enum.
    The versions of both APIs are bumped; BlockBuilder to 7 and Core to 5.

    -

    Motivation

    +

    Motivation

    There are three main features that motivate for this RFC:

    1. Multi-Block-Migrations: These make it possible to split a migration over multiple blocks.
    2. @@ -4816,7 +4940,7 @@ The versions of both APIs are bumped; BlockBuilder to 7 and C
    3. The runtime can tell the block author to not include any transactions in the block.
    4. The runtime can execute logic right after all pallet-provided inherents have been applied.
    -

    Stakeholders

    +

    Stakeholders

    • Substrate Maintainers: They have to implement this, including tests, audit and maintenance burden.
    • @@ -4824,7 +4948,7 @@ maintenance burden.
    • Polkadot Parachain Teams: They also have to adapt to the breaking changes but then eventually have multi-block migrations available.
    -

    Explanation

    +

    Explanation

    Core::initialize_block

    This runtime API function is changed from returning () to ExtrinsicInclusionMode:

    #![allow(unused)]
    @@ -4844,26 +4968,26 @@ multi-block migrations available.
     

    1. Multi-Block-Migrations: The runtime is being put into lock-down mode for the duration of the migration process by returning OnlyInherents from initialize_block. This ensures that no user provided transaction can interfere with the migration process. It is absolutely necessary to ensure this, since otherwise a transaction could call into un-migrated storage and violate storage invariants. The entry-point for the MBM logic is last_inherent. This is a good spot, because any data that is touched in inherents, is not MBM-migratable anyway. It could also be done before all other inherents or at the end of the block in finalize_block, but there is no downside from doing it in last_inherent and the other two features are in favour of this.

    2. poll becomes possible by using last_inherent as entry-point. It would not be possible to use a pallet inherent like System::last_inherent to achieve this for two reasons. First is that pallets do not have access to AllPalletsWithSystem that is required to invoke the poll hook on all pallets. Second is that the runtime does currently not enforce an order of inherents.

    3. System::PostInherents can be done in the same manner as poll.

    -

    Drawbacks

    +

    Drawbacks

    As noted in the review comments: this cements some assumptions about the order of inherents into the BlockBuilder traits. It was criticized for being to rigid in its assumptions.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    Compliance of a block author can be tested by adding specific code to the last_inherent hook and checking that it always executes. The new logic of initialize_block can be tested by checking that the block-builder will skip transactions and optional hooks when OnlyInherents is returned.

    Security: n/a

    Privacy: n/a

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    The performance overhead is minimal in the sense that no clutter was added after fulfilling the requirements. A slight performance penalty is expected from invoking last_inherent once per block.

    -

    Ergonomics

    +

    Ergonomics

    The new interface allows for more extensible runtime logic. In the future, this will be utilized for multi-block-migrations which should be a huge ergonomic advantage for parachain developers.

    -

    Compatibility

    +

    Compatibility

    The advice here is OPTIONAL and outside of the RFC. To not degrade user experience, it is recommended to ensure that an updated node can still import historic blocks.

    -

    Prior Art and References

    +

    Prior Art and References

    The RFC is currently being implemented in polkadot-sdk#1781. Related issues and merge requests:

    -

    Unresolved Questions

    +

    Unresolved Questions

    Please suggest a better name for BlockExecutiveMode. We already tried: RuntimeExecutiveMode, ExtrinsicInclusionMode. The names of the modes Normal and Minimal were also called AllExtrinsics and OnlyInherents, so if you have naming preferences; please post them.
    => renamed to ExtrinsicInclusionMode

    Is post_inherents more consistent instead of last_inherent? Then we should change it.
    => renamed to last_inherent

    - +

    The long-term future here is to move the block building logic into the runtime. Currently there is a tight dance between the block author and the runtime; the author has to call into different runtime functions in quick succession and exact order. Any misstep causes the built block to be invalid.
    This can be unified and simplified by moving both parts of the logic into the runtime.

    (source)

    @@ -4911,9 +5035,9 @@ This can be unified and simplified by moving both parts of the logic into the ru AuthorsJonas Gehrlein -

    Summary

    +

    Summary

    This document is a proposal for restructuring the bulk markets in the Polkadot UC's coretime allocation system to improve efficiency and fairness. The proposal suggests separating the BULK_PERIOD into MARKET_PERIOD and RENEWAL_PERIOD, allowing for a market-driven price discovery through a clearing price Dutch auction during the MARKET_PERIOD followed by renewal offers at the MARKET_PRICE during the RENEWAL_PERIOD. The new system ensures synchronicity between renewal and market prices, fairness among all current tenants, and efficient price discovery, while preserving price caps to provide security for current tenants. It seeks to start a discussion about the possibility of long-term leases.

    -

    Motivation

    +

    Motivation

    While the initial RFC-1 has provided a robust framework for Coretime allocation within the Polkadot UC, this proposal builds upon its strengths and uses many provided building blocks to address some areas that could be further improved.

    In particular, this proposal introduces the following changes:

      @@ -4933,14 +5057,14 @@ This can be unified and simplified by moving both parts of the logic into the ru

    The premise of this proposal is to reduce complexity by introducing a common price (that develops releative to capacity consumption of Polkadot UC), while still allowing for market forces to add efficiency. Longterm lease owners still receive priority IF they can pay (close to) the market price. This prevents a situation where the renewal price significantly diverges from renewal prices which allows for core captures. While maximum price increase certainty might seem contradictory to efficient price discovery, the proposed model aims to balance these elements, utilizing market forces to determine the price and allocate cores effectively within certain bounds. It must be stated, that potential price increases remain predictable (in the worst-case) but could be higher than in the originally proposed design. The argument remains, however, that we need to allow market forces to affect all prices for an efficient Coretime pricing and allocation.

    Ultimately, this the framework proposed here adheres to all requirements stated in RFC-1.

    -

    Stakeholders

    +

    Stakeholders

    Primary stakeholder sets are:

    • Protocol researchers and developers, largely represented by the Polkadot Fellowship and Parity Technologies' Engineering division.
    • Polkadot Parachain teams both present and future, and their users.
    • Polkadot DOT token holders.
    -

    Explanation

    +

    Explanation

    Bulk Markets

    The BULK_PERIOD has been restructured into two primary segments: the MARKET_PERIOD and RENEWAL_PERIOD, along with an auxiliary SETTLEMENT_PERIOD. This latter period doesn't necessitate any actions from the coretime system chain, but it facilitates a more efficient allocation of coretime in secondary markets. A significant departure from the original proposal lies in the timing of renewals, which now occur post-market phase. This adjustment aims to harmonize renewal prices with their market counterparts, ensuring a more consistent and equitable pricing model.

    Market Period (14 days)

    @@ -4987,12 +5111,12 @@ This can be unified and simplified by moving both parts of the logic into the ru
    • Long-term Coretime: The Polkadot UC is undergoing a transition from two-year leases without an instantaneous market to a model encompassing instantaneous and one-month leases. This shift seems to pivot from one extreme to another. While the introduction of short-term leases, both instantaneous and for one month, is a constructive move to lower barriers to entry and promote experimentation, it seems to be the case that established projects might benefit from more extended lease options. We could consider offering another product, such as a six-month Coretime lease, using the same mechanism described herein. Although the majority of leases would still be sold on a one-month basis, the addition of this option would enhance market efficiency as it would strengthen the impact of a secondary market.
    -

    Drawbacks

    +

    Drawbacks

    There are trade-offs that arise from this proposal, compared to the initial model. The most notable one is that here, I prioritize requirement 6 over requirement 2. The price, in the very "worst-case" (meaning a huge explosion in demand for coretime) could lead to a much larger increase of prices in Coretime. From an economic perspective, this (rare edgecase) would also mean that we'd vastly underprice Coretime in the original model, leading to highly inefficient allocations.

    -

    Prior Art and References

    +

    Prior Art and References

    This RFC builds extensively on the available ideas put forward in RFC-1.

    Additionally, I want to express a special thanks to Samuel Haefner and Shahar Dobzinski for fruitful discussions and helping me structure my thoughts.

    -

    Unresolved Questions

    +

    Unresolved Questions

    The technical feasability needs to be assessed.

    (source)

    Table of Contents

    @@ -5024,9 +5148,9 @@ This can be unified and simplified by moving both parts of the logic into the ru AuthorsChaosDAO -

    Summary

    +

    Summary

    This RFC proposes a change to the duration of the confirmation period for the treasurer track from 3 hours to at least 48 hours.

    -

    Motivation

    +

    Motivation

    Track parameters for Polkadot OpenGov should be configured in a way that their "difficulty" increases relative to the power associated with their respective origin. When we look at the confirmation periods for treasury based tracks, we can see that this is clearly the case - with the one notable exception to the trend being the treasurer track:

    @@ -5039,7 +5163,7 @@ This can be unified and simplified by moving both parts of the logic into the ru

    The confirmation period is one of the last lines of defence for the collective Polkadot stakeholders to react to a potentially bad referendum and vote NAY in order for its confirmation period to be aborted.

    Since the power / privilege level of the treasurer track is greater than that of the the big spender track – their confirmation period should be either equal, or the treasurer track's should be higher (note: currently the big spender track has a longer confirmation period than even the root track).

    -

    Stakeholders

    +

    Stakeholders

    The primary stakeholders of this RFC are:

    • DOT token holders – as this affects the protocol's treasury
    • @@ -5049,17 +5173,17 @@ This can be unified and simplified by moving both parts of the logic into the ru
    • Leemo - expressed interest to change this parameter
    • Paradox - expressed interest to change this parameter
    -

    Explanation

    +

    Explanation

    This RFC proposes to change the duration of the confirmation period for the treasurer track. In order to achieve that, the confirm_period parameter for the treasurer track in runtime/polkadot/src/governance/tracks.rs must be changed.

    Currently it is set to confirm_period: 3 * HOURS

    It should be changed to confirm_period: 48 * HOURS as a minimum.

    It may make sense for it to be changed to a value greater than 48 hours since the treasurer track has more power than the big spender track (48 hour confirmation period); however, the root track's confirmation period is 24 hours. 48 hours may be on the upper bounds of a trade-off between security and flexibility.

    -

    Drawbacks

    +

    Drawbacks

    The drawback of changing the treasurer track's confirmation period would be that the lifecycle of a referendum submitted on the treasurer track would ultimately be longer. However, the security of the protocol and its treasury should take priority here.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    This change will enhance / improve the security of the protocol as it relates to its treasury. The confirmation period is one of the last lines of defence for the collective Polkadot stakeholders to react to a potentially bad referendum and vote NAY in order for its confirmation period to be aborted. It makes sense for the treasurer track's confirmation period duration to be either equal to, or higher than, the big spender track confirmation period.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    This is a simple change (code wise) which should not affect the performance of the Polkadot protocol, outside of increasing the duration of the confirmation period on the treasurer track.

    Ergonomics & Compatibility

    If the proposal alters exposed interfaces to developers or end-users, which types of usage patterns have been optimized for?

    @@ -5069,14 +5193,14 @@ This can be unified and simplified by moving both parts of the logic into the ru
  • Polkassembly - directly uses on-chain data via rpc to fetch trackInfo so the change will be automatically reflected.
  • SubSquare - scan script will update their app to the latest parameters and it will be automatically reflected in their app.
  • -

    Prior Art and References

    +

    Prior Art and References

    N/A

    -

    Unresolved Questions

    +

    Unresolved Questions

    The proposed change to the confirmation period duration for the treasurer track is to set it to 48 hours. This is equal to the current confirmation period for the big spender track.

    Typically it seems that track parameters increase in difficulty (duration, etc.) based on the power level of their associated origin.

    The longest confirmation period is that of the big spender, at 48 hours. There may be value in discussing whether or not the treasurer track confirmation period should be longer than 48 hours – a discussion of the trade-offs between security vs flexibility/agility.

    As a side note, the root track confirmation period is 24 hours.

    - +

    This RFC hopefully reminds the greater Polkadot community that it is possible to submit changes to the parameters of Polkadot OpenGov, and the greater protocol as a whole through the RFC process.

    (source)

    Table of Contents

    @@ -5108,7 +5232,7 @@ This can be unified and simplified by moving both parts of the logic into the ru
    Track DescriptionConfirmation Period Duration
    Small Tipper10 Min
    ChaosDAO
    -

    Summary

    +

    Summary

    This RFC proposes to make modifications to voting power delegations as part of the Conviction Voting pallet. The changes being proposed include:

    1. Allow a Delegator to vote independently of their Delegate if they so desire.
    2. @@ -5116,7 +5240,7 @@ This can be unified and simplified by moving both parts of the logic into the ru
    3. Make a change so that when a delegate votes abstain their delegated votes also vote abstain.
    4. Allow a Delegator to delegate/ undelegate their votes for all tracks with a single call.
    -

    Motivation

    +

    Motivation

    It has become clear since the launch of OpenGov that there are a few common tropes which pop up time and time again:

    1. The frequency of referenda is often too high for network participants to have sufficient time to review, comprehend, and ultimately vote on each individual referendum. This means that these network participants end up being inactive in on-chain governance.
    2. @@ -5124,13 +5248,13 @@ This can be unified and simplified by moving both parts of the logic into the ru
    3. Delegating votes for all tracks currently requires long batched calls which result in high fees for the Delegator - resulting in a reluctance from many to delegate their votes.

    We believe (based on feedback from token holders with a larger stake in the network) that if there were some changes made to delegation mechanics, these larger stake holders would be more likely to delegate their voting power to active network participants – thus greatly increasing the support turnout.

    -

    Stakeholders

    +

    Stakeholders

    The primary stakeholders of this RFC are:

    • The Polkadot Technical Fellowship who will have to research and implement the technical aspects of this RFC
    • DOT token holders in general
    -

    Explanation

    +

    Explanation

    This RFC proposes to make 4 changes to the convictionVoting pallet logic in order to improve the user experience of those delegating their voting power to another account.

    1. @@ -5146,21 +5270,21 @@ This can be unified and simplified by moving both parts of the logic into the ru

      Allow a Delegator to delegate/ undelegate their votes for all tracks with a single call - in order to delegate votes across all tracks, a user must batch 15 calls - resulting in high costs for delegation. A single call for delegate_all/ undelegate_all would reduce the complexity and therefore costs of delegations considerably for prospective Delegators.

    -

    Drawbacks

    +

    Drawbacks

    We do not foresee any drawbacks by implementing these changes. If anything we believe that this should help to increase overall voter turnout (via the means of delegation) which we see as a net positive.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    We feel that the Polkadot Technical Fellowship would be the most competent collective to identify the testing requirements for the ideas presented in this RFC.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    This change may add extra chain storage requirements on Polkadot, especially with respect to nested delegations.

    Ergonomics & Compatibility

    The change to add nested delegations may affect governance interfaces such as Nova Wallet who will have to apply changes to their indexers to support nested delegations. It may also affect the Polkadot Delegation Dashboard as well as Polkassembly & SubSquare.

    We want to highlight the importance for ecosystem builders to create a mechanism for indexers and wallets to be able to understand that changes have occurred such as increasing the pallet version, etc.

    -

    Prior Art and References

    +

    Prior Art and References

    N/A

    -

    Unresolved Questions

    +

    Unresolved Questions

    N/A

    - +

    Additionally we would like to re-open the conversation about the potential for there to be free delegations. This was discussed by Dr Gavin Wood at Sub0 2022 and we feel like this would go a great way towards increasing the amount of network participants that are delegating: https://youtu.be/hSoSA6laK3Q?t=526

    Overall, we strongly feel that delegations are a great way to increase voter turnout, and the ideas presented in this RFC would hopefully help in that aspect.

    (source)

    @@ -5190,9 +5314,9 @@ This can be unified and simplified by moving both parts of the logic into the ru AuthorsSebastian Kunert -

    Summary

    +

    Summary

    This RFC proposes a new host function for parachains, storage_proof_size. It shall provide the size of the currently recorded storage proof to the runtime. Runtime authors can use the proof size to improve block utilization by retroactively reclaiming unused storage weight.

    -

    Motivation

    +

    Motivation

    The number of extrinsics that are included in a parachain block is limited by two constraints: execution time and proof size. FRAME weights cover both concepts, and block-builders use them to decide how many extrinsics to include in a block. However, these weights are calculated ahead of time by benchmarking on a machine with reference hardware. The execution-time properties of the state-trie and its storage items are unknown at benchmarking time. Therefore, we make some assumptions about the state-trie:

    • Trie Depth: We assume a trie depth to account for intermediary nodes.
    • @@ -5201,12 +5325,12 @@ This can be unified and simplified by moving both parts of the logic into the ru

      These pessimistic assumptions lead to an overestimation of storage weight, negatively impacting block utilization on parachains.

      In addition, the current model does not account for multiple accesses to the same storage items. While these repetitive accesses will not increase storage-proof size, the runtime-side weight monitoring will account for them multiple times. Since the proof size is completely opaque to the runtime, we can not implement retroactive storage weight correction.

      A solution must provide a way for the runtime to track the exact storage-proof size consumed on a per-extrinsic basis.

      -

      Stakeholders

      +

      Stakeholders

      • Parachain Teams: They MUST include this host function in their runtime and node.
      • Light-client Implementors: They SHOULD include this host function in their runtime and node.
      -

      Explanation

      +

      Explanation

      This RFC proposes a new host function that exposes the storage-proof size to the runtime. As a result, runtimes can implement storage weight reclaiming mechanisms that improve block utilization.

      This RFC proposes the following host function signature:

      #![allow(unused)]
      @@ -5214,14 +5338,14 @@ This can be unified and simplified by moving both parts of the logic into the ru
       fn ext_storage_proof_size_version_1() -> u64;
       }

      The host function MUST return an unsigned 64-bit integer value representing the current proof size. In block-execution and block-import contexts, this function MUST return the current size of the proof. To achieve this, parachain node implementors need to enable proof recording for block imports. In other contexts, this function MUST return 18446744073709551615 (u64::MAX), which represents disabled proof recording.

      -

      Performance, Ergonomics, and Compatibility

      -

      Performance

      +

      Performance, Ergonomics, and Compatibility

      +

      Performance

      Parachain nodes need to enable proof recording during block import to correctly implement the proposed host function. Benchmarking conducted with balance transfers has shown a performance reduction of around 0.6% when proof recording is enabled.

      -

      Ergonomics

      +

      Ergonomics

      The host function proposed in this RFC allows parachain runtime developers to keep track of the proof size. Typical usage patterns would be to keep track of the overall proof size or the difference between subsequent calls to the host function.

      -

      Compatibility

      +

      Compatibility

      Parachain teams will need to include this host function to upgrade.

      -

      Prior Art and References

      +

      Prior Art and References

      • Pull Request including proposed host function: PoV Reclaim (Clawback) Node Side.
      • Issue with discussion: [FRAME core] Clawback PoV Weights For Dispatchables
      • @@ -5265,9 +5389,9 @@ This can be unified and simplified by moving both parts of the logic into the ru AuthorsSergej Sakac -

        Summary

        +

        Summary

        This RFC proposes a new model for a sustainable on-demand parachain registration, involving a smaller initial deposit and periodic rent payments. The new model considers that on-demand chains may be unregistered and later re-registered. The proposed solution also ensures a quick startup for on-demand chains on Polkadot in such cases.

        -

        Motivation

        +

        Motivation

        With the support of on-demand parachains on Polkadot, there is a need to explore a new, more cost-effective model for registering validation code. In the current model, the parachain manager is responsible for reserving a unique ParaId and covering the cost of storing the validation code of the parachain. These costs can escalate, particularly if the validation code is large. We need a better, sustainable model for registering on-demand parachains on Polkadot to help smaller teams deploy more easily.

        This RFC suggests a new payment model to create a more financially viable approach to on-demand parachain registration. In this model, a lower initial deposit is required, followed by recurring payments upon parachain registration.

        This new model will coexist with the existing one-time deposit payment model, offering teams seeking to deploy on-demand parachains on Polkadot a more cost-effective alternative.

        @@ -5281,11 +5405,11 @@ This can be unified and simplified by moving both parts of the logic into the ru
      • The solution MUST allow anyone to pay the rent.
      • The solution MUST prevent the removal of validation code if it could still be required for disputes or approval checking.
      • -

        Stakeholders

        +

        Stakeholders

        • Future Polkadot on-demand Parachains
        -

        Explanation

        +

        Explanation

        This RFC proposes a set of changes that will enable the new rent based approach to registering and storing validation code on-chain. The new model, compared to the current one, will require periodic rent payments. The parachain won't be pruned automatically if the rent is not paid, but by permitting anyone to prune the parachain and rewarding the caller, there will be an incentive for the removal of the validation code.

        On-demand parachains should still be able to utilize the current one-time payment model. However, given the size of the deposit required, it's highly likely that most on-demand parachains will opt for the new rent-based model.

        @@ -5392,27 +5516,27 @@ pub(super) type CheckedCodeHash<T: Config> = StorageMap<_, Twox64Concat, ParaId, ValidationCodeHash>; }

    To enable parachain re-registration, we should introduce a new extrinsic in the paras-registrar pallet that allows this. The logic of this extrinsic will be same as regular registration, with the distinction that it can be called by anyone, and the required deposit will be smaller since it only has to cover for the storage of the validation code.

    -

    Drawbacks

    +

    Drawbacks

    This RFC does not alter the process of reserving a ParaId, and therefore, it does not propose reducing it, even though such a reduction could be beneficial.

    Even though this RFC doesn't delve into the specifics of the configuration values for parachain registration but rather focuses on the mechanism, configuring it carelessly could lead to potential problems.

    Since the validation code hash and head data are not removed when the parachain is pruned but only when the deregister extrinsic is called, the T::DataDepositPerByte must be set to a higher value to create a strong enough incentive for removing it from the state.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    The implementation of this RFC will be tested on Rococo first.

    Proper research should be conducted on setting the configuration values of the new system since these values can have great impact on the network.

    An audit is required to ensure the implementation's correctness.

    The proposal introduces no new privacy concerns.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    This RFC should not introduce any performance impact.

    -

    Ergonomics

    +

    Ergonomics

    This RFC does not affect the current parachains, nor the parachains that intend to use the one-time payment model for parachain registration.

    -

    Compatibility

    +

    Compatibility

    This RFC does not break compatibility.

    -

    Prior Art and References

    +

    Prior Art and References

    Prior discussion on this topic: https://github.com/paritytech/polkadot-sdk/issues/1796

    -

    Unresolved Questions

    +

    Unresolved Questions

    None at this time.

    - +

    As noted in this GitHub issue, we want to raise the per-byte cost of on-chain data storage. However, a substantial increase in this cost would make it highly impractical for on-demand parachains to register on Polkadot. This RFC offers an alternative solution for on-demand parachains, ensuring that the per-byte cost increase doesn't overly burden the registration process.

    (source)

    @@ -5446,24 +5570,24 @@ This RFC offers an alternative solution for on-demand parachains, ensuring that AuthorsBastian Köcher -

    Summary

    +

    Summary

    When rotating/generating the SessionKeys of a node, the node calls into the runtime using the SessionKeys::generate_session_keys runtime api. This runtime api function needs to be changed to add an extra parameter owner and to change the return value to also include the proof of ownership. The owner should be the account id of the account setting the SessionKeys on chain to allow the on chain logic the verification of the proof. The on chain logic is then able to proof the possession of the private keys of the SessionKeys using the proof.

    -

    Motivation

    +

    Motivation

    When a user sets new SessionKeys on chain the chain can currently not ensure that the user actually has control over the private keys of the SessionKeys. With the RFC applied the chain is able to ensure that the user actually is in possession of the private keys.

    -

    Stakeholders

    +

    Stakeholders

    • Polkadot runtime implementors
    • Polkadot node implementors
    • Validator operators
    -

    Explanation

    +

    Explanation

    We are first going to explain the proof format being used:

    #![allow(unused)]
     fn main() {
    @@ -5496,26 +5620,26 @@ actual exported function signature looks like:

    already gets the proof passed as Vec<u8>. This proof needs to be decoded to the actual Proof type as explained above. The proof and the SCALE encoded account_id of the sender are used to verify the ownership of the SessionKeys.

    -

    Drawbacks

    +

    Drawbacks

    Validator operators need to pass the their account id when rotating their session keys in a node. This will require updating some high level docs and making users familiar with the slightly changed ergonomics.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    Testing of the new changes is quite easy as it only requires passing an appropriate owner for the current testing context. The changes to the proof generation and verification got audited to ensure they are correct.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    Does not have any impact on the overall performance, only setting SessionKeys will require more weight.

    -

    Ergonomics

    +

    Ergonomics

    If the proposal alters exposed interfaces to developers or end-users, which types of usage patterns have been optimized for?

    -

    Compatibility

    +

    Compatibility

    Introduces a new version of the SessionKeys runtime api. Thus, nodes should be updated before a runtime is enacted that contains these changes otherwise they will fail to generate session keys.

    -

    Prior Art and References

    +

    Prior Art and References

    None.

    -

    Unresolved Questions

    +

    Unresolved Questions

    None.

    - +

    Substrate implementation of the RFC.

    (source)

    Table of Contents

    @@ -5548,16 +5672,16 @@ a runtime is enacted that contains these changes otherwise they will fail to gen AuthorsPierre Krieger -

    Summary

    +

    Summary

    Rather than enforce a limit to the total memory consumption on the client side by loading the value at :heappages, enforce that limit on the runtime side.

    -

    Motivation

    +

    Motivation

    From the early days of Substrate up until recently, the runtime was present in two forms: the wasm runtime (wasm bytecode passed through an interpreter) and the native runtime (native code directly run by the client).

    Since the wasm runtime has a lower amount of available memory (4 GiB maximum) compared to the native runtime, and in order to ensure sure that the wasm and native runtimes always produce the same outcome, it was necessary to clamp the amount of memory available to both runtimes to the same value.

    In order to achieve this, a special storage key (a "well-known" key) :heappages was introduced and represents the number of "wasm pages" (one page equals 64kiB) of memory that are available to the memory allocator of the runtimes. If this storage key is absent, it defaults to 2048, which is 128 MiB.

    The native runtime has since then been disappeared, but the concept of "heap pages" still exists. This RFC proposes a simplification to the design of Polkadot by removing the concept of "heap pages" as is currently known, and proposes alternative ways to achieve the goal of limiting the amount of memory available.

    -

    Stakeholders

    +

    Stakeholders

    Client implementers and low-level runtime developers.

    -

    Explanation

    +

    Explanation

    This RFC proposes the following changes to the client:

    • The client no longer considers :heappages as special.
    • @@ -5583,141 +5707,28 @@ a runtime is enacted that contains these changes otherwise they will fail to gen

    Each parachain can choose the option that they prefer, but the author of this RFC strongly suggests either option C or B.

    -

    Drawbacks

    +

    Drawbacks

    In case of path A, there is one situation where the behaviour pre-RFC is not equivalent to the one post-RFC: when a host function that performs an allocation (for example ext_storage_get) is called, without this RFC this allocation might fail due to reaching the maximum heap pages, while after this RFC this will always succeed. This is most likely not a problem, as storage values aren't supposed to be larger than a few megabytes at the very maximum.

    In the unfortunate event where the runtime runs out of memory, path B would make it more difficult to relax the memory limit, as we would need to re-upload the entire Wasm, compared to updating only :heappages in path A or before this RFC. In the case where the runtime runs out of memory only in the specific event where the Wasm runtime is modified, this could brick the chain. However, this situation is no different than the thousands of other ways that a bug in the runtime can brick a chain, and there's no reason to be particularily worried about this situation in particular.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    This RFC would reduce the chance of a consensus issue between clients. The :heappages are a rather obscure feature, and it is not clear what happens in some corner cases such as the value being too large (error? clamp?) or malformed. This RFC would completely erase these questions.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    -

    In case of path A, it is unclear how performances would be affected. Path A consists in moving client-side operations to the runtime without changing these operations, and as such performance differences are expected to be minimal. Overall, we're talking about one addition/subtraction per malloc and per free, so this is more than likely completely negligible.

    -

    In case of path B and C, the performance gain would be a net positive, as this RFC strictly removes things.

    -

    Ergonomics

    -

    This RFC would isolate the client and runtime more from each other, making it a bit easier to reason about the client or the runtime in isolation.

    -

    Compatibility

    -

    Not a breaking change. The runtime-side changes can be applied immediately (without even having to wait for changes in the client), then as soon as the runtime is updated, the client can be updated without any transition period. One can even consider updating the client before the runtime, as it corresponds to path C.

    -

    Prior Art and References

    -

    None.

    -

    Unresolved Questions

    -

    None.

    - -

    This RFC follows the same path as https://github.com/polkadot-fellows/RFCs/pull/4 by scoping everything related to memory allocations to the runtime.

    -

    (source)

    -

    Table of Contents

    - -

    RFC-0059: Add a discovery mechanism for nodes based on their capabilities

    -
    - - - -
    Start Date2023-12-18
    DescriptionNodes having certain capabilities register themselves in the DHT to be discoverable
    AuthorsPierre Krieger
    -
    -

    Summary

    -

    This RFC proposes to make the mechanism of RFC #8 more generic by introducing the concept of "capabilities".

    -

    Implementations can implement certain "capabilities", such as serving old block headers or being a parachain bootnode.

    -

    The discovery mechanism of RFC #8 is extended to be able to discover nodes of specific capabilities.

    -

    Motivation

    -

    The Polkadot peer-to-peer network is made of nodes. Not all these nodes are equal. Some nodes store only the headers of recently blocks, some nodes store all the block headers and bodies since the genesis, some nodes store the storage of all blocks since the genesis, and so on.

    -

    It is currently not possible to know ahead of time (without connecting to it and asking) which nodes have which data available, and it is not easily possible to build a list of nodes that have a specific piece of data available.

    -

    If you want to download for example the header of block 500, you have to connect to a randomly-chosen node, ask it for block 500, and if it says that it doesn't have the block, disconnect and try another randomly-chosen node. -In certain situations such as downloading the storage of old blocks, nodes that have the information are relatively rare, and finding through trial and error a node that has the data can take a long time.

    -

    This RFC attempts to solve this problem by giving the possibility to build a list of nodes that are capable of serving specific data.

    -

    Stakeholders

    -

    Low-level client developers. -People interested in accessing the archive of the chain.

    -

    Explanation

    -

    Reading RFC #8 first might help with comprehension, as this RFC is very similar.

    -

    Please keep in mind while reading that everything below applies for both relay chains and parachains, except mentioned otherwise.

    -

    Capabilities

    -

    This RFC defines a list of so-called capabilities:

    -
      -
    • Head of chain provider. An implementation with this capability must be able to serve to other nodes block headers, block bodies, justifications, calls proofs, and storage proofs of "recent" (see below) blocks, and, for relay chains, to serve to other nodes warp sync proofs where the starting block is a session change block and must participate in Grandpa and Beefy gossip.
    • -
    • History provider. An implementation with this capability must be able to serve to other nodes block headers and block bodies of any block since the genesis, and must be able to serve to other nodes justifications of any session change block since the genesis up until and including their currently finalized block.
    • -
    • Archive provider. This capability is a superset of History provider. In addition to the requirements of History provider, an implementation with this capability must be able to serve call proofs and storage proof requests of any block since the genesis up until and including their currently finalized block.
    • -
    • Parachain bootnode (only for relay chains). An implementation with this capability must be able to serve the network request described in RFC 8.
    • -
    -

    In the context of the head of chain provider, the word "recent" means: any not-finalized-yet block that is equal to or an ancestor of a block that it has announced through a block announce, and any finalized block whose height is superior to its current finalized block minus 16. -This does not include blocks that have been pruned because they're not a descendant of its current finalized block. In other words, blocks that aren't a descendant of the current finalized block can be thrown away. -A gap of blocks is required due to race conditions: when a node finalizes a block, it takes some time for its peers to be made aware of this, during which they might send requests concerning older blocks. The exact gap is arbitrary.

    -

    Substrate is currently by default a head of chain provider provider. After it has finished warp syncing, it downloads the list of old blocks, after which it becomes a history provider. -If Substrate is instead configured as an archive node, then it downloads the state of all blocks since the genesis, after which it becomes an archive provider, history provider, and head of chain provider. -If blocks pruning is enabled and the chain is a relay chain, then Substrate unfortunately doesn't implement any of these capabilities, not even head of chain provider. This is considered as a bug that should be fixed, see https://github.com/paritytech/polkadot-sdk/issues/2733.

    -

    DHT provider registration

    -

    This RFC heavily relies on the functionalities of the Kademlia DHT already in use by Polkadot. You can find a link to the specification here.

    -

    Implementations that have the history provider capability should register themselves as providers under the key sha256(concat("history", randomness)).

    -

    Implementations that have the archive provider capability should register themselves as providers under the key sha256(concat("archive", randomness)).

    -

    Implementations that have the parachain bootnode capability should register themselves as provider under the key sha256(concat(scale_compact(para_id), randomness)), as described in RFC 8.

    -

    "Register themselves as providers" consists in sending ADD_PROVIDER requests to nodes close to the key, as described in the Content provider advertisement section of the specification.

    -

    The value of randomness can be found in the randomness field when calling the BabeApi_currentEpoch function.

    -

    In order to avoid downtimes when the key changes, nodes should also register themselves as a secondary key that uses a value of randomness equal to the randomness field when calling BabeApi_nextEpoch.

    -

    Implementers should be aware that their implementation of Kademlia might already hash the key before XOR'ing it. The key is not meant to be hashed twice.

    -

    Implementations must not register themselves if they don't fulfill the capability yet. For example, a node configured to be an archive node but that is still building its archive state in the background must register itself only after it has finished building its archive.

    -

    Secondary DHTs

    -

    Implementations that have the history provider capability must also participate in a secondary DHT that comprises only of nodes with that capability. The protocol name of that secondary DHT must be /<genesis-hash>/kad/history.

    -

    Similarly, implementations that have the archive provider capability must also participate in a secondary DHT that comprises only of nodes with that capability and whose protocol name is /<genesis-hash>/kad/archive.

    -

    Just like implementations must not register themselves if they don't fulfill their capability yet, they must also not participate in the secondary DHT if they don't fulfill their capability yet.

    -

    Head of the chain providers

    -

    Implementations that have the head of the chain provider capability do not register themselves as providers, but instead are the nodes that participate in the main DHT. In other words, they are the nodes that serve requests of the /<genesis_hash>/kad protocol.

    -

    Any implementation that isn't a head of the chain provider (read: light clients) must not participate in the main DHT. This is already presently the case.

    -

    Implementations must not participate in the main DHT if they don't fulfill the capability yet. For example, a node that is still in the process of warp syncing must not participate in the main DHT. However, assuming that warp syncing doesn't last more than a few seconds, it is acceptable to ignore this requirement in order to avoid complicating implementations too much.

    -

    Drawbacks

    -

    None that I can see.

    -

    Testing, Security, and Privacy

    -

    The content of this section is basically the same as the one in RFC 8.

    -

    This mechanism doesn't add or remove any security by itself, as it relies on existing mechanisms.

    -

    Due to the way Kademlia works, it would become the responsibility of the 20 Polkadot nodes whose sha256(peer_id) is closest to the key (described in the explanations section) to store the list of nodes that have specific capabilities. -Furthermore, when a large number of providers are registered, only the providers closest to the key are kept, up to a certain implementation-defined limit.

    -

    For this reason, an attacker can abuse this mechanism by randomly generating libp2p PeerIds until they find the 20 entries closest to the key representing the target capability. They are then in control of the list of nodes with that capability. While doing this can in no way be actually harmful, it could lead to eclipse attacks.

    -

    Because the key changes periodically and isn't predictable, and assuming that the Polkadot DHT is sufficiently large, it is not realistic for an attack like this to be maintained in the long term.

    Performance, Ergonomics, and Compatibility

    Performance

    -

    The DHT mechanism generally has a low overhead, especially given that publishing providers is done only every 24 hours.

    -

    Doing a Kademlia iterative query then sending a provider record shouldn't take more than around 50 kiB in total of bandwidth for the parachain bootnode.

    -

    Assuming 1000 nodes with a specific capability, the 20 Polkadot full nodes corresponding to that capability will each receive a sudden spike of a few megabytes of networking traffic when the key rotates. Again, this is relatively negligible. If this becomes a problem, one can add a random delay before a node registers itself to be the provider of the key corresponding to BabeApi_next_epoch.

    -

    Maybe the biggest uncertainty is the traffic that the 20 Polkadot full nodes will receive from light clients that desire knowing the nodes with a capability. If this every becomes a problem, this value of 20 is an arbitrary constant that can be increased for more redundancy.

    +

    In case of path A, it is unclear how performances would be affected. Path A consists in moving client-side operations to the runtime without changing these operations, and as such performance differences are expected to be minimal. Overall, we're talking about one addition/subtraction per malloc and per free, so this is more than likely completely negligible.

    +

    In case of path B and C, the performance gain would be a net positive, as this RFC strictly removes things.

    Ergonomics

    -

    Irrelevant.

    +

    This RFC would isolate the client and runtime more from each other, making it a bit easier to reason about the client or the runtime in isolation.

    Compatibility

    -

    Irrelevant.

    +

    Not a breaking change. The runtime-side changes can be applied immediately (without even having to wait for changes in the client), then as soon as the runtime is updated, the client can be updated without any transition period. One can even consider updating the client before the runtime, as it corresponds to path C.

    Prior Art and References

    -

    Unknown.

    +

    None.

    Unresolved Questions

    -

    While it fundamentally doesn't change much to this RFC, using BabeApi_currentEpoch and BabeApi_nextEpoch might be inappropriate. I'm not familiar enough with good practices within the runtime to have an opinion here. Should it be an entirely new pallet?

    +

    None.

    -

    This RFC would make it possible to reliably discover archive nodes, which would make it possible to reliably send archive node requests, something that isn't currently possible. This could solve the problem of finding archive RPC node providers by migrating archive-related request to using the native peer-to-peer protocol rather than JSON-RPC.

    -

    If we ever decide to break backwards compatibility, we could divide the "history" and "archive" capabilities in two, between nodes capable of serving older blocks and nodes capable of serving newer blocks. -We could even add to the peer-to-peer network nodes that are only capable of serving older blocks (by reading from a database) but do not participate in the head of the chain, and that just exist for historical purposes.

    +

    This RFC follows the same path as https://github.com/polkadot-fellows/RFCs/pull/4 by scoping everything related to memory allocations to the runtime.

    diff --git a/proposed/000x-lowering-deposits-assethub.html b/proposed/000x-lowering-deposits-assethub.html index 1f144d7..619861c 100644 --- a/proposed/000x-lowering-deposits-assethub.html +++ b/proposed/000x-lowering-deposits-assethub.html @@ -90,7 +90,7 @@ diff --git a/proposed/0026-sassafras-consensus.html b/proposed/0026-sassafras-consensus.html index 7deea88..a52b031 100644 --- a/proposed/0026-sassafras-consensus.html +++ b/proposed/0026-sassafras-consensus.html @@ -90,7 +90,7 @@ diff --git a/proposed/0034-xcm-absolute-location-account-derivation.html b/proposed/0034-xcm-absolute-location-account-derivation.html index 48ea0d7..5b6e5e2 100644 --- a/proposed/0034-xcm-absolute-location-account-derivation.html +++ b/proposed/0034-xcm-absolute-location-account-derivation.html @@ -90,7 +90,7 @@ diff --git a/proposed/0042-extrinsics-state-version.html b/proposed/0042-extrinsics-state-version.html index 7bf9c16..d51a58b 100644 --- a/proposed/0042-extrinsics-state-version.html +++ b/proposed/0042-extrinsics-state-version.html @@ -90,7 +90,7 @@ diff --git a/proposed/0046-metadata-for-offline-signers.html b/proposed/0046-metadata-for-offline-signers.html index e302642..f96f954 100644 --- a/proposed/0046-metadata-for-offline-signers.html +++ b/proposed/0046-metadata-for-offline-signers.html @@ -90,7 +90,7 @@ @@ -195,9 +195,6 @@
  • Merging protocol
  • Complete Binary Merkle Tree construction protocol
  • Digest
  • -
  • Shortening
  • -
  • Transmission
  • -
  • Offline verification
  • Chain verification
  • @@ -328,11 +325,11 @@ Node 0 is root
    1. The metadata is converted into lean modular form (vector of chunks)
    2. A Merkle tree is constructed from the metadata chunks
    3. -
    4. A root of tree (as a left element) is merged with Metadata Descriptor (as a right element)
    5. +
    6. A root of tree is merged with the hash of the MetadataDescriptor
    7. Resulting value is a constant to be included in additionalSigned to prove that the metadata seen by cold device is genuine

    Metadata modularization

    -

    Structure of types in shortened metadata exactly matches structure of types in scale-info, but doc field is always empty

    +

    Structure of types in shortened metadata exactly matches structure of types in scale-info at MetadataV14 state, but doc field is always empty

    struct Type {
       path: Path, // vector of strings
       type_params: Vec<TypeParams>,
    @@ -386,6 +383,26 @@ modularized_registry.sort(|a, b| {
     
  • Right node and then left node is popped from the front of the nodes queue and merged; the result is sent to the end of the queue.
  • Step (4) is repeated until only one node remains; this is tree root.
  • +
    queue = empty_queue
    +
    +while (leaves.length>1) {
    +  right = leaves.pop_last
    +  left = leaves.pop_last
    +  queue.push_back(merge(left, right))
    +}
    +
    +if leaves.length == 1 {
    +  queue.push_front(leaves.last)
    +}
    +
    +while queue.len() > 1 {
    +  right = queue.pop_front
    +  left = queue.pop_front
    +  queue.push_back(merge(left, right))
    +}
    +
    +return queue.pop
    +
    Resulting tree for metadata consisting of 5 nodes (numbered from 0 to 4):
     
            root
    @@ -403,14 +420,8 @@ modularized_registry.sort(|a, b| {
     
  • Root hash of this tree (left) is merged with metadata descriptor blake3 hash (right); this is metadata digest.
  • Version number and corresponding resulting metadata digest MUST be included into Signed Extensions as specified in Chain Verification section below.

    -

    Shortening

    -

    For shortening, an attempt to decode transaction completely using provided metadata is performed with the same algorithm that would be used on the cold side. All chunks are associated with their leaf indices. An example of this protocol is proposed in metadata-shortener that is based on substrate-parser decoding protocol; any decoding protocol could be used here as long as cold signer's design finds it appropriate for given security model.

    -

    Transmission

    -

    Shortened metadata chunks MAY be trasmitted into cold device together with Merkle proof in its entirety or in parts, depending on memory capabilities of the cold device and it ability to reconstruct larger fraction of tree. This document does not specify the manner of transmission. The order of metadata chunks MAY be arbitrary, the only requirement is that indices of leaf nodes in Merkle tree corresponding to chunks MUST be communicated. Community MAY handle proof format standartization independently.

    -

    Offline verification

    -

    The transmitted metadata chunks are hashed together with proof lemmas to obtain root that MAY be transmitted along with the rest of payload. Verification that the root transmitted with message matches with calculated root is optional; the transmitted root SHOULD NOT be used in signature, calculated root MUST be used; however, there is no mechanism to enforce this - it should be done during cold signers code audit.

    Chain verification

    -

    The root of metadata computed by cold device MAY be included into Signed Extensions; this way the transaction will pass as valid iff hash of metadata as seen by cold storage device is identical to consensus hash of metadata, ensuring fair signing protocol.

    +

    The root of metadata computed by cold device MAY be included into Signed Extensions; if it is included, the transaction will pass as valid iff hash of metadata as seen by cold storage device is identical to consensus hash of metadata, ensuring fair signing protocol.

    The Signed Extension representing metadata digest is a single byte representing both digest vaule inclusion and shortening protocol version; this MUST be included in Signed Extensions set. Depending on its value, a digest value is included as additionalSigned to signature computation according to following specification:

    diff --git a/proposed/0047-assignment-of-availability-chunks.html b/proposed/0047-assignment-of-availability-chunks.html index 3575ac1..bce477f 100644 --- a/proposed/0047-assignment-of-availability-chunks.html +++ b/proposed/0047-assignment-of-availability-chunks.html @@ -90,7 +90,7 @@ @@ -467,7 +467,7 @@ dispute scenarios.

    - @@ -481,7 +481,7 @@ dispute scenarios.

    - diff --git a/stale/0059-nodes-capabilities-discovery.html b/proposed/0059-nodes-capabilities-discovery.html similarity index 84% rename from stale/0059-nodes-capabilities-discovery.html rename to proposed/0059-nodes-capabilities-discovery.html index 8d4bfb3..99a8394 100644 --- a/stale/0059-nodes-capabilities-discovery.html +++ b/proposed/0059-nodes-capabilities-discovery.html @@ -90,7 +90,7 @@ @@ -292,10 +292,13 @@ We could even add to the peer-to-peer network nodes that are only capable of ser @@ -303,10 +306,13 @@ We could even add to the peer-to-peer network nodes that are only capable of ser diff --git a/proposed/0061-allocator-inside-of-runtime.html b/proposed/0061-allocator-inside-of-runtime.html index 030ed12..dda9c5b 100644 --- a/proposed/0061-allocator-inside-of-runtime.html +++ b/proposed/0061-allocator-inside-of-runtime.html @@ -90,7 +90,7 @@ @@ -294,7 +294,7 @@ For the first year, we SHALL disable the v1 by default, and enable it by default
    signed extension valuedigest valuecomment
    0x00digest is not included