diff --git a/404.html b/404.html index 7ace199..478e623 100644 --- a/404.html +++ b/404.html @@ -91,7 +91,7 @@ diff --git a/approved/0001-agile-coretime.html b/approved/0001-agile-coretime.html index 649767f..23a3352 100644 --- a/approved/0001-agile-coretime.html +++ b/approved/0001-agile-coretime.html @@ -90,7 +90,7 @@ diff --git a/approved/0005-coretime-interface.html b/approved/0005-coretime-interface.html index d810a1f..d21806e 100644 --- a/approved/0005-coretime-interface.html +++ b/approved/0005-coretime-interface.html @@ -90,7 +90,7 @@ diff --git a/approved/0007-system-collator-selection.html b/approved/0007-system-collator-selection.html index 2cc546f..d55d3be 100644 --- a/approved/0007-system-collator-selection.html +++ b/approved/0007-system-collator-selection.html @@ -90,7 +90,7 @@ diff --git a/approved/0008-parachain-bootnodes-dht.html b/approved/0008-parachain-bootnodes-dht.html index 3a0d7a0..3541162 100644 --- a/approved/0008-parachain-bootnodes-dht.html +++ b/approved/0008-parachain-bootnodes-dht.html @@ -90,7 +90,7 @@ diff --git a/approved/0010-burn-coretime-revenue.html b/approved/0010-burn-coretime-revenue.html index 815f165..1a59f44 100644 --- a/approved/0010-burn-coretime-revenue.html +++ b/approved/0010-burn-coretime-revenue.html @@ -90,7 +90,7 @@ diff --git a/approved/0012-process-for-adding-new-collectives.html b/approved/0012-process-for-adding-new-collectives.html index c602c91..8727935 100644 --- a/approved/0012-process-for-adding-new-collectives.html +++ b/approved/0012-process-for-adding-new-collectives.html @@ -90,7 +90,7 @@ diff --git a/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html b/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html index 4b4d2f2..adeb1c0 100644 --- a/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html +++ b/approved/0013-prepare-blockbuilder-and-core-runtime-apis-for-mbms.html @@ -90,7 +90,7 @@ diff --git a/approved/0014-improve-locking-mechanism-for-parachains.html b/approved/0014-improve-locking-mechanism-for-parachains.html index 0c6cadc..eb57053 100644 --- a/approved/0014-improve-locking-mechanism-for-parachains.html +++ b/approved/0014-improve-locking-mechanism-for-parachains.html @@ -90,7 +90,7 @@ diff --git a/approved/0022-adopt-encointer-runtime.html b/approved/0022-adopt-encointer-runtime.html index cbfdfe2..06ad20a 100644 --- a/approved/0022-adopt-encointer-runtime.html +++ b/approved/0022-adopt-encointer-runtime.html @@ -90,7 +90,7 @@ diff --git a/approved/0026-sassafras-consensus.html b/approved/0026-sassafras-consensus.html index 92fd96b..8cf361f 100644 --- a/approved/0026-sassafras-consensus.html +++ b/approved/0026-sassafras-consensus.html @@ -90,7 +90,7 @@ diff --git a/approved/0032-minimal-relay.html b/approved/0032-minimal-relay.html index 8e9464d..7995389 100644 --- a/approved/0032-minimal-relay.html +++ b/approved/0032-minimal-relay.html @@ -90,7 +90,7 @@ diff --git a/approved/0042-extrinsics-state-version.html b/approved/0042-extrinsics-state-version.html index 830e44f..beeb3b5 100644 --- a/approved/0042-extrinsics-state-version.html +++ b/approved/0042-extrinsics-state-version.html @@ -90,7 +90,7 @@ diff --git a/approved/0043-storage-proof-size-hostfunction.html b/approved/0043-storage-proof-size-hostfunction.html index 58d30ed..3d6932a 100644 --- a/approved/0043-storage-proof-size-hostfunction.html +++ b/approved/0043-storage-proof-size-hostfunction.html @@ -90,7 +90,7 @@ diff --git a/approved/0045-nft-deposits-asset-hub.html b/approved/0045-nft-deposits-asset-hub.html index 126619f..7633033 100644 --- a/approved/0045-nft-deposits-asset-hub.html +++ b/approved/0045-nft-deposits-asset-hub.html @@ -90,7 +90,7 @@ diff --git a/approved/0047-assignment-of-availability-chunks.html b/approved/0047-assignment-of-availability-chunks.html index 1d72af0..30939b8 100644 --- a/approved/0047-assignment-of-availability-chunks.html +++ b/approved/0047-assignment-of-availability-chunks.html @@ -90,7 +90,7 @@ diff --git a/approved/0048-session-keys-runtime-api.html b/approved/0048-session-keys-runtime-api.html index 56df239..e5f475b 100644 --- a/approved/0048-session-keys-runtime-api.html +++ b/approved/0048-session-keys-runtime-api.html @@ -90,7 +90,7 @@ diff --git a/approved/0050-fellowship-salaries.html b/approved/0050-fellowship-salaries.html index 5c514ae..93616b8 100644 --- a/approved/0050-fellowship-salaries.html +++ b/approved/0050-fellowship-salaries.html @@ -90,7 +90,7 @@ diff --git a/approved/0056-one-transaction-per-notification.html b/approved/0056-one-transaction-per-notification.html index fa5c533..6485ca7 100644 --- a/approved/0056-one-transaction-per-notification.html +++ b/approved/0056-one-transaction-per-notification.html @@ -90,7 +90,7 @@ diff --git a/approved/0059-nodes-capabilities-discovery.html b/approved/0059-nodes-capabilities-discovery.html index a62daeb..e7300a1 100644 --- a/approved/0059-nodes-capabilities-discovery.html +++ b/approved/0059-nodes-capabilities-discovery.html @@ -90,7 +90,7 @@ diff --git a/approved/0078-merkleized-metadata.html b/approved/0078-merkleized-metadata.html index 484357a..334b9c4 100644 --- a/approved/0078-merkleized-metadata.html +++ b/approved/0078-merkleized-metadata.html @@ -90,7 +90,7 @@ diff --git a/approved/0084-general-transaction-extrinsic-format.html b/approved/0084-general-transaction-extrinsic-format.html index f02622c..eda681e 100644 --- a/approved/0084-general-transaction-extrinsic-format.html +++ b/approved/0084-general-transaction-extrinsic-format.html @@ -90,7 +90,7 @@ diff --git a/approved/0091-dht-record-creation-time.html b/approved/0091-dht-record-creation-time.html index b89f34b..a744f03 100644 --- a/approved/0091-dht-record-creation-time.html +++ b/approved/0091-dht-record-creation-time.html @@ -90,7 +90,7 @@ diff --git a/approved/0097-unbonding_queue.html b/approved/0097-unbonding_queue.html index 83866a4..a0e2059 100644 --- a/approved/0097-unbonding_queue.html +++ b/approved/0097-unbonding_queue.html @@ -90,7 +90,7 @@ diff --git a/approved/0099-transaction-extension-version.html b/approved/0099-transaction-extension-version.html index 0a5ffdc..efee74c 100644 --- a/approved/0099-transaction-extension-version.html +++ b/approved/0099-transaction-extension-version.html @@ -90,7 +90,7 @@ diff --git a/approved/0100-xcm-multi-type-asset-transfer.html b/approved/0100-xcm-multi-type-asset-transfer.html index 739cab8..73fc9e3 100644 --- a/approved/0100-xcm-multi-type-asset-transfer.html +++ b/approved/0100-xcm-multi-type-asset-transfer.html @@ -90,7 +90,7 @@ diff --git a/approved/0101-xcm-transact-remove-max-weight-param.html b/approved/0101-xcm-transact-remove-max-weight-param.html index 849b172..3959494 100644 --- a/approved/0101-xcm-transact-remove-max-weight-param.html +++ b/approved/0101-xcm-transact-remove-max-weight-param.html @@ -90,7 +90,7 @@ diff --git a/approved/0103-introduce-core-index-commitment.html b/approved/0103-introduce-core-index-commitment.html index 71e505d..18ec56b 100644 --- a/approved/0103-introduce-core-index-commitment.html +++ b/approved/0103-introduce-core-index-commitment.html @@ -90,7 +90,7 @@ diff --git a/approved/0105-xcm-improved-fee-mechanism.html b/approved/0105-xcm-improved-fee-mechanism.html index 4e0bd85..58c77f5 100644 --- a/approved/0105-xcm-improved-fee-mechanism.html +++ b/approved/0105-xcm-improved-fee-mechanism.html @@ -90,7 +90,7 @@ diff --git a/approved/0107-xcm-execution-hints.html b/approved/0107-xcm-execution-hints.html index 8ce5779..350e07c 100644 --- a/approved/0107-xcm-execution-hints.html +++ b/approved/0107-xcm-execution-hints.html @@ -90,7 +90,7 @@ diff --git a/approved/0108-xcm-remove-testnet-ids.html b/approved/0108-xcm-remove-testnet-ids.html index 2715781..8d980f6 100644 --- a/approved/0108-xcm-remove-testnet-ids.html +++ b/approved/0108-xcm-remove-testnet-ids.html @@ -90,7 +90,7 @@ diff --git a/approved/0122-alias-origin-on-asset-transfers.html b/approved/0122-alias-origin-on-asset-transfers.html index fb543f9..cc7ff5c 100644 --- a/approved/0122-alias-origin-on-asset-transfers.html +++ b/approved/0122-alias-origin-on-asset-transfers.html @@ -90,7 +90,7 @@ diff --git a/index.html b/index.html index f375523..1bad9bc 100644 --- a/index.html +++ b/index.html @@ -90,7 +90,7 @@ diff --git a/introduction.html b/introduction.html index f375523..1bad9bc 100644 --- a/introduction.html +++ b/introduction.html @@ -90,7 +90,7 @@ diff --git a/new/0130-JAM-Service-for-Validating-Ethereum-Optimistic-Rollups.html b/new/0130-JAM-Service-for-Validating-Ethereum-Optimistic-Rollups.html index f2c5832..d4a34b0 100644 --- a/new/0130-JAM-Service-for-Validating-Ethereum-Optimistic-Rollups.html +++ b/new/0130-JAM-Service-for-Validating-Ethereum-Optimistic-Rollups.html @@ -90,7 +90,7 @@ @@ -410,7 +410,7 @@ All rollups, whether built on Substrate or optimistic turned cynical, are commit - @@ -424,7 +424,7 @@ All rollups, whether built on Substrate or optimistic turned cynical, are commit - diff --git a/print.html b/print.html index 9321f9b..6f6fc2c 100644 --- a/print.html +++ b/print.html @@ -91,7 +91,7 @@ @@ -407,345 +407,6 @@ All rollups, whether built on Substrate or optimistic turned cynical, are commit

This JAM Service does not turn optimistic rollups into cynical rollups. A method to do so is not known.

Acknowledgements

We are deeply grateful for the ongoing encouragement and feedback from Polkadot heavyweights (Rob Habermeier, Alistair Stewart, Jeff Burdges, Bastian Kocher), Polkadot fellows, fellow JAM Implementers/Service Builders, and the broader community.

-

(source)

-

Table of Contents

- -

RFC-0000: Feature Name Here

-
- - - -
Start Date13 July 2024
DescriptionImplement off-chain parachain runtime upgrades
Authorseskimor
-
-

Summary

-

Change the upgrade process of a parachain runtime upgrade to become an off-chain -process with regards to the relay chain. Upgrades are still contained in -parachain blocks, but will no longer need to end up in relay chain blocks nor in -relay chain state.

-

Motivation

-

Having parachain runtime upgrades go through the relay chain has always been -seen as a scalability concern. Due to optimizations in statement -distribution and asynchronous backing it became less crucial and got -de-prioritized, the original issue can be found -here.

-

With the introduction of Agile Coretime and in general our efforts to reduce -barrier to entry more for Polkadot more, the issue becomes more relevant again: -We would like to reduce the required storage deposit for PVF registration, with -the aim to not only make it cheaper to run a parachain (bulk + on-demand -coretime), but also reduce the amount of capital required for the deposit. With -this we would hope for far more parachains to get registered, thousands -potentially even ten thousands. With so many PVFs registered, updates are -expected to become more frequent and even attacks on service quality for other -parachains would become a higher risk.

-

Stakeholders

- -

Explanation

-

The issues with on-chain runtime upgrades are:

-
    -
  1. Needlessly costly.
    2. -
  2. A single runtime upgrade more or less occupies an entire relay chain block, thus it -might affect also other parachains, especially if their candidates are also -not negligible due to messages for example or they want to uprade their -runtime at the same time.
    3. -
  3. The signalling of the parachain to notify the relay chain of an upcoming -runtime upgrade already contains the upgrade. Therefore the only way to rate -limit upgrades is to drop an already distributed update in the size of -megabytes: With the result that the parachain missed a block and more -importantly it will try again with the very next block, until it finally -succeeds. If we imagine to reduce capacity of runtime upgrades to let's say 1 -every 100 relay chain blocks, this results in lot's of wasted effort and lost -blocks.
    4. -
-

We discussed introducing a separate signalling before submitting the actual -runtime, but I think we should just go one step further and make upgrades fully -off-chain. Which also helps bringing down deposit costs in a secure way, as we -are also actually reducing costs for the network.

-

Introduce a new UMP message type RequestCodeUpgrade

-

As part of elastic scaling we are already planning to increase flexibility of UMP -messages, we can now use this to our advantage and introduce another UMP message:

-
#![allow(unused)]
-fn main() {
-enum UMPSignal {
-  // For elastic scaling
-  OnCore(CoreIndex),
-  // For off-chain upgrades
-  RequestCodeUpgrade(Hash),
-}
-}
-

We could also make that new message a regular XCM, calling an extrinsic on the -relay chain, but we will want to look into that message right after validation -on the backers on the node side, making a straight forward semantic message more -apt for the purpose.

-

Handle RequestCodeUpgrade on backers

-

We will introduce a new request/response protocol for both collators and -validators, with the following request/response:

-
#![allow(unused)]
-fn main() {
-struct RequestBlob {
-  blob_hash: Hash,
-}
-
-struct BlobResponse {
-  blob: Vec<u8>
-}
-}
-

This protocol will be used by backers to request the PVF from collators in the -following conditions:

-
    -
  1. They received a collation sending RequestCodeUpgrade.
    2. -
  2. They received a collation, but they don't yet have the code that was -previously registered on the relaychain. (E.g. disk pruned, new validator)
    3. -
-

In case they received the collation via PoV distribution instead of from the -collator itself, they will use the exact same message to fetch from the valiator -they got the PoV from.

-

Get the new code to all validators

-

Once the candidate issuing RequestCodeUpgrade got backed on chain, validators -will start fetching the code from the backers as part of availability -distribution.

-

To mitigate attack vectors we should make sure that serving requests for code -can be treated as low priority requests. Thus I am suggesting the following -scheme:

-

Validators will notice via a runtime API (TODO: Define) that a new code has been requested, the -API will return the Hash and a counter, which starts at some configurable -value e.g. 10. The validators are now aware of the new hash and start fetching, -but they don't have to wait for the fetch to succeed to sign their bitfield.

-

Then on each further candidate from that chain that counter gets decremented. -Validators which have not yet succeeded fetching will now try again. This game -continues until the counter reached 0. Now it is mandatory to have to code in -order to sign a 1 in the bitfield.

-

PVF pre-checking will happen after the candidate which brought the counter to -0 has been successfully included and thus is also able to assume that 2/3 of -the validators have the code.

-

This scheme serves two purposes:

-
    -
  1. Fetching can happen over a longer period of time with low priority. E.g. if -we waited for the PVF at the very first avaialbility distribution, this might -actually affect liveness of other chains on the same core. Distributing -megabytes of data to a thousand validators, might take a bit. Thus this helps -isolating parachains from each other.
    2. -
  2. By configuring the initial counter value we can affect how much an upgrade -costs. E.g. forcing the parachain to produce 10 blocks, means 10x the cost -for issuing an update. If too frequent upgrades ever become a problem for the -system, we have a knob to make them more costly.
    3. -
-

On-chain code upgrade process

-

First when a candidate is backed we need to make the new hash available -(together with a counter) via a -runtime API so validators in availability distribution can check for it and -fetch it if changed (see previous section). For performance reasons, I think we -should not do an additional call, but replace the existing one with one containing the new additional information (Option<(Hash, Counter)>).

-

Once the candidate gets included (counter 0), the hash is given to pre-checking -and only after pre-checking succeeded (and a full session passed) it is finally -enacted and the parachain can switch to the new code. (Same process as it used -to be.)

-

Handling new validators

-

Backers

-

If a backer receives a collation for a parachain it does not yet have the code -as enacted on chain (see "On-chain code upgrade process"), it will use above -request/response protocol to fetch it from whom it received the collation.

-

Availablity Distribution

-

Validators in availability distribution will be changed to only sign a 1 in -the bitfield of a candidate if they not only have the chunk, but also the -currently active PVF. They will fetch it from backers in case they don't have it -yet.

-

How do other parties get hold of the PVF?

-

Two ways:

-
    -
  1. Discover collators via relay chain DHT and request from them: Preferred way, -as it is less load on validators.
    2. -
  2. Request from validators, which will serve on a best effort basis.
    3. -
-

Pruning

-

We covered how validators get hold of new code, but when can they prune old ones? -In principle it is not an issue, if some validors prune code, because:

-
    -
  1. We changed it so that a candidate is not deemed available if validators were -not able to fetch the PVF.
    2. -
  2. Backers can always fetch the PVF from collators as part of the collation -fetching.
    3. -
-

But the majority of validators should always keep the latest code of any -parachain and only prune the previous one, once the first candidate using the -new code got finalized. This ensures that disputes will always be able to -resolve.

-

Drawbacks

-

The major drawback of this solution is the same as any solution the moves work -off-chain, it adds complexity to the node. E.g. nodes needing the PVF, need to -store them separately, together with their own pruning strategy as well.

-

Testing, Security, and Privacy

-

Implementations adhering to this RFC, will respond to PVF requests with the -actual PVF, if they have it. Requesters will persist received PVFs on disk for -as long as they are replaced by a new one. Implementations must not be lazy -here, if validators only fetched the PVF when needed, they can be prevented from -participating in disputes.

-

Validators should treat incoming requests for PVFs in general with rather low -priority, but should prefer fetches from other validators over requests from -random peers.

-

Given that we are altering what set bits in the availability bitfields mean (not -only chunk, but also PVF available), it is important to have enough validators -upgraded, before we allow collators to make use of the new runtime upgrade -mechanism. Otherwise we would risk disputes to not being able to succeed.

-

This RFC has no impact on privacy.

-

Performance, Ergonomics, and Compatibility

-

Performance

-

This proposal lightens the load on the relay chain and is thus in general -beneficial for the performance of the network, this is achieved by the -following:

-
    -
  1. Code upgrades are still propagated to all validators, but only once, not -twice (First statements, then via the containing relay chain block).
    2. -
  2. Code upgrades are only communicated to validators and other nodes which are -interested, not any full node as it has been before.
    3. -
  3. Relay chain block space is preserved. Previously we could only do one runtime -upgrade per relay chain block, occupying almost all of the blockspace.
    4. -
  4. Signalling an upgrade no longer contains the upgrade, hence if we need to -push back on an upgrade for whatever reason, no network bandwidth and core -time gets wasted because of this.
    5. -
-

Ergonomics

-

End users are only affected by better performance and more stable block times. -Parachains will need to implement the introduced request/response protocol and -adapt to the new signalling mechanism via an UMP message, instead of sending -the code upgrade directly.

-

For parachain operators we should emit events on initiated runtime upgrade and -each block reporting the current counter and how many blocks to go until the -upgrade gets passed to pre-checking. This is especially important for on-demand -chains or bulk users not occupying a full core. Further more that behaviour of -requiring multiple blocks to fully initiate a runtime upgrade needs to be well -documented.

-

Compatibility

-

We will continue to support the old mechanism for code upgrades for a while, but -will start to impose stricter limits over time, with the number of registered -parachains going up. With those limits in place parachains not migrating to the -new scheme might be having a harder time upgrading and will miss more blocks. I -guess we can be lenient for a while still, so the upgrade path for -parachains should be rather smooth.

-

In total the protocol changes we need are:

-

For validators and collators:

-
    -
  1. New request/response protocol for fetching PVF data from collators and -validators.
    2. -
  2. New UMP message type for signalling a runtime upgrade.
    3. -
-

Only for validators:

-
    -
  1. New runtime API for determining to be enacted code upgrades.
    2. -
  2. Different behaviour of bitfields (only sign a 1 bit, if validator has chunk + -"hot" PVF).
    3. -
  3. Altered behaviour in availability-distribution: Fetch missing PVFS.
    4. -
-

Prior Art and References

-

Off-chain runtime upgrades have been discussed before, the architecture -described here is simpler though as it piggybacks on already existing features, -namely:

-
    -
  1. availability-distribution: No separate I have code messages anymore.
    2. -
  2. Existing pre-checking.
    3. -
-

https://github.com/paritytech/polkadot-sdk/issues/971

-

Unresolved Questions

-
    -
  1. What about the initial runtime, shall we make that off-chain as well?
    2. -
  2. Good news, at least after the first upgrade, no code will be stored on chain -any more, this means that we also have to redefine the storage deposit now. -We no longer charge for chain storage, but validator disk storage -> Should -be cheaper. Solution to this: Not only store the hash on chain, but also the -size of the data. Then define a price per byte and charge that, but: - -
    3. -
-

TODO: Fully resolve these questions and incorporate in RFC text.

- -

Further Hardening

-

By no longer having code upgrade go through the relay chain, occupying a full relay -chain block, the impact on other parachains is already greatly reduced, if we -make distribution and PVF pre-checking low-priority processes on validators. The -only thing attackers might be able to do is delay upgrades of other parachains.

-

Which seems like a problem to be solved once we actually see it as a problem in -the wild (and can already be mitigated by adjusting the counter). The good thing -is that we have all the ingredients to go further if need be. Signalling no -longer actually includes the code, hence there is no need to reject the -candidate: The parachain can make progress even if we choose not to immediately -act on the request and no relay chain resources are wasted either.

-

We could for example introduce another UMP Signalling message -RequestCodeUpgradeWithPriority which not just requests a code upgrade, but -also offers some DOT to get ranked up in a queue.

-

Generalize this off-chain storage mechanism?

-

Making this storage mechanism more general purpose is worth thinking about. E.g. -by resolving above "fee" question, we might also be able to resolve the pruning -question in a more generic way and thus could indeed open this storage facility -for other purposes as well. E.g. smart contracts, so the PoV would only need to -reference contracts by hash and the actual PoV is stored on validators and -collators and thus no longer needs to be part of the PoV.

-

A possible avenue would be to change the response to:

-
#![allow(unused)]
-fn main() {
-enum BlobResponse {
-  Blob(Vec<u8>),
-  Blobs(MerkleTree),
-}
-}
-

With this the hash specified in the request can also be a merkle root and the -responder will respond with the entire merkle tree (only hashes, no payload). -Then the requester can traverse the leaf hashes and use the same request -response protocol to request any locally missing blobs in that tree.

-

One leaf would for example be the PVF others could be smart contracts. With a -properly specified format (e.g. which leaf is the PVF?), what we got here is -that a parachain can not only update its PVF, but additional data, -incrementally. E.g. adding another smart contract, does not require resubmitting -the entire PVF to validators, only the root hash on the relay chain gets -updated, then validators fetch the merkle tree and only fetch any missing -leaves. That additional data could be made available to the PVF via a to be -added host function. The nice thing about this approach is, that while we can -upgrade incrementally, lifetime is still tied to the PVF and we get all the same -guarantees. Assuming the validators store blobs by hash, we even get disk -sharing if multiple parachains use the same data (e.g. same smart contracts).

(source)

Table of Contents

Given that these mistakes are likely, it is necessary to provide a solution to either prevent them or enable access to a pure account on a target chain.

-

Stakeholders

+

Stakeholders

Runtime Users, Runtime Devs, wallets, cross-chain dApps.

-

Explanation

+

Explanation

One possible solution is to allow a proxy to create or replicate a pure proxy relationship for the same pure account on a target chain. For example, Alice, as the proxy of the pure1 pure account on parachain A, should be able to set a proxy for the same pure1 account on parachain B.

To minimise security risks, the parachain B should grant the parachain A the least amount of permission necessary for the replication. First, Parachain A claims to Parachain B that the operation is commanded by the pure account, and thus by its proxy, and second, provides proof that the account is keyless.

The replication process will be facilitated by XCM, with the first claim made using the DescendOrigin instruction. The replication call on parachain A would require a signed origin by the pure account and construct an XCM program for parachain B, where it first descends the origin, resulting in the ParachainA/AccountId32(pure1) origin location on the receiving side.

@@ -846,7 +507,7 @@ mod pallet_proxy_replica { } } -

Drawbacks

+

Drawbacks

There are two disadvantages to this approach:

-

Motivation

+

Motivation

In Substrate, runtime APIs facilitate off-chain clients in reading the state of the consensus system. However, different chains may expose different APIs for a similar query or have varying data types, such as doing custom transformations on direct data, or differing AccountId types. This diversity also extends to client-side, which may require custom computations over runtime APIs in various use cases. Therefore, tools and UI developers often access storage directly and reimplement custom computations to convert data into user-friendly representations, leading to duplicated code between Rust runtime logic and UI JS/TS logic. This duplication increases workload and potential for bugs.

Therefore, a system is needed to serve as an intermediary layer between concrete chain runtime implementations and tools/UIs, to provide a unified interface for cross-chain queries.

-

Stakeholders

+

Stakeholders

-

Explanation

+

Explanation

The overall query pattern of XCQ consists of three components:

-

Drawbacks

+

Drawbacks

Performance issues

-

Testing, Security, and Privacy

+

Testing, Security, and Privacy

-

Performance, Ergonomics, and Compatibility

-

Performance

+

Performance, Ergonomics, and Compatibility

+

Performance

It's a new functionality, which doesn't modify the existing implementations.

-

Ergonomics

+

Ergonomics

The proposal facilitate the wallets and dApps developers. Developers no longer need to examine every concrete implementation to support conceptually similar operations across different chains. Additionally, they gain a more modular development experience through encapsulating custom computations over the exposed APIs in PolkaVM programs.

-

Compatibility

+

Compatibility

The proposal defines new apis, which doesn't break compatibility with existing interfaces.

-

Prior Art and References

+

Prior Art and References

There are several discussions related to the proposal, including:

-

Unresolved Questions

- +

Unresolved Questions

+

(source)

Table of Contents

-

Prior Art and References

+

Prior Art and References

Robert Habermeier initially wrote on the subject of Polkadot blockspace-centric in the article Polkadot Blockspace over Blockchains. While not going into details, the article served as an early reframing piece for moving beyond one-slot-per-chain models and building out secondary market infrastructure for resource allocation.

(source)

Table of Contents

@@ -2792,10 +2453,10 @@ InstaPoolHistory: (empty) AuthorsGavin Wood, Robert Habermeier -

Summary

+

Summary

In the Agile Coretime model of the Polkadot Ubiquitous Computer, as proposed in RFC-1 and RFC-3, it is necessary for the allocating parachain (envisioned to be one or more pallets on a specialised Brokerage System Chain) to communicate the core assignments to the Relay-chain, which is responsible for ensuring those assignments are properly enacted.

This is a proposal for the interface which will exist around the Relay-chain in order to communicate this information and instructions.

-

Motivation

+

Motivation

The background motivation for this interface is splitting out coretime allocation functions and secondary markets from the Relay-chain onto System parachains. A well-understood and general interface is necessary for ensuring the Relay-chain receives coretime allocation instructions from one or more System chains without introducing dependencies on the implementation details of either side.

Requirements

-

Stakeholders

+

Stakeholders

Primary stakeholder sets are:

Socialization:

This content of this RFC was discussed in the Polkdot Fellows channel.

-

Explanation

+

Explanation

The interface has two sections: The messages which the Relay-chain is able to receive from the allocating parachain (the UMP message types), and messages which the Relay-chain is able to send to the allocating parachain (the DMP message types). These messages are expected to be able to be implemented in a well-known pallet and called with the XCM Transact instruction.

Future work may include these messages being introduced into the XCM standard.

UMP Message Types

@@ -2890,17 +2551,17 @@ assert_eq!(targets.iter().map(|x| x.1).sum(), 57600);

Realistic Limits of the Usage

For request_revenue_info, a successful request should be possible if when is no less than the Relay-chain block number on arrival of the message less 100,000.

For assign_core, a successful request should be possible if begin is no less than the Relay-chain block number on arrival of the message plus 10 and workload contains no more than 100 items.

-

Performance, Ergonomics and Compatibility

+

Performance, Ergonomics and Compatibility

No specific considerations.

-

Testing, Security and Privacy

+

Testing, Security and Privacy

Standard Polkadot testing and security auditing applies.

The proposal introduces no new privacy concerns.

- +

RFC-1 proposes a means of determining allocation of Coretime using this interface.

RFC-3 proposes a means of implementing the high-level allocations within the Relay-chain.

Drawbacks, Alternatives and Unknowns

None at present.

-

Prior Art and References

+

Prior Art and References

None.

(source)

Table of Contents

@@ -2946,13 +2607,13 @@ assert_eq!(targets.iter().map(|x| x.1).sum(), 57600); AuthorsJoe Petrowski -

Summary

+

Summary

As core functionality moves from the Relay Chain into system chains, so increases the reliance on the liveness of these chains for the use of the network. It is not economically scalable, nor necessary from a game-theoretic perspective, to pay collators large rewards. This RFC proposes a mechanism -- part technical and part social -- for ensuring reliable collator sets that are resilient to attemps to stop any subsytem of the Polkadot protocol.

-

Motivation

+

Motivation

In order to guarantee access to Polkadot's system, the collators on its system chains must propose blocks (provide liveness) and allow all transactions to eventually be included. That is, some collators may censor transactions, but there must exist one collator in the set who will include a @@ -2988,12 +2649,12 @@ to censor any subset of transactions.

  • Collators selected by governance SHOULD have a reasonable expectation that the Treasury will reimburse their operating costs.
    • -

    Stakeholders

    +

    Stakeholders

    -

    Explanation

    +

    Explanation

    This protocol builds on the existing Collator Selection pallet and its notion of Invulnerables. Invulnerables are collators (identified by their AccountIds) who @@ -3029,27 +2690,27 @@ approximately:

  • of which 15 are Invulnerable, and
  • five are elected by bond.
    • -

    Drawbacks

    +

    Drawbacks

    The primary drawback is a reliance on governance for continued treasury funding of infrastructure costs for Invulnerable collators.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    The vast majority of cases can be covered by unit testing. Integration test should ensure that the Collator Selection UpdateOrigin, which has permission to modify the Invulnerables and desired number of Candidates, can handle updates over XCM from the system's governance location.

    -

    Performance, Ergonomics, and Compatibility

    +

    Performance, Ergonomics, and Compatibility

    This proposal has very little impact on most users of Polkadot, and should improve the performance of system chains by reducing the number of missed blocks.

    -

    Performance

    +

    Performance

    As chains have strict PoV size limits, care must be taken in the PoV impact of the session manager. Appropriate benchmarking and tests should ensure that conservative limits are placed on the number of Invulnerables and Candidates.

    -

    Ergonomics

    +

    Ergonomics

    The primary group affected is Candidate collators, who, after implementation of this RFC, will need to compete in a bond-based election rather than a race to claim a Candidate spot.

    -

    Compatibility

    +

    Compatibility

    This RFC is compatible with the existing implementation and can be handled via upgrades and migration.

    -

    Prior Art and References

    +

    Prior Art and References

    Written Discussions

    -

    Unresolved Questions

    +

    Unresolved Questions

    None at this time.

    - +

    There may exist in the future system chains for which this model of collator selection is not appropriate. These chains should be evaluated on a case-by-case basis.

    (source)

    @@ -3105,10 +2766,10 @@ appropriate. These chains should be evaluated on a case-by-case basis.

    AuthorsPierre Krieger -

    Summary

    +

    Summary

    The full nodes of the Polkadot peer-to-peer network maintain a distributed hash table (DHT), which is currently used for full nodes discovery and validators discovery purposes.

    This RFC proposes to extend this DHT to be used to discover full nodes of the parachains of Polkadot.

    -

    Motivation

    +

    Motivation

    The maintenance of bootnodes has long been an annoyance for everyone.

    When a bootnode is newly-deployed or removed, every chain specification must be updated in order to take the update into account. This has lead to various non-optimal solutions, such as pulling chain specifications from GitHub repositories. When it comes to RPC nodes, UX developers often have trouble finding up-to-date addresses of parachain RPC nodes. With the ongoing migration from RPC nodes to light clients, similar problems would happen with chain specifications as well.

    @@ -3117,9 +2778,9 @@ When it comes to RPC nodes, UX developers often have trouble finding up-to-date

    Because the list of bootnodes in chain specifications is so annoying to modify, the consequence is that the number of bootnodes is rather low (typically between 2 and 15). In order to better resist downtimes and DoS attacks, a better solution would be to use every node of a certain chain as potential bootnode, rather than special-casing some specific nodes.

    While this RFC doesn't solve these problems for relay chains, it aims at solving it for parachains by storing the list of all the full nodes of a parachain on the relay chain DHT.

    Assuming that this RFC is implemented, and that light clients are used, deploying a parachain wouldn't require more work than registering it onto the relay chain and starting the collators. There wouldn't be any need for special infrastructure nodes anymore.

    -

    Stakeholders

    +

    Stakeholders

    This RFC has been opened on my own initiative because I think that this is a good technical solution to a usability problem that many people are encountering and that they don't realize can be solved.

    -

    Explanation

    +

    Explanation

    The content of this RFC only applies for parachains and parachain nodes that are "Substrate-compatible". It is in no way mandatory for parachains to comply to this RFC.

    Note that "Substrate-compatible" is very loosely defined as "implements the same mechanisms and networking protocols as Substrate". The author of this RFC believes that "Substrate-compatible" should be very precisely specified, but there is controversy on this topic.

    While a lot of this RFC concerns the implementation of parachain nodes, it makes use of the resources of the Polkadot chain, and as such it is important to describe them in the Polkadot specification.

    @@ -3156,10 +2817,10 @@ message Response {

    The maximum size of a response is set to an arbitrary 16kiB. The responding side should make sure to conform to this limit. Given that fork_id is typically very small and that the only variable-length field is addrs, this is easily achieved by limiting the number of addresses.

    Implementers should be aware that addrs might be very large, and are encouraged to limit the number of addrs to an implementation-defined value.

    -

    Drawbacks

    +

    Drawbacks

    The peer_id and addrs fields are in theory not strictly needed, as the PeerId and addresses could be always equal to the PeerId and addresses of the node being registered as the provider and serving the response. However, the Cumulus implementation currently uses two different networking stacks, one of the parachain and one for the relay chain, using two separate PeerIds and addresses, and as such the PeerId and addresses of the other networking stack must be indicated. Asking them to use only one networking stack wouldn't feasible in a realistic time frame.

    The values of the genesis_hash and fork_id fields cannot be verified by the requester and are expected to be unused at the moment. Instead, a client that desires connecting to a parachain is expected to obtain the genesis hash and fork ID of the parachain from the parachain chain specification. These fields are included in the networking protocol nonetheless in case an acceptable solution is found in the future, and in order to allow use cases such as discovering parachains in a not-strictly-trusted way.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    Because not all nodes want to be used as bootnodes, implementers are encouraged to provide a way to disable this mechanism. However, it is very much encouraged to leave this mechanism on by default for all parachain nodes.

    This mechanism doesn't add or remove any security by itself, as it relies on existing mechanisms. However, if the principle of chain specification bootnodes is entirely replaced with the mechanism described in this RFC (which is the objective), then it becomes important whether the mechanism in this RFC can be abused in order to make a parachain unreachable.

    @@ -3168,22 +2829,22 @@ Furthermore, when a large number of providers (here, a provider is a bootnode) a

    For this reason, an attacker can abuse this mechanism by randomly generating libp2p PeerIds until they find the 20 entries closest to the key representing the target parachain. They are then in control of the parachain bootnodes. Because the key changes periodically and isn't predictable, and assuming that the Polkadot DHT is sufficiently large, it is not realistic for an attack like this to be maintained in the long term.

    Furthermore, parachain clients are expected to cache a list of known good nodes on their disk. If the mechanism described in this RFC went down, it would only prevent new nodes from accessing the parachain, while clients that have connected before would not be affected.

    -

    Performance, Ergonomics, and Compatibility

    -

    Performance

    +

    Performance, Ergonomics, and Compatibility

    +

    Performance

    The DHT mechanism generally has a low overhead, especially given that publishing providers is done only every 24 hours.

    Doing a Kademlia iterative query then sending a provider record shouldn't take more than around 50 kiB in total of bandwidth for the parachain bootnode.

    Assuming 1000 parachain full nodes, the 20 Polkadot full nodes corresponding to a specific parachain will each receive a sudden spike of a few megabytes of networking traffic when the key rotates. Again, this is relatively negligible. If this becomes a problem, one can add a random delay before a parachain full node registers itself to be the provider of the key corresponding to BabeApi_next_epoch.

    Maybe the biggest uncertainty is the traffic that the 20 Polkadot full nodes will receive from light clients that desire knowing the bootnodes of a parachain. Light clients are generally encouraged to cache the peers that they use between restarts, so they should only query these 20 Polkadot full nodes at their first initialization. If this every becomes a problem, this value of 20 is an arbitrary constant that can be increased for more redundancy.

    -

    Ergonomics

    +

    Ergonomics

    Irrelevant.

    -

    Compatibility

    +

    Compatibility

    Irrelevant.

    -

    Prior Art and References

    +

    Prior Art and References

    None.

    -

    Unresolved Questions

    +

    Unresolved Questions

    While it fundamentally doesn't change much to this RFC, using BabeApi_currentEpoch and BabeApi_nextEpoch might be inappropriate. I'm not familiar enough with good practices within the runtime to have an opinion here. Should it be an entirely new pallet?

    - +

    It is possible that in the future a client could connect to a parachain without having to rely on a trusted parachain specification.

    (source)

    Table of Contents

    @@ -3204,13 +2865,13 @@ If this every becomes a problem, this value of 20 is an arbitrary constant that AuthorsJonas Gehrlein -

    Summary

    +

    Summary

    The Polkadot UC will generate revenue from the sale of available Coretime. The question then arises: how should we handle these revenues? Broadly, there are two reasonable paths – burning the revenue and thereby removing it from total issuance or divert it to the Treasury. This Request for Comment (RFC) presents arguments favoring burning as the preferred mechanism for handling revenues from Coretime sales.

    -

    Motivation

    +

    Motivation

    How to handle the revenue accrued from Coretime sales is an important economic question that influences the value of DOT and should be properly discussed before deciding for either of the options. Now is the best time to start this discussion.

    -

    Stakeholders

    +

    Stakeholders

    Polkadot DOT token holders.

    -

    Explanation

    +

    Explanation

    This RFC discusses potential benefits of burning the revenue accrued from Coretime sales instead of diverting them to Treasury. Here are the following arguments for it.

    It's in the interest of the Polkadot community to have a consistent and predictable Treasury income, because volatility in the inflow can be damaging, especially in situations when it is insufficient. As such, this RFC operates under the presumption of a steady and sustainable Treasury income flow, which is crucial for the Polkadot community's stability. The assurance of a predictable Treasury income, as outlined in a prior discussion here, or through other equally effective measures, serves as a baseline assumption for this argument.

    Consequently, we need not concern ourselves with this particular issue here. This naturally begs the question - why should we introduce additional volatility to the Treasury by aligning it with the variable Coretime sales? It's worth noting that Coretime revenues often exhibit an inverse relationship with periods when Treasury spending should ideally be ramped up. During periods of low Coretime utilization (indicated by lower revenue), Treasury should spend more on projects and endeavours to increase the demand for Coretime. This pattern underscores that Coretime sales, by their very nature, are an inconsistent and unpredictable source of funding for the Treasury. Given the importance of maintaining a steady and predictable inflow, it's unnecessary to rely on another volatile mechanism. Some might argue that we could have both: a steady inflow (from inflation) and some added bonus from Coretime sales, but burning the revenue would offer further benefits as described below.

    @@ -3253,13 +2914,13 @@ If this every becomes a problem, this value of 20 is an arbitrary constant that AuthorsJoe Petrowski -

    Summary

    +

    Summary

    Since the introduction of the Collectives parachain, many groups have expressed interest in forming new -- or migrating existing groups into -- on-chain collectives. While adding a new collective is relatively simple from a technical standpoint, the Fellowship will need to merge new pallets into the Collectives parachain for each new collective. This RFC proposes a means for the network to ratify a new collective, thus instructing the Fellowship to instate it in the runtime.

    -

    Motivation

    +

    Motivation

    Many groups have expressed interest in representing collectives on-chain. Some of these include:

    -

    Drawbacks

    +

    Drawbacks

    Other than all other system chains, development and maintenance of the Encointer Network is mainly financed by the KSM Treasury and possibly the DOT Treasury in the future. Encointer is dedicated to maintaining its network and runtime code for as long as possible, but there is a dependency on funding which is not in the hands of the fellowship. The only risk in the context of funding, however, is that the Encointer runtime will see less frequent updates if there's less funding.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    No changes to the existing system are proposed. Only changes to how maintenance is organized.

    -

    Performance, Ergonomics, and Compatibility

    +

    Performance, Ergonomics, and Compatibility

    No changes

    -

    Prior Art and References

    +

    Prior Art and References

    Existing Encointer runtime repo

    -

    Unresolved Questions

    +

    Unresolved Questions

    None identified

    - +

    More info on Encointer: encointer.org

    (source)

    Table of Contents

    @@ -4551,11 +4212,11 @@ other privacy-enhancing mechanisms to address this concern. AuthorsJoe Petrowski, Gavin Wood -

    Summary

    +

    Summary

    The Relay Chain contains most of the core logic for the Polkadot network. While this was necessary prior to the launch of parachains and development of XCM, most of this logic can exist in parachains. This is a proposal to migrate several subsystems into system parachains.

    -

    Motivation

    +

    Motivation

    Polkadot's scaling approach allows many distinct state machines (known generally as parachains) to operate with common guarantees about the validity and security of their state transitions. Polkadot provides these common guarantees by executing the state transitions on a strict subset (a backing @@ -4567,13 +4228,13 @@ blockspace) to the network.

    By minimising state transition logic on the Relay Chain by migrating it into "system chains" -- a set of parachains that, with the Relay Chain, make up the Polkadot protocol -- the Polkadot Ubiquitous Computer can maximise its primary offering: secure blockspace.

    -

    Stakeholders

    +

    Stakeholders

    -

    Explanation

    +

    Explanation

    The following pallets and subsystems are good candidates to migrate from the Relay Chain:

     -

    Drawbacks

    +

    Drawbacks

    The main drawback of adding the additional complexity directly to the broker pallet is the potential increase in maintenance overhead. Therefore, we propose adding additional functionality as a separate pallet on the Coretime chain. To take the pressure off from implementing these features, implementation along with unit tests would be taken care of by Lastic (Aurora Makovac, Philip Lucsok).

    There are potential risks of security vulnerabilities in the new market functionalities, such as unauthorized region transfers or incorrect balance adjustments. Therefore, extensive security measures would have to be implemented.

    -

    Testing, Security, and Privacy

    +

    Testing, Security, and Privacy

    Testing