pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-05-31 06:21:02 +00:00
Code Issues Packages Projects Releases Wiki Activity

Implement HRMP (#1900)

Browse Source 
* HRMP: Update the impl guide

* HRMP: Incorporate the channel notifications into the guide

* HRMP: Renaming in the impl guide

* HRMP: Constrain the maximum number of HRMP messages per candidate

This commit addresses the HRMP part of https://github.com/paritytech/polkadot/issues/1869

* XCM: Introduce HRMP related message types

* HRMP: Data structures and plumbing

* HRMP: Configuration

* HRMP: Data layout

* HRMP: Acceptance & Enactment

* HRMP: Test base logic

* Update adder collator

* HRMP: Runtime API for accessing inbound messages

Also, removing some redundant fully-qualified names.

* HRMP: Add diagnostic logging in acceptance criteria

* HRMP: Additional tests

* Self-review fixes

* save test refactorings for the next time

* Missed a return statement.

* a formatting blip

* Add missing logic for appending HRMP digests

* Remove the channel contents vectors which became empty

* Tighten HRMP channel digests invariants.

* Apply suggestions from code review

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Remove a note about sorting for channel id

* Add missing rustdocs to the configuration

* Clarify and update the invariant for HrmpChannelDigests

* Make the onboarding invariant less sloppy

Namely, introduce `Paras::is_valid_para` (in fact, it already is present
in the implementation) and hook up the invariant to that.

Note that this says "within a session" because I don't want to make it
super strict on the session boundary. The logic on the session boundary
should be extremely careful.

* Make `CandidateCheckContext` use T::BlockNumber for hrmp_watermark

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
This commit is contained in:
Sergei Shulepov
2020-11-06 16:35:36 +01:00
committed by GitHub
parent 8a2911b85d
commit c96f8cfcca
33 changed files with 2024 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files
polkadot/roadmap/implementers-guide/src/runtime/inclusion.md
+3 -4
View File
@@ -70,8 +70,7 @@ All failed checks should lead to an unrecoverable error making the block invalid
1. call `Router::check_upward_messages(para, commitments.upward_messages)` to check that the upward messages are valid.
1. call `Router::check_processed_downward_messages(para, commitments.processed_downward_messages)` to check that the DMQ is properly drained.
1. call `Router::check_hrmp_watermark(para, commitments.hrmp_watermark)` for each candidate to check rules of processing the HRMP watermark.
1. check that in the commitments of each candidate the horizontal messages are sorted by ascending recipient ParaId and there is no two horizontal messages have the same recipient.
1. using `Router::verify_outbound_hrmp(sender, commitments.horizontal_messages)` ensure that the each candidate send a valid set of horizontal messages
1. using `Router::check_outbound_hrmp(sender, commitments.horizontal_messages)` ensure that the each candidate sent a valid set of horizontal messages
1. create an entry in the `PendingAvailability` map for each backed candidate with a blank `availability_votes` bitfield.
1. create a corresponding entry in the `PendingAvailabilityCommitments` with the commitments.
1. Return a `Vec<CoreIndex>` of all scheduled cores of the list of passed assignments that a candidate was successfully backed for, sorted ascending by CoreIndex.
@@ -79,9 +78,9 @@ All failed checks should lead to an unrecoverable error making the block invalid
1. If the receipt contains a code upgrade, Call `Paras::schedule_code_upgrade(para_id, code, relay_parent_number + config.validationl_upgrade_delay)`.
> TODO: Note that this is safe as long as we never enact candidates where the relay parent is across a session boundary. In that case, which we should be careful to avoid with contextual execution, the configuration might have changed and the para may de-sync from the host's understanding of it.
1. call `Router::enact_upward_messages` for each backed candidate, using the [`UpwardMessage`s](../types/messages.md#upward-message) from the [`CandidateCommitments`](../types/candidate.md#candidate-commitments).
1. call `Router::queue_outbound_hrmp` with the para id of the candidate and the list of horizontal messages taken from the commitment,
1. call `Router::prune_hrmp` with the para id of the candiate and the candidate's `hrmp_watermark`.
1. call `Router::prune_dmq` with the para id of the candidate and the candidate's `processed_downward_messages`.
1. call `Router::prune_hrmp` with the para id of the candiate and the candidate's `hrmp_watermark`.
1. call `Router::queue_outbound_hrmp` with the para id of the candidate and the list of horizontal messages taken from the commitment,
1. Call `Paras::note_new_head` using the `HeadData` from the receipt and `relay_parent_number`.
* `collect_pending`:
polkadot/roadmap/implementers-guide/src/runtime/paras.md
+1
View File
@@ -111,6 +111,7 @@ OutgoingParas: Vec<ParaId>;
* `note_new_head(ParaId, HeadData, BlockNumber)`: note that a para has progressed to a new head, where the new head was executed in the context of a relay-chain block with given number. This will apply pending code upgrades based on the block number provided.
* `validation_code_at(ParaId, at: BlockNumber, assume_intermediate: Option<BlockNumber>)`: Fetches the validation code to be used when validating a block in the context of the given relay-chain height. A second block number parameter may be used to tell the lookup to proceed as if an intermediate parablock has been included at the given relay-chain height. This may return past, current, or (with certain choices of `assume_intermediate`) future code. `assume_intermediate`, if provided, must be before `at`. If the validation code has been pruned, this will return `None`.
* `is_parathread(ParaId) -> bool`: Returns true if the para ID references any live parathread.
* `is_valid_para(ParaId) -> bool`: Returns true if the para ID references either a live parathread or live parachain.
* `last_code_upgrade(id: ParaId, include_future: bool) -> Option<BlockNumber>`: The block number of the last scheduled upgrade of the requested para. Includes future upgrades if the flag is set. This is the `expected_at` number, not the `activated_at` number.
* `persisted_validation_data(id: ParaId) -> Option<PersistedValidationData>`: Get the PersistedValidationData of the given para, assuming the context is the parent block. Returns `None` if the para is not known.
polkadot/roadmap/implementers-guide/src/runtime/router.md
+70 -29
View File
@@ -78,10 +78,12 @@ struct HrmpOpenChannelRequest {
age: SessionIndex,
/// The amount that the sender supplied at the time of creation of this request.
sender_deposit: Balance,
/// The maximum message size that could be put into the channel.
max_message_size: u32,
/// The maximum number of messages that can be pending in the channel at once.
limit_used_places: u32,
max_capacity: u32,
/// The maximum total size of the messages that can be pending in the channel at once.
limit_used_bytes: u32,
max_total_size: u32,
}
/// A metadata of an HRMP channel.
@@ -91,25 +93,25 @@ struct HrmpChannel {
/// The amount that the recipient supplied as a deposit when accepting opening this channel.
recipient_deposit: Balance,
/// The maximum number of messages that can be pending in the channel at once.
limit_used_places: u32,
max_capacity: u32,
/// The maximum total size of the messages that can be pending in the channel at once.
limit_used_bytes: u32,
max_total_size: u32,
/// The maximum message size that could be put into the channel.
limit_message_size: u32,
max_message_size: u32,
/// The current number of messages pending in the channel.
/// Invariant: should be less or equal to `limit_used_places`.
used_places: u32,
/// Invariant: should be less or equal to `max_capacity`.
msg_count: u32,
/// The total size in bytes of all message payloads in the channel.
/// Invariant: should be less or equal to `limit_used_bytes`.
used_bytes: u32,
/// Invariant: should be less or equal to `max_total_size`.
total_size: u32,
/// A head of the Message Queue Chain for this channel. Each link in this chain has a form:
/// `(prev_head, B, H(M))`, where
/// - `prev_head`: is the previous value of `mqc_head`.
/// - `prev_head`: is the previous value of `mqc_head` or zero if none.
/// - `B`: is the [relay-chain] block number in which a message was appended
/// - `H(M)`: is the hash of the message being appended.
/// This value is initialized to a special value that consists of all zeroes which indicates
/// that no messages were previously added.
mqc_head: Hash,
mqc_head: Option<Hash>,
}
```
HRMP related storage layout
@@ -144,14 +146,26 @@ HrmpCloseChannelRequests: map HrmpChannelId => Option<()>;
HrmpCloseChannelRequestsList: Vec<HrmpChannelId>;
/// The HRMP watermark associated with each para.
/// Invariant:
/// - each para `P` used here as a key should satisfy `Paras::is_valid_para(P)` within a session.
HrmpWatermarks: map ParaId => Option<BlockNumber>;
/// HRMP channel data associated with each para.
/// Invariant:
/// - each participant in the channel should satisfy `Paras::is_valid_para(P)` within a session.
HrmpChannels: map HrmpChannelId => Option<HrmpChannel>;
/// The indexes that map all senders to their recievers and vise versa.
/// Ingress/egress indexes allow to find all the senders and receivers given the opposite
/// side. I.e.
///
/// (a) ingress index allows to find all the senders for a given recipient.
/// (b) egress index allows to find all the recipients for a given sender.
///
/// Invariants:
/// - for each ingress index entry for `P` each item `I` in the index should present in `HrmpChannels` as `(I, P)`.
/// - for each egress index entry for `P` each item `E` in the index should present in `HrmpChannels` as `(P, E)`.
/// - for each ingress index entry for `P` each item `I` in the index should present in `HrmpChannels`
/// as `(I, P)`.
/// - for each egress index entry for `P` each item `E` in the index should present in `HrmpChannels`
/// as `(P, E)`.
/// - there should be no other dangling channels in `HrmpChannels`.
/// - the vectors are sorted.
HrmpIngressChannelsIndex: map ParaId => Vec<ParaId>;
HrmpEgressChannelsIndex: map ParaId => Vec<ParaId>;
/// Storage for the messages for each channel.
@@ -159,7 +173,11 @@ HrmpEgressChannelsIndex: map ParaId => Vec<ParaId>;
HrmpChannelContents: map HrmpChannelId => Vec<InboundHrmpMessage>;
/// Maintains a mapping that can be used to answer the question:
/// What paras sent a message at the given block number for a given reciever.
/// Invariant: The para ids vector is never empty.
/// Invariants:
/// - The inner `Vec<ParaId>` is never empty.
/// - The inner `Vec<ParaId>` cannot store two same `ParaId`.
/// - The outer vector is sorted ascending by block number and cannot store two items with the same
/// block number.
HrmpChannelDigests: map ParaId => Vec<(BlockNumber, Vec<ParaId>)>;
```
@@ -184,12 +202,14 @@ Candidate Acceptance Function:
1. `new_hrmp_watermark` should be either
1. equal to the context's block number
1. or in `HrmpChannelDigests` for `P` an entry with the block number should exist
* `verify_outbound_hrmp(sender: ParaId, Vec<OutboundHrmpMessage>)`:
* `check_outbound_hrmp(sender: ParaId, Vec<OutboundHrmpMessage>)`:
1. Checks that there are at most `config.hrmp_max_message_num_per_candidate` messages.
1. Checks that horizontal messages are sorted by ascending recipient ParaId and there is no two horizontal messages have the same recipient.
1. For each horizontal message `M` with the channel `C` identified by `(sender, M.recipient)` check:
1. exists
1. `M`'s payload size doesn't exceed a preconfigured limit `C.limit_message_size`
1. `M`'s payload size summed with the `C.used_bytes` doesn't exceed a preconfigured limit `C.limit_used_bytes`.
1. `C.used_places + 1` doesn't exceed a preconfigured limit `C.limit_used_places`.
1. `M`'s payload size doesn't exceed a preconfigured limit `C.max_message_size`
1. `M`'s payload size summed with the `C.total_size` doesn't exceed a preconfigured limit `C.max_total_size`.
1. `C.msg_count + 1` doesn't exceed a preconfigured limit `C.max_capacity`.
Candidate Enactment:
@@ -197,17 +217,21 @@ Candidate Enactment:
1. For each horizontal message `HM` with the channel `C` identified by `(sender, HM.recipient)`:
1. Append `HM` into `HrmpChannelContents` that corresponds to `C` with `sent_at` equals to the current block number.
1. Locate or create an entry in `HrmpChannelDigests` for `HM.recipient` and append `sender` into the entry's list.
1. Increment `C.used_places`
1. Increment `C.used_bytes` by `HM`'s payload size
1. Increment `C.msg_count`
1. Increment `C.total_size` by `HM`'s payload size
1. Append a new link to the MQC and save the new head in `C.mqc_head`. Note that the current block number as of enactment is used for the link.
* `prune_hrmp(recipient, new_hrmp_watermark)`:
1. From `HrmpChannelDigests` for `recipient` remove all entries up to an entry with block number equal to `new_hrmp_watermark`.
1. From the removed digests construct a set of paras that sent new messages within the interval between the old and new watermarks.
1. For each channel `C` identified by `(sender, recipient)` for each `sender` coming from the set, prune messages up to the `new_hrmp_watermark`.
1. For each pruned message `M` from channel `C`:
1. Decrement `C.used_places`
1. Decrement `C.used_bytes` by `M`'s payload size.
1. Decrement `C.msg_count`
1. Decrement `C.total_size` by `M`'s payload size.
1. Set `HrmpWatermarks` for `P` to be equal to `new_hrmp_watermark`
> NOTE: That collecting digests can be inefficient and the time it takes grows very fast. Thanks to the aggresive
> parametrization this shouldn't be a big of a deal.
> If that becomes a problem consider introducing an extra dictionary which says at what block the given sender
> sent a message to the recipient.
* `prune_dmq(P: ParaId, processed_downward_messages)`:
1. Remove the first `processed_downward_messages` from the `DownwardMessageQueues` of `P`.
* `enact_upward_messages(P: ParaId, Vec<UpwardMessage>)`:
@@ -251,10 +275,10 @@ The following entry-points are meant to be used for HRMP channel management.
Those entry-points are meant to be called from a parachain. `origin` is defined as the `ParaId` of
the parachain executed the message.
* `hrmp_init_open_channel(recipient, max_places, max_message_size)`:
* `hrmp_init_open_channel(recipient, proposed_max_capacity, proposed_max_message_size)`:
1. Check that the `origin` is not `recipient`.
1. Check that `max_places` is less or equal to `config.hrmp_channel_max_places` and greater than zero.
1. Check that `max_message_size` is less or equal to `config.hrmp_channel_max_message_size` and greater than zero.
1. Check that `proposed_max_capacity` is less or equal to `config.hrmp_channel_max_capacity` and greater than zero.
1. Check that `proposed_max_message_size` is less or equal to `config.hrmp_channel_max_message_size` and greater than zero.
1. Check that `recipient` is a valid para.
1. Check that there is no existing channel for `(origin, recipient)` in `HrmpChannels`.
1. Check that there is no existing open channel request (`origin`, `recipient`) in `HrmpOpenChannelRequests`.
@@ -268,9 +292,15 @@ the parachain executed the message.
1. Append `(origin, recipient)` to `HrmpOpenChannelRequestsList`.
1. Add a new entry to `HrmpOpenChannelRequests` for `(origin, recipient)`
1. Set `sender_deposit` to `config.hrmp_sender_deposit`
1. Set `limit_used_places` to `max_places`
1. Set `limit_message_size` to `max_message_size`
1. Set `limit_used_bytes` to `config.hrmp_channel_max_size`
1. Set `max_capacity` to `proposed_max_capacity`
1. Set `max_message_size` to `proposed_max_message_size`
1. Set `max_total_size` to `config.hrmp_channel_max_total_size`
1. Send a downward message to `recipient` notifying about an inbound HRMP channel request.
- The DM is sent using `queue_downward_message`.
- The DM is represented by the `HrmpNewChannelOpenRequest` XCM message.
- `sender` is set to `origin`,
- `max_message_size` is set to `proposed_max_message_size`,
- `max_capacity` is set to `proposed_max_capacity`.
* `hrmp_accept_open_channel(sender)`:
1. Check that there is an existing request between (`sender`, `origin`) in `HrmpOpenChannelRequests`
1. Check that it is not confirmed.
@@ -283,12 +313,23 @@ the parachain executed the message.
1. Reserve the deposit for the `origin` according to `config.hrmp_recipient_deposit`
1. For the request in `HrmpOpenChannelRequests` identified by `(sender, P)`, set `confirmed` flag to `true`.
1. Increase `HrmpAcceptedChannelRequestCount` by 1 for `origin`.
1. Send a downward message to `sender` notifying that the channel request was accepted.
- The DM is sent using `queue_downward_message`.
- The DM is represented by the `HrmpChannelAccepted` XCM message.
- `recipient` is set to `origin`.
* `hrmp_close_channel(ch)`:
1. Check that `origin` is either `ch.sender` or `ch.recipient`
1. Check that `HrmpChannels` for `ch` exists.
1. Check that `ch` is not in the `HrmpCloseChannelRequests` set.
1. If not already there, insert a new entry `Some(())` to `HrmpCloseChannelRequests` for `ch`
and append `ch` to `HrmpCloseChannelRequestsList`.
1. Send a downward message to the opposite party notifying about the channel closing.
- The DM is sent using `queue_downward_message`.
- The DM is represented by the `HrmpChannelClosing` XCM message with:
- `initator` is set to `origin`,
- `sender` is set to `ch.sender`,
- `recipient` is set to `ch.recipient`.
- The opposite party is `ch.sender` if `origin` is `ch.recipient` and `ch.recipient` if `origin` is `ch.sender`.
## Session Change
polkadot/roadmap/implementers-guide/src/types/messages.md
-2
View File
@@ -42,8 +42,6 @@ that we use the first item tuple for the sender and the second for the recipient
is allowed between two participants in one direction, i.e. there cannot be 2 different channels
identified by `(A, B)`.
`HrmpChannelId` has a defined ordering: first `sender` and tie is resolved by `recipient`.
```rust,ignore
struct HrmpChannelId {
sender: ParaId,
polkadot/roadmap/implementers-guide/src/types/runtime.md
+8 -2
View File
@@ -77,18 +77,24 @@ struct HostConfiguration {
/// The deposit that the recipient should provide for accepting opening an HRMP channel.
pub hrmp_recipient_deposit: u32,
/// The maximum number of messages allowed in an HRMP channel at once.
pub hrmp_channel_max_places: u32,
pub hrmp_channel_max_capacity: u32,
/// The maximum total size of messages in bytes allowed in an HRMP channel at once.
pub hrmp_channel_max_size: u32,
pub hrmp_channel_max_total_size: u32,
/// The maximum number of inbound HRMP channels a parachain is allowed to accept.
pub hrmp_max_parachain_inbound_channels: u32,
/// The maximum number of inbound HRMP channels a parathread is allowed to accept.
pub hrmp_max_parathread_inbound_channels: u32,
/// The maximum size of a message that could ever be put into an HRMP channel.
///
/// This parameter affects the upper bound of size of `CandidateCommitments`.
pub hrmp_channel_max_message_size: u32,
/// The maximum number of outbound HRMP channels a parachain is allowed to open.
pub hrmp_max_parachain_outbound_channels: u32,
/// The maximum number of outbound HRMP channels a parathread is allowed to open.
pub hrmp_max_parathread_outbound_channels: u32,
/// The maximum number of outbound HRMP messages can be sent by a candidate.
///
/// This parameter affects the upper bound of size of `CandidateCommitments`.
pub hrmp_max_message_num_per_candidate: u32,
}
```