Commit Graph

16327 Commits

Author SHA1 Message Date
Peter Goodspeed-Niklaus 21cec309a4 implement provisioner (#1473)
* sketch out provisioner basics

* handle provisionable data

* stub out select_inherent_data

* split runtime APIs into sub-chapters to improve linkability

* explain SignedAvailabilityBitfield semantics

* add internal link to further documentation

* some more work figuring out how the provisioner can do its thing

* fix broken link

* don't import enum variants where it's one layer deep

* make request_availability_cores a free fn in util

* document more precisely what should happen on block production

* finish first-draft implementation of provisioner

* start working on the full and proper backed candidate selection rule

* Pass number of block under construction via RequestInherentData

* Revert "Pass number of block under construction via RequestInherentData"

This reverts commit 850fe62cc0dfb04252580c21a985962000e693c8.

That initially looked like the better approach--it spent the time
budget for fetching the block number in the proposer, instead of
the provisioner, and that felt more appropriate--but it turns out
not to be obvious how to get the block number of the block under
construction from within the proposer. The Chain API may be less
ideal, but it should be easier to implement.

* wip: get the block under production from the Chain API

* add ChainApiMessage to AllMessages

* don't break the run loop if a provisionable data channel closes

* clone only those backed candidates which are coherent

* propagate chain_api subsystem through various locations

* add delegated_subsystem! macro to ease delegating subsystems

Unfortunately, it doesn't work right:

```
error[E0446]: private type `CandidateBackingJob` in public interface
   --> node/core/backing/src/lib.rs:775:1
    |
86  | struct CandidateBackingJob {
    | - `CandidateBackingJob` declared as private
...
775 | delegated_subsystem!(CandidateBackingJob as CandidateBackingSubsystem);
    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ can't leak private type
```

I'm not sure precisely what's going wrong, here; I suspect the problem is
the use of `$job as JobTrait>::RunArgs` and `::ToJob`; the failure would be
that it's not reifying the types to verify that the actual types are public,
but instead referring to them via `CandidateBackingJob`, which is in fact private;
that privacy is the point.

Going to see if I can generic my way out of this, but we may be headed for a
quick revert here.

* fix delegated_subsystem

The invocation is a bit more verbose than I'd prefer, but it's also
more explicit about what types need to be public. I'll take it as a win.

* add provisioning subsystem; reduce public interface of provisioner

* deny missing docs in provisioner

* refactor core selection per code review suggestion

This is twice as much code when measured by line, but IMO it is
in fact somewhat clearer to read, so overall a win.

Also adds an improved rule for selecting availability bitfields,
which (unlike the previous implementation) guarantees that the
appropriate postconditions hold there.

* fix bad merge double-declaration

* update guide with (hopefully) complete provisioner candidate selection procedure

* clarify candidate selection algorithm

* Revert "clarify candidate selection algorithm"

This reverts commit c68a02ac9cf42b3a4a28eb197d38633a40d0e3e6.

* clarify candidate selection algorithm

* update provisioner to implement candidate selection per the guide

* add test that no more than one bitfield is selected per validator

* add test that each selected bitfield corresponds to an occupied core

* add test that more set bits win conflicts

* add macro for specializing runtime requests; specailize all runtime requests

* add tests harness for select_candidates tests

* add first real select_candidates test, fix test_harness

* add mock overseer and test that success is possible

* add test that the candidate selection algorithm picks the right ones

* make candidate selection test somewhat more stringent
2020-08-06 18:12:15 +02:00
Sergei Shulepov 877a5059aa Implementer's guide: downward messages and HRMP, take 2 (#1503)
* First stab at downward messages.

That also includes a notion of horizontal messages.

* Add some structure to the router.

* Update `ValidationOutputs`

* Add `processed_downward_messages` to `ValidationOutputs`.

Forgot to check that in.

* s/AccountId/ParaId

* DownwardMessage::ParachainSpecfic

* s/ensure_horizontal_messages_fits/ensure_horizontal_messages_fit

* Clarify that Router called for each candidate

* Update the preamble for Router.

* Rewrite the relay-chain extrinsic routines

* Update gloassary

* Add DMP to the glossary

* If the queue is empty, `processed_downward_messages` can be 0

* WIP

* Add condemned list

* Pivot to message-storing channel based HRMP

* Finished draft

* Tidy up

* Remove a duplicate glossary entry

* Fix typo

* Fix wording to emphasize that the channel is unidirectional

* Proper decrement `HrmpOpenChannelRequestCount`

* Add a comment for `HrmpOpenChannelRequestCount`.

* Remove old configuration values.

* Be more specific about the para{chain,thread} hrmp chan limits.

* Fix indentation so the lists are rendendered properly

* "to answer **the**" question instead of "a"

* Add a missing call to `check_processed_downward_messages`

* Clean more stuff during offboarding

* Fix typo

* Fix typo for the config

* Add a call to `prune_dmq`

* Add explicit invariants for ingress/egress indexes

* Add comments for the sender/reciever deposit config fields

* Document various fields and structs in Router module

* More docs

* Missing docs in Candidate.md

* Tabs to spaces in router.md

* Apply Rob's suggestion

* Add the hrmp_ prefix to the router messages

* Those are entry points

* Use SessionIndex type for the `age` field

* Use a struct to represent `HrmpChannelId`

* Put only MQCs into the LocalValidationData

* Close request can be initiated by the runtime directly

* Close request can be initiated by the runtime directly

* tabs/spaces

* Maintain the list of the outgoing paras in Router

* Update roadmap/implementers-guide/src/runtime/inclusion.md

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* fix typo

* Remove an unnecessary pair of code quotes

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2020-08-06 17:29:05 +02:00
Bastian Köcher 8729fb7d4b Revert "Ignore checks for companion PRs (#1455)" (#1549) 2020-08-06 17:08:06 +02:00
Gavin Wood b1cf8a57da Bump Substrate (#1548)
* Bump Substrate

* Bumb substrate again

Co-authored-by: Ashley Ruglys <ashley.ruglys@gmail.com>
2020-08-06 13:42:05 +00:00
Pierre Krieger dbc66479f3 Delay network startup to after complete initialization (#6833)
* Delay network startup to after complete initialization

* Update client/service/src/builder.rs

Co-authored-by: Ashley <ashley.ruglys@gmail.com>

Co-authored-by: Ashley <ashley.ruglys@gmail.com>
2020-08-06 12:46:34 +00:00
Pierre Krieger 39ed1a2b99 Companion PR to delaying network startup to after initialization (#1547) 2020-08-06 14:30:19 +02:00
Cecile Tonglet dbde3c4630 Don't validate a block announcement when syncing (#177) 2020-08-06 13:30:55 +02:00
Hamza Tokuchi 9e438c2fc1 Add RPC Builder to Substrate Node Template (#6808)
* Pulled RPC from node and populated the node-template's RPC builder with one example implementation

* surpress build errror

* dead_code

* Fixed module usage, removed copyright, removed rpc builder for light client + some comments

* added a comment for rpc extension

* Update bin/node-template/node/src/rpc.rs

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update rpc.rs

* fix spacing

* more space to tabs

* more space to tabs

* Documenation nitpick

* Documentation nitpick

* Documentation nitpick

* Documentation nitpick

* Documentation nitpick

* pre-format

* Updated transaction payment API implemented for node template

* fix space and commented code

* fix long line

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Dan Forbes <dan@danforbes.dev>
2020-08-06 10:31:03 +00:00
Dan Forbes 1089e65494 Updated README (#176)
* Updated docs

* README 100-character line length

* We're keeping the name
2020-08-06 11:39:48 +02:00
André Silva 886d79e0cb grandpa: never overwrite current rounds voter state (#6823)
* grandpa: never overwrite current rounds voter state

* grandpa: add test for voter state overwrite
2020-08-06 11:30:29 +02:00
Cecile Tonglet e818944cbe Add SyncOracle to network's Service (#1543)
* WIP

* WIP

Forked at: e91642361133dba084a9c9b3efa45fcb6cd3c5ca
Parent branch: origin/rococo-branch

* WIP

Forked at: e91642361133dba084a9c9b3efa45fcb6cd3c5ca
Parent branch: origin/rococo-branch

* Apply suggestions from code review

* Update collator/src/lib.rs

* Update collator/src/lib.rs

* Update network/src/protocol/mod.rs
2020-08-06 10:15:21 +02:00
Cecile Tonglet 497772da18 Ignore checks for companion PRs (#1455) 2020-08-06 06:52:03 +02:00
Fedor Sakharov bd85b3c7d3 Add an Origin to parachains v1 (#1542) 2020-08-05 22:24:20 +03:00
s3krit 6d0f9df65a [CI] Fix draft release publishing (#1546)
Small oversight, realised I was checking `git log version..old_version` rather than the other way round. Not sure how this slipped through
2020-08-05 19:32:03 +02:00
Pierre Krieger ee35dc9415 Fix legacy substream fallback not working (#6826)
* Fix legacy substream fallback not working

* Make it nicer
2020-08-05 16:20:09 +00:00
André Silva 4a94fe860d service: remove collection of system/process metrics (#6822) 2020-08-05 15:58:07 +00:00
Gavin Wood 03e696ab76 Bump Substrate, version (#1541)
* Bump Substrate, version

* 0.8.22
2020-08-05 16:46:04 +02:00
Shawn Tabrizi 97ac45c2f2 Successful note_imminent_preimage is free (#6793)
* Successful `note_imminent_preimage` is free

* update docs

* Add test for duplicate preimage
2020-08-05 16:22:21 +02:00
s3krit 0a44e7a63a Update check_labels runtimenoteworthy label (#1540) 2020-08-05 15:41:16 +02:00
André Silva a7b6c91f80 revert enabling authority discovery by default (#1532) 2020-08-05 14:10:43 +02:00
Guillaume Thiolliere 8a37f60844 Remove generation of instance trait by decl_storage. (#6812)
* remove generation of instance trait, no breaking change

* doc

* doc

* Update frame/support/src/traits.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update frame/support/procedural/src/storage/instance_trait.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-08-05 11:37:01 +00:00
Shawn Tabrizi e4e0e79ad7 Better default balances weights (#6813) 2020-08-05 09:40:31 +00:00
Pierre Krieger 840781cddb Fix warning being printed by authority-discovery (#6820) 2020-08-05 09:16:19 +00:00
Robert Habermeier f554868cd4 Network Bridge Refactoring (#1535)
* rename protocol ID to network capability

* guide: `ProtocolId` -> `NetworkCapability`

* guide: remove `RegisterEventProducer`

* capabilities and expand on underlying network assumptions

* guide: create network.md types file

* guide: network bridge is aware of network messages

* revert changes in code

* Update roadmap/implementers-guide/src/SUMMARY.md

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* remove references to NetworkCapability

* Update roadmap/implementers-guide/src/types/network.md

Co-authored-by: Sergei Shulepov <sergei@parity.io>

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Sergei Shulepov <sergei@parity.io>
2020-08-05 09:04:05 +00:00
ddorgan 1e0ec19945 Use DNS hostnames for westend bootnodes (#1528) 2020-08-05 10:55:18 +02:00
ddorgan 0023111d9b Use DNS hostnames for flaming fir bootnodes (#6807)
* Use dns hostnames for flaming fir bootnodes

* Remove newline
2020-08-05 10:39:13 +02:00
Guillaume Thiolliere 106eee7bf1 Companion PR: add weightinfo for democracy (#1522)
* add weight for democracy

* update companion

* update companion

* remove democracy weight tests

* fix import

* fix name

* fix merge

* cargo update -p sp-io

* fix compile

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-08-04 21:17:23 +02:00
Guillaume Thiolliere b6505d743f pallet-democracy use of weightinfo (#6783)
* democracy use of weightinfo

* fix some doc and benchs

* todo generate from parity machine

* factorize and add license

* use final weights

* add slightly more sensible default weight

* refactor

* rename benchmark to avoid confusion

* just make remove_other_vote benchmark being the worst case of the extrinsic
2020-08-04 17:58:03 +00:00
Dan Forbes e739334f76 Rename cumulus-test-parachain-collator to rococo-collator (#173)
Ran `find . -type f -name "*" -print0 | xargs -0 sed -i -e "s/cumulus-test-parachain-collator/rococo-collator/g"`
2020-08-04 19:44:02 +02:00
Bastian Köcher 23c1954c74 Add parachains for rococo (#144)
* Add parachains for rococo

* Fix chain specs

* Update to revert log rotation

* Support selecting the chain in `export-genesis-state`

* Add subcommand for exporting the genesis wasm of a Parachain

* Update stuff

* Fix `export-genesis-wasm`

* Update the polkadot ref

* Add bootNodes address

* Add bootNodes address - track

* Add bootNodes address - trick

* Fix incorrect peer id (trick)

* Fixes https://github.com/paritytech/cumulus/issues/157

* Update chainspecs

* Update specs again

* Set correct sudo account

* Update properties

* Update readme

* Update `Cargo.lock`

* Switch to rococo-branch

Co-authored-by: Evaldo <contato@evaldofelipe.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
2020-08-04 17:33:20 +02:00
Shawn Tabrizi 73f09e5154 Unalias Substrate Imports (#1530)
* cargo.toml updates

* session and system

* more

* more

* more

* more

* more

* fix

* compiles

* fix tests

* fix more tests

* fix mock

* fix deleted space

* Update validation/Cargo.toml

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update Cargo.lock

* update rococo

* remove unused warning

* update add benchmarks

* rename weight file

* forgot a file

* Update chain_spec.rs

* Revert "remove unused warning"

This reverts commit 4227cd0d1525286fb466dccb817564c9b37f8645.

* fix merge

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-08-04 13:23:33 +00:00
Bastian Köcher f5fbe3c195 Fix block announcement validation (#171)
* Fix block announce

* Fix compilation
2020-08-04 15:19:36 +02:00
Bastian Köcher c01aa8bae8 Rewrite client handling (#1531)
* Rewrite client handling

We are supporting muliple polkadot-like chains and all have different
client types. This pr reworks the client handling by having all of them
in one enum combined. Besides that, there is added a special trait
`ExecuteWithClient` to use the internal client.

* Apply suggestions from code review

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Up the versions

* Fix Cargo.lock

* Fix merge conflict

* ......................

* ....v2

* yep

* I'm dumb...

* Browser lol

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2020-08-04 14:53:47 +02:00
Andronik Ordian 07a9d2de3f Integrate ChainApi with all messages (#1533)
* propagate chain_api subsystem through various locations

* add ChainApi to AllMessages

Co-authored-by: Peter Goodspeed-Niklaus <peter.r.goodspeedniklaus@gmail.com>
2020-08-04 12:32:42 +00:00
Bastian Köcher 5a4bca765e Add Rococo test network (#1363)
* Add Rococo test network

* Correct license header

* Add bootNodes address

* Fix compile errors

* Change bootNodes

* Change rococo bootNodes

* Add new bootNodes

* Fix bootNodes typo

* Remove public telemetry

* Add rococo-local chain spec

* Remove staking

* Remove staking from chain spec

* use rococo-staging chain spec and preserve bootnodes

* Same but with --raw

* update chain name and remove telemetry

* Empty commit to re-trigger CI

* Fix revision (temp. until I merge master)

* Revert to branch = master to avoid conflicts

* Revert to branch = master to avoid conflicts

* Disable test (temp)

* Revert to branch = master to avoid conflicts

* Revert to branch = master to avoid conflicts

* Change bootNodes IPs for second rococo network

* Revert "Disable test (temp)"

This reverts commit a159f12e3131d1a25dabb1a4d2834642f2bcdc26.

* Revert purchase

* Update chain spec

* FUUU

* Update service/src/lib.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Update runtime/rococo/src/lib.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Fix compilation

Co-authored-by: Evaldo <contato@evaldofelipe.com>
Co-authored-by: David Dorgan <david@parity.io>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2020-08-04 09:41:39 +00:00
Sergei Shulepov 9eb9d41418 Fix a typo parathreads -> parachains (#1529) 2020-08-04 11:25:48 +03:00
Shaopeng Wang 3a853d4dde Add mutate_exists to StorageDoubleMap. (#6704) 2020-08-04 10:02:24 +02:00
Fedor Sakharov 124c699cd4 Cleanup upcoming paras (#1527)
* Cleanup upcoming paras

* Also remove pending genesis

* Not there but here
2020-08-03 20:11:06 +00:00
Fedor Sakharov a9e3a1b11d Sudo wrapper for paras (#1517)
* Sudo wrapper for paras

* Move to separate module

* Add some docs
2020-08-03 21:32:57 +02:00
Sergei Shulepov 2afa2b035f Implementer's guide: notes on contextual execution (#1525)
* Add notes about contextual execution

* Clarify that `validation_data_hash` consists of global and local data

* Update roadmap/implementers-guide/src/runtime/inclusion.md

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Incorporate rphmeier's suggestion.

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2020-08-03 19:30:12 +00:00
Alex Siman c7d6a11460 Add ss58 address for Subsocial (#6800) 2020-08-03 18:04:56 +03:00
Pierre Krieger e449330e27 Add details to legacy requests (#6747) 2020-08-03 14:55:32 +00:00
Ashley 07facb13c2 Convert spaces to tabs (#6799) 2020-08-03 15:48:32 +02:00
Kian Paimani c36b47c242 Companion for substrate/6782 (#1523)
* Fix test-runtime

* Remove duplicate tests
2020-08-03 15:26:24 +02:00
Kian Paimani 1a84ad87c0 Add integrity test for slash defer duration (#6782)
* Add integrity test for slash defer duration

* Wrap in externalities

* Update frame/staking/src/lib.rs
2020-08-03 13:26:09 +00:00
Shawn Tabrizi dd6ef0a3a5 Improve Benchmark Writer: Remove Unused Components, Remove Multiply by Zero, Files Split by Pallet (#6785)
* initial improvements

* better file management, ignore unused components

* Output warning when components unused

* update comment

* Write even when base weight is zero

* remove unwrap where possible

* Dont sort components to dedup

* undo delete

* improve clarity of unused components

* remove unused dep

* Update Process.json
2020-08-03 12:41:54 +00:00
Arkadiy Paronyan 09ce64bf24 Sort out validation errors (#1516)
* Sort out validation errors

* Typo

* Fixed wasm/android build

* Fixed bad merge
2020-08-03 10:45:26 +00:00
Alexander Theißen 6671d017d6 seal: Fix and improve error reporting (#6773)
* seal: Rework ext_transfer, ext_instantiate, ext_call error handling

* Deny calling plain accounts (must use transfer now)
* Return proper module error rather than ad-hoc strings
* Return the correct error codes from call,instantiate (documentation was wrong)
* Make ext_transfer fallible again to make it consistent with ext_call

* seal: Improve error messages on memory access failures

* seal: Convert contract trapped to module error

* seal: Add additional tests for transfer, call, instantiate

These tests verify that those functions return the error types
which are declared in its docs.

* Make it more pronounced that to_execution_result handles trap_reason

* Improve ReturnCode docs

* Fix whitespace issues in wat files

* Improve ReturnCode doc

* Improve ErrorOrigin doc and variant naming

* Improve docs on ExecResult and ExecError

* Encode u32 sentinel value as hex

* with_nested_context no longer accepts an Option for trie

* Fix successful typo

* Rename InvalidContractCalled to NotCallable
2020-08-03 10:03:22 +00:00
Max Inden 0553dabe32 client/network: Fix wrong metric help text (#6794)
The `sub_libp2p_kademlia_query_duration` metric only has the dimension
`type` not `protocol`.
2020-08-03 09:08:06 +00:00
Bastian Köcher 277fd75179 Fix transaction payment runtime api companion (#1519)
* Fix transaction payment runtime api companion

Companion for Substrate pr: https://github.com/paritytech/substrate/pull/6792

* 'Update substrate'

Co-authored-by: parity-processbot <>
2020-08-03 09:06:03 +00:00