* Introduce collation fetching protocol
also move to mod.rs
* Allow `PeerId`s in requests to network bridge.
* Fix availability distribution tests.
* Move CompressedPoV to primitives.
* Request based collator protocol: validator side
- Missing: tests
- Collator side
- don't connect, if not connected
* Fixes.
* Basic request based collator side.
* Minor fix on collator side.
* Don't connect in requests in collation protocol.
Also some cleanup.
* Fix PoV distribution
* Bump substrate
* Add back metrics + whitespace fixes.
* Add back missing spans.
* More cleanup.
* Guide update.
* Fix tests
* Handle results in tests.
* Fix weird compilation issue.
* Add missing )
* Get rid of dead code.
* Get rid of redundant import.
* Fix runtime build.
* Cleanup.
* Fix wasm build.
* Format fixes.
Thanks @andronik !
Currently we will take all leaves and give that to the overseer on
startup, but this is a bad idea when the finality is lagging for
example. There can be many of unfinalized leaves, we don't even need to
look at anymore. To solve this, the pr adds a maximum of 4 leaves we
forward to the overseer and the pr also checks that we only pass uncles
of the best block.
* TODOs
* Add auctions.rs, comment on changes needed.
* Remove cruft from slots
* Remove more from auctions.rs
* More logic drafting in slots.
* More logic in slots.rs
* patch some errors
* more fixes
* last nit
* Cleanups in slots.rs
* Cleanups in slots.rs
* patches
* make build
* crowdloan to new api
* auction compile
* Use ParaId instead of FundIndex in Crowdloan (#2303)
* use paraid instead of fundindex
* Update crowdloan.rs
* check caller is manager
* Auction tests and fix build warnings.
* Configurable origin for initiating auctions
* Remove on_finalize
* #2303 (manual merge)
* Tests for Slots
* some registrar tests
* Apply suggestions from code review
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update runtime/common/src/slots.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Slots uses Registrar for CurrentChains
* swap works test
* on swap impl
* traitify parachain cleanup
* explicit lifecycle tracking for paras
* initial implementation of lifecycles and upgrades
* clean up a bit
* Update runtime/common/src/slots.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* fix doc comment
* more rigid lifecycle checks
* include paras which are transitioning, and lifecycle query
* format guide
* update api
* update guide
* explicit outgoing state, fix genesis
* handle outgoing with transitioning paras
* Revert "explicit lifecycle tracking for paras"
This reverts commit 4177af7ba473bbd9c26bccd861793f25265b6657.
* remove lifecycle tracking from registrar
* do not include transitioning paras in identifier
* Update paras_registrar.rs
* final patches to registrar
* Fix test
* use noop in test
* clean up pending swap on deregistration
* finish registrar tests
* Update roadmap/implementers-guide/src/runtime/paras.md
* Update roadmap/implementers-guide/src/runtime/paras.md
* Update roadmap/implementers-guide/src/runtime/paras.md
* Apply suggestions from code review
* Use matches macro
* Correct terms
* Apply suggestions from code review
* Remove direct need for Slots and Registrar from Crowdloan
* Rejig things slightly
* actions queue
* Revert "actions queue"
This reverts commit b2e9011ec8937d6c73e99292416c9692aeb30f73.
* Traitify Auction interface.
* Mockups and initial code for Crowdloan testing
* One test...
* collapse onboarding state
* fix some crowdloan tests
* one more
* start benchmarks for auctions
* benchmark bid
* fix more crowdloan tests
* onboard and begin retirement no longer exist
* Revert "onboard and begin retirement no longer exist"
This reverts commit 2e100fd94e3540bff5f172328b5d917896f1c6fc.
* Simplify crowdloan and make it work.
* Fixes
* fix some
* finish merge fixes
* fix refund bug in auctions
* Add traits to Registrar for tests and benchmarks
* fix more auction benchmarks
* Fix TestAuctioneer
* finish crowdloan benchmarks
* start setting up full integration tests
* expand integration tests
* finish basic integration test
* add more integration tests
* begin slots benchmarks
* start paras registrar benchmarks
* fix merge
* fix tests
* clean up paras registrar
* remove println
* remove outdated cleanup config
* update benchmarks
* Add WeightInfo
* enable runtime-benchmarks feature flag
* complete swap benchmark
* add parachains and onboarding into westend
* add benchmarks and genesis
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* fix benchmark execution
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=paras_registrar --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* Use `new_raise_len` in crowdloan on_initialize
* Update paras_registrar.rs
* fix westend merge
* impl on_swap for crowdloan
* Check fund exists before create
* update for crowdloan sig
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* slots on_initialize
* use integration tests environment for benchmarks
* fix hrmp event
* auction on_initialize
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* fix storage name in auctions
* add auction_index to winning data
* winning data takes into account current auction index
* remove println
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=slots --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* Revert "add auction_index to winning data"
* PastRandomness.
* Fixes
* Use new randomness
* fix use of randomness in auctions and runtime config
* expose consts
* fix auction test
* add deposit per byte for para registration
* basic swap integration test
* make swap test more comprehensive
* Add WinningVec for easier retrieval in the front-end.
* clean up `WinningVec` at the end
* Add event for when a new best bid comes in
* Fix propagation of winners in ending period
* fix benchmarks, refund weight in dissolve
* fix unused
* remove some TODOs
* setup opaque keys for paras in westend
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=crowdloan --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* remove unused
* cargo run --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=auctions --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
* back to regular runtime config
* use saturating math where user input can be
* better first slot check
* Update runtime/common/src/claims.rs
* update westend onswap impl
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
* runtime: use babe as randomness source
* runtime: fix randomness api
* bridges: fix random_seed
* parachains: use mock TestRandomness from frame_support_test
* parachains: use mock TestRandomness from frame_support_test
* runtime: update randomness source in test-runtime
* runtime: remove unused import
* parachains: add todo to audit usage of randomness api
* "Update Substrate"
Co-authored-by: parity-processbot <>
* Add a config field to babe epochs
* Fix test
* Add BABE_GENESIS_EPOCH_CONFIG consts
* Use PrimaryAndSecondaryVRFSlots and remove newlines
* Make epoch_configs Some
* Fix tests
* Fix test service tests
* Add a BabeEpochConfigMigrations OnRuntimeUpgrade
* Apply suggestions
* Use PrimaryAndSecondaryPlainSlots in kusama
* Remove migration from test runtime and rococo
* Add HasPalletPrefix
* Rename to BabePalletPrefix and change BabeApi -> Babe
* "Update Substrate"
* Update substrate
* Resolve parantheses errors
Co-authored-by: parity-processbot <>
* WIP: Whole subsystem test.
* New tests compile.
* Avoid needless runtime queries for no validator nodes.
* Make tx and rx publicly accessible in virtual overseer.
This simplifies mocking in some cases, as tx can be cloned, but rx can
not.
* Whole subsystem test working.
* Update node/network/availability-distribution/src/session_cache.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update node/network/availability-distribution/src/session_cache.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Document better what `None` return value means.
* Get rid of BitVec dependency.
* Update Cargo.lock
* Hopefully fixed implementers guide build.
Co-authored-by: Andronik Ordian <write@reusable.software>
* Lol
* Add the features
* Remove some more runtime logger init calls
* Make companion check work
* Revert "Make companion check work"
This reverts commit a255c798076466c0fa20a4db713fc712772c2b4d.
* Update Substrate
* WIP
* availability distribution, still very wip.
Work on the requesting side of things.
* Some docs on what I intend to do.
* Checkpoint of session cache implementation
as I will likely replace it with something smarter.
* More work, mostly on cache
and getting things to type check.
* Only derive MallocSizeOf and Debug for std.
* availability-distribution: Cache feature complete.
* Sketch out logic in `FetchTask` for actual fetching.
- Compile fixes.
- Cleanup.
* Format cleanup.
* More format fixes.
* Almost feature complete `fetch_task`.
Missing:
- Check for cancel
- Actual querying of peer ids.
* Finish FetchTask so far.
* Directly use AuthorityDiscoveryId in protocol and cache.
* Resolve `AuthorityDiscoveryId` on sending requests.
* Rework fetch_task
- also make it impossible to check the wrong chunk index.
- Export needed function in validator_discovery.
* From<u32> implementation for `ValidatorIndex`.
* Fixes and more integration work.
* Make session cache proper lru cache.
* Use proper lru cache.
* Requester finished.
* ProtocolState -> Requester
Also make sure to not fetch our own chunk.
* Cleanup + fixes.
* Remove unused functions
- FetchTask::is_finished
- SessionCache::fetch_session_info
* availability-distribution responding side.
* Cleanup + Fixes.
* More fixes.
* More fixes.
adder-collator is running!
* Some docs.
* Docs.
* Fix reporting of bad guys.
* Fix tests
* Make all tests compile.
* Fix test.
* Cleanup + get rid of some warnings.
* state -> requester
* Mostly doc fixes.
* Fix test suite.
* Get rid of now redundant message types.
* WIP
* Rob's review remarks.
* Fix test suite.
* core.relay_parent -> leaf for session request.
* Style fix.
* Decrease request timeout.
* Cleanup obsolete errors.
* Metrics + don't fail on non fatal errors.
* requester.rs -> requester/mod.rs
* Panic on invalid BadValidator report.
* Fix indentation.
* Use typed default timeout constant.
* Make channel size 0, as each sender gets one slot anyways.
* Fix incorrect metrics initialization.
* Fix build after merge.
* More fixes.
* Hopefully valid metrics names.
* Better metrics names.
* Some tests that already work.
* Slightly better docs.
* Some more tests.
* Fix network bridge test.
* Add an optional verifier to crowdloan
* add tests
* verify signatures
* benchmark with signatures
* update crowdloan benchmark
* try to get keystore working in test
* rewrite to avoid traits
* Use MultiSignature and MultiSigner
* refactor and update benchmarks
* optimize check order
* fix no_std build
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* add tracing to approval voting
* notify if session info is not working
* add dispute period to chain specs
* propagate genesis session to parachains runtime
* use `on_genesis_session`
* protect against zero cores in computation
* tweak voting rule to be based off of best and add logs
* genesis configuration should use VRF slots only
* swallow more keystore errors
* add some docs
* make validation-worker args non-optional and update clap
* better tracing for bitfield signing and provisioner
* pass amount of bits in bitfields to inclusion instead of recomputing
* debug -> warn for some logs
* better tracing for availability recovery
* a little av-store tracing
* bridge: forward availability recovery messages
* add missing try_from impl
* some more tracing
* improve approval distribution tracing
* guide: hold onto pending approval messages until NewBlocks
* Hold onto pending approval messages until NewBlocks
* guide: adjust comment
* process all actions for one wakeup at a time
* vec
* fix network bridge test
* replace randomness-collective-flip with Babe
* remove PairNotFound
* Use KeyValueDB in approval-voting
* use KVDB instead of AuxStore
* add rocksdb to cargo toml
* add a Config struct
* create new DB in service
* fix dep for regular node
* make optional
* post merge fix
Co-authored-by: Andronik Ordian <write@reusable.software>
As futures most of the time are catching panics and we don't check this,
it could happen that we have some statement that isn't correct but the
test succeeds successfully.
* integrate approval voting into overseer
* expose public API and make keystore arc
* integrate overseer in service
* guide: `ApprovedAncestor` returns block number
* return block number along with hash from ApprovedAncestor
* introduce a voting rule for reporting on approval checking
* integrate the delay voting rule
* Rococo configuration
* fix compilation and add slack
* fix web-wasm build
* tweak parameterization
* migrate voting rules to asycn
* remove hack comment
* Update shared-memory to new version & refactor
This two are combined in a single commit because the new version of
shared-memory doesn't provide the used functionality anymore.
Therefore in order to update the version of this crate we implement the
functionality that we need by ourselves, providing a cleaner API along
the way.
* Significantly decrease the required memory for a workspace
For some reason it was allocating an entire GiB of memory. I suspect
this has something to do with the current memory size limit of a PVF
execution environment (the prior name suggests that). However, we don't
need so much memory anywhere near that amount.
In fact, we could reduce the allocated size even more, but that maybe
for the next time.
* Unlink shmem just after opening
That will make sure that we don't leak the shmem accidentally.
* Do not compile workspace mod for androind and wasm
* Address some review comments
* Fix the test runner
* Fix missed +1 for the attached flag
* Use .expect rather than .unwrap
* Add a rustdoc for the workspace module
* fixup! Use .expect rather than .unwrap
* Add some doc comments to pub members
* Warn on error removing shm_unlink
* Change the alignment implementation
* Fix the comment nit
Shaun Wang