4842faf65d
Elastic scaling: runtime dependency tracking and enactment ( #3479 )
...
Changes needed to implement the runtime part of elastic scaling:
https://github.com/paritytech/polkadot-sdk/issues/3131 ,
https://github.com/paritytech/polkadot-sdk/issues/3132 ,
https://github.com/paritytech/polkadot-sdk/issues/3202
Also fixes https://github.com/paritytech/polkadot-sdk/issues/3675
TODOs:
- [x] storage migration
- [x] optimise process_candidates from O(N^2)
- [x] drop backable candidates which form cycles
- [x] fix unit tests
- [x] add more unit tests
- [x] check the runtime APIs which use the pending availability storage.
We need to expose all of them, see
https://github.com/paritytech/polkadot-sdk/issues/3576
- [x] optimise the candidate selection. we're currently picking randomly
until we satisfy the weight limit. we need to be smart about not
breaking candidate chains while being fair to all paras -
https://github.com/paritytech/polkadot-sdk/pull/3573
Relies on the changes made in
https://github.com/paritytech/polkadot-sdk/pull/3233 in terms of the
inclusion policy and the candidate ordering
---------
Signed-off-by: alindima <alin@parity.io >
Co-authored-by: command-bot <>
Co-authored-by: eskimor <eskimor@users.noreply.github.com >
2024-03-21 10:10:45 +00:00
367098bae5
Take into account size as well in weight limiting. ( #7369 )
...
* Take into account size as well in weight limiting.
* Fix logging.
* More logs.
* Remove randomized selection in provisioner
No longer supported by runtime.
* Fix and simplify weight calculation.
Random filtering of remote disputes got dropped.
* Make existing tests pass.
* Tests for size limiting.
* Fix provisioner.
* Remove rand dependency.
* Better default block length for tests.
* ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent
* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent
* ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent
* Update runtime/parachains/src/paras_inherent/mod.rs
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io >
* Update runtime/parachains/src/paras_inherent/mod.rs
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com >
* Add back missing line.
* Fix test.
* fmt fix.
* Add missing test annotation
---------
Co-authored-by: eskimor <eskimor@no-such-url.com >
Co-authored-by: command-bot <>
Co-authored-by: Tsvetomir Dimitrov <tsvetomir@parity.io >
Co-authored-by: Chris Sosnin <48099298+slumber@users.noreply.github.com >
2023-08-01 12:43:54 +00:00
4c0af07370
Transition to VM weights ( #7342 )
...
* ".git/.scripts/commands/bench-vm/bench-vm.sh" all westend
* ".git/.scripts/commands/bench-vm/bench-vm.sh" all polkadot
* ".git/.scripts/commands/bench-vm/bench-vm.sh" all kusama
---------
Co-authored-by: command-bot <>
2023-06-21 14:01:38 +01:00
596a9ccd02
Unify code paths of create_inherent and enter ( #7137 )
...
* Remove redundant enter call.
* Remove optionality in dispute signature checking
* Make enter_inner and create_inherent the same.
* Remove redundant metric.
* Unification: enter and create_inherent.
* Remove `enter_inner` function.
* Remove dead code.
* Remove redundant import.
* Remove dead code in disputes.
* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent
* ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent
* ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent
* Merge fix.
* Fix tests.
* Remove obsolete comment.
* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent
* ".git/.scripts/commands/bench/bench.sh" runtime westend runtime_parachains::paras_inherent
* ".git/.scripts/commands/bench/bench.sh" runtime kusama runtime_parachains::paras_inherent
* Review remarks, fixes.
* Guide updates.
* Fmt.
* ".git/.scripts/commands/bench/bench.sh" runtime polkadot runtime_parachains::paras_inherent
---------
Co-authored-by: eskimor <eskimor@no-such-url.com >
Co-authored-by: command-bot <>
2023-06-14 10:57:06 +02:00
958cf435b1
update weights ( #7298 )
...
* polkadot: update weights
* westend: update weights
* kusama: update weights
* rococo: update weights
2023-06-13 13:54:10 +02:00
d50ff483f8
update weights (sync with 0.9.42) ( #7149 )
...
* polkadot: update weights
* westend: update weights
* kusama: update weights
* rococo: update weights
2023-05-02 13:12:47 +00:00
64660ee8d2
Remove years from copyright notes ( #7034 )
...
* Happy New Year!
* Remove year entierly
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Remove years from copyright notice in the entire repo
---------
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
2023-04-08 20:38:35 +00:00
4d904951fd
update weights ( #6897 )
...
* polkadot: update weights
* westend: update weights
* kusama: update weights
* rococo: update weights
2023-03-17 10:09:21 +00:00
177bedc160
Migrate to Weight::from_parts ( #6794 )
...
* Migrate to from_parts
Have a look at https://github.com/ggwpez/substrate-scripts/blob/master/migrate-from-parts.py
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Adapt to master
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* update lockfile for {"substrate"}
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: parity-processbot <>
2023-03-03 00:32:30 +01:00
63d1641b1e
update weights ( #6798 )
...
* polkadot: update weights
* westend: update weights
* kusama: update weights
* rococo: update weights
2023-02-28 14:08:21 +00:00
0119a6dae1
New runners for weights and new weights ( #6508 )
...
* [DO NOT MERGE] test new vm runner for weights 2
* add image
* fix merge conflict
* apply weights
* new weights
Co-authored-by: parity-processbot <>
2023-01-24 16:35:26 +01:00
8ff50d0fa7
update weights (0.9.37) ( #6546 ) ( #6555 )
...
* kusama: update weights
* polkadot: update weights
* westend: update weights
* rococo: update weights
2023-01-19 13:43:19 +00:00
0185170137
update weights (sync with 0.9.33) ( #6362 )
...
* update weights (0.9.33) (#6299 )
* kusama: update weights
* polkadot: update weights
* rococo: update weights
* westend: update weights
* fix deps
* Resolve merge
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Reset Kusama whitelist weights
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
2022-12-08 13:23:24 +00:00
db0fc60344
update weights (sync with v0.9.29) ( #5989 )
...
* kusama: update weights
* polkadot: update weights
* westend: update weights
* rococo: update weights
* fixup
* revert block weights
2022-09-12 11:48:27 +00:00
41eff346cb
Companion of paritytech/substrate#12157 ( #5964 )
...
* Remove RefTimeWeight
* Fixes
* update lockfile for {"substrate"}
Co-authored-by: parity-processbot <>
2022-09-02 19:12:06 +00:00
e28bf2e476
Companion for Weight v1.5 Follow Up ( #5949 )
...
* updates
* remove new
* fix up some stuff
* fix cargo files
* fix
* fix template
* update lockfile for {"substrate"}
* Update block_weights.rs
* remove unused
* remove unused
Co-authored-by: parity-processbot <>
2022-09-01 19:00:51 +00:00
28e94d97dd
Companion for Weight v1.5 ( #5943 )
...
* fix to latest substrate pr
* update weights
* cargo build -p polkadot-runtime-parachains
* fix xcm-builder
* fix import
* fix a bunch
* fix a bunch of weight stuff
* kusama compile
* unused
* builds
* maybe fix
* cargo test -p polkadot-runtime-parachains
* xcm simulator example
* fix tests
* xcm sim fuzz
* fix runtime tests
* remove unused
* fix integration tests
* scalar div
* update lockfile for {"substrate"}
Co-authored-by: parity-processbot <>
2022-08-31 11:59:39 +00:00
83217b3e04
update weights ( #5911 )
...
* rococo: update weigths
* polkadot: update weigths
* kusama: update weigths
* westend: update weights
Co-authored-by: parity-processbot <>
2022-08-25 10:11:40 +00:00
cb82d21708
update weights ( #5844 )
...
* westend: update weights
* kusama: update weights
* polkadot: update weights
* rococo: update weights
* update BlockExecutionWeight
* kusama: readd phragmen remove_member_wrong_refund weight
* polkadot: readd phragmen remove_member_wrong_refund weight
2022-08-01 13:13:21 +00:00
1a8b087129
update weights ( #5767 )
...
* polkadot: update weights
* kusama: update weights
* westend: update weights
* rococo: update weights
* Reduce testing constants (#5787 )
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
2022-07-28 19:35:39 +00:00
8ef2b701c1
update weights ( #5704 )
...
* westend: update weights
* kusama: update weights
* polkadot: update weights
* rococo: update weights
2022-06-21 12:58:01 +02:00
3ea40ba1f7
update weights ( #5601 )
...
* polkadot: update weights
* kusama: update weights
* westend: update weights
* rococo: update weights
2022-05-30 12:41:11 +02:00
5e458f6acb
update weights ( #5507 )
...
* rococo: update weights
* polkadot: update weights
* kusama: update weights
* westend: update weights
2022-05-12 10:44:21 +00:00
bb3cc7b041
update weights ( #5361 )
...
* polkadot: update weights
* kusama: update weights
* rococo: update weights
* westend: update weights
2022-04-22 09:26:19 +00:00
8ba34ef1fe
update weights ( #5097 )
...
* westend: update weights
* kusama: update weights
* polkadot: update weights
* westend: update weights (production profile)
* kusama: update weights (production profile)
* polkadot: update weights (production profile)
* kusama: update weights (production profile pt 2)
* westend: update weights (production profile pt 2)
* fixup
* fixup
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=frame_system --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=frame_system --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=frame_system --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/
Co-authored-by: Parity Bot <admin@parity.io >
2022-03-16 16:47:22 +01:00
39310088b0
Run paras inherent benchmarks ( #4454 )
...
* Some new lines
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras_inherent.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
Co-authored-by: Parity Bot <admin@parity.io >
2021-12-06 20:12:12 +00:00
90e1273462
Inherent filtering follow up ( #4305 )
...
* Add feature more feature gating for benchmarking + tests
* New line
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
* Do not assume we use max validators per core
* Use kusama weights for rococo (hopefully temp)
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras_inherent.rs
* Add more validity votes when neccesary
* Some fixes for the last commit
* Restore westend weights
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/westend/src/weights/runtime_parachains_paras_inherent.rs
* Revert bad westend weights write
* Make sure to update val idx before skipping
* Fix validity vote range to max at group size'
* Temp setup for rococo
* cargo run --quiet --release --features runtime-benchmarks -- benchmark --chain=rococo-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./runtime/rococo/src/weights/runtime_parachains_paras_inherent.rs --header=./file_header.txt
* Augment generated Rococo weights
* Make it compile
* Revert range for enter_backed_candidates_variable
* Delete runtime/kusama/src/weights/runtime_paras_paras_inherent.rs
Co-authored-by: Parity Bot <admin@parity.io >
2021-11-23 23:46:37 +00:00
1aa6a4aba4
move paras inherent filtering to runtime ( #4028 )
...
* move things around, add filter methods
* validator keys, modify availability bitfields according to disputes
* simplify, keep the filter -> sanitize generic for both usecases
* minor
* assure tests still work, reduce changeset
* integration
* start entropy passing
* fixins
* compile, 1 failing test
* filter with coverage
* fixins
* Update runtime/parachains/src/paras_inherent.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com >
* slip of the pen
* improve test cases
* misc
* fix
* fixins
* test avoid extra into() calls in assert_noop!
* chores
* ff
* test fixup superfluous into call
* chore: pfmt
* improve apply_block_weight_limit to try to maximize the number of sufficiently backed
blocks and add extra bitfields in a round-robin fashion
* new code treats the lack of backed candidates as ok
* Use vrf based entropy
* fixup vrf random
* add warn
* slip of the pen
* fixup
* assure ordering
* rethink apply_weights
* mock
* use a closure as predicate check
* extract and use DisputedBitfield
* chore: simplify
* remove stray dbg
* chore: fmt
* address feedback
* fix test, halfway there
* stage1
* dbg stuff
* make group selection align
* fix session index
* fix wrongly returned candidates
* cleanup
* chore fmt
* fix ensure check
* make good case test work
* more tests for bitfields
* create sanitize_backed_candidates
* fixup tests
* update guide
* add check referenced in the guide
* improve weights code
* fmt
* fixins
* Update roadmap/implementers-guide/src/runtime/inclusion.md
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com >
* compiling + address review
* add comments
* fix weight calc
* address review comments and test failure
* fix
* fix: condition
* Fix random_sel function
* Fix overlength block check
* Zeke + Ladi commit for disputes filtering + integration test builder + runtime benchmarks + integration tests
* Add benchmarks for code upgrades
* Code upgrade bench; Feature gate TestWeightInfo
* Try and make CI happier
* Feature gate enter test to not(benchmarks)
* Make sure no unused imports/fn
* refactor, re-use, the beginning
* Fix issue with frame benchmarking dep compilation
* More precise feature gating for some derives
* integrate piece-wise
* foo
* fixins
* chore fmt
* fixins
* rename const generic
* Update runtime/parachains/src/paras_inherent.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com >
* Fix compilation
* limit to test
* remove unused spam slots
* spellcheck
* remove a tick, fix a typo
* Add Code upgrade weights
* comment improvements + >=
Co-authored-by: Zeke Mostov <z.mostov@gmail.com >
* remove another tick
* Update runtime/parachains/src/paras_inherent/benchmarking.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com >
* saturating fixins + some spaces
* fix
* benchmarking - preliminary results
* Add training wheels
* Refactor some early exit logic for enter
* Gracefully handle filtering bitfields & candidates (#4280 )
This updates the logic for sanitize_bitfields and sanitize_backed_candidates to never error when there is an issue, but instead to simply skip the problematic items.
* Refactor inherent data weight limiting logic (#4287 )
* Apply suggestions from code review
* Update runtime/parachains/src/builder.rs
Co-authored-by: Zeke Mostov <z.mostov@gmail.com >
* Update runtime/parachains/src/builder.rs
* Update runtime/parachains/src/paras_inherent.rs
* final pass
* Run cargo +nightly-2021-10-29 fmt
* Update implementors guide with `sanitize_*` & `enter` (#4294 )
* Make spell check happier
* Make wasm runtimes compile with benchmarks enabled (#4303 )
* comment stuff out, use old toml
* Seems to be working?
* Remove feature gating from builder
* Remove commented out stuff
* Remove generic from digest
* Update weight files for runtime
Co-authored-by: Robert Habermeier <rphmeier@gmail.com >
Co-authored-by: Zeke Mostov <32168567+emostov@users.noreply.github.com >
Co-authored-by: Lldenaurois <Ljdenaurois@gmail.com >
Co-authored-by: Zeke Mostov <z.mostov@gmail.com >
Co-authored-by: Bastian Köcher <info@kchr.de >
2021-11-16 18:39:39 +00:00
Alin Dima