Simplified BEEFY worker logic based on the invariant that GRANDPA
will always finalize 1st block of each new session, meaning BEEFY
worker is guaranteed to receive finality notification for the
BEEFY mandatory blocks.
Under these conditions the current design is as follows:
- session changes are detected based on BEEFY Digest present in
BEEFY mandatory blocks,
- on each new session new `Rounds` of voting is created, with old
rounds being dropped (for gossip rounds, last 3 are still alive
so votes are still being gossiped),
- after processing finality for a block, the worker votes if
a new voting target has become available as a result of said
block finality processing,
- incoming votes as well as self-created votes are processed
and signed commitments are created for completed BEEFY voting
rounds,
- the worker votes if a new voting target becomes available
once a round successfully completes.
On worker startup, the current validator set is retrieved from
the BEEFY pallet. If it is the genesis validator set, worker
starts voting right away considering Block #1 as session start.
Otherwise (not genesis), the worker will vote starting with
mandatory block of the next session.
Later on when we add the BEEFY initial-sync (catch-up) logic,
the worker will sync all past mandatory blocks Signed Commitments
and will be able to start voting right away.
BEEFY mandatory block is the block with header containing the BEEFY
`AuthoritiesChange` Digest, this block is guaranteed to be finalized
by GRANDPA.
This session-boundary block is signed by the ending-session's
validator set. Next blocks will be signed by the new session's
validator set. This behavior is consistent with what GRANDPA does
as well.
Also drop the limit N on active gossip rounds. In an adversarial
network, a bad actor could create and gossip N invalid votes with
round numbers larger than the current correct round number. This
would lead to votes for correct rounds to no longer be gossiped.
Add unit-tests for all components, including full voter consensus
tests.
Signed-off-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: David Salami <Wizdave97>
* Upgraded dependencies
* Adapting code to scale v3
* Empty commit to trigger CI
* Triggering CI
* Fixing UI test
* Remove superfluous dev-dep added by #9228
* Cryout for CI
* starting
* Updated from other branch.
* setting flag
* flag in storage struct
* fix flagging to access and insert.
* added todo to fix
* also missing serialize meta to storage proof
* extract meta.
* Isolate old trie layout.
* failing test that requires storing in meta when old hash scheme is used.
* old hash compatibility
* Db migrate.
* runing tests with both states when interesting.
* fix chain spec test with serde default.
* export state (missing trie function).
* Pending using new branch, lacking genericity on layout resolution.
* extract and set global meta
* Update to branch 4
* fix iterator with root flag (no longer insert node).
* fix trie root hashing of root
* complete basic backend.
* Remove old_hash meta from proof that do not use inner_hashing.
* fix trie test for empty (force layout on empty deltas).
* Root update fix.
* debug on meta
* Use trie key iteration that do not include value in proofs.
* switch default test ext to use inner hash.
* small integration test, and fix tx cache mgmt in ext.
test failing
* Proof scenario at state-machine level.
* trace for db upgrade
* try different param
* act more like iter_from.
* Bigger batches.
* Update trie dependency.
* drafting codec changes and refact
* before removing unused branch no value alt hashing.
more work todo rename all flag var to alt_hash, and remove extrinsic
replace by storage query at every storage_root call.
* alt hashing only for branch with value.
* fix trie tests
* Hash of value include the encoded size.
* removing fields(broken)
* fix trie_stream to also include value length in inner hash.
* triedbmut only using alt type if inner hashing.
* trie_stream to also only use alt hashing type when actually alt hashing.
* Refactor meta state, logic should work with change of trie treshold.
* Remove NoMeta variant.
* Remove state_hashed trigger specific functions.
* pending switching to using threshold, new storage root api does not
make much sense.
* refactoring to use state from backend (not possible payload changes).
* Applying from previous state
* Remove default from storage, genesis need a special build.
* rem empty space
* Catch problem: when using triedb with default: we should not revert
nodes: otherwhise thing as trie codec cannot decode-encode without
changing state.
* fix compilation
* Right logic to avoid switch on reencode when default layout.
* Clean up some todos
* remove trie meta from root upstream
* update upstream and fix benches.
* split some long lines.
* UPdate trie crate to work with new design.
* Finish update to refactored upstream.
* update to latest triedb changes.
* Clean up.
* fix executor test.
* rust fmt from master.
* rust format.
* rustfmt
* fix
* start host function driven versioning
* update state-machine part
* still need access to state version from runtime
* state hash in mem: wrong
* direction likely correct, but passing call to code exec for genesis
init seem awkward.
* state version serialize in runtime, wrong approach, just initialize it
with no threshold for core api < 4 seems more proper.
* stateversion from runtime version (core api >= 4).
* update trie, fix tests
* unused import
* clean some TODOs
* Require RuntimeVersionOf for executor
* use RuntimeVersionOf to resolve genesis state version.
* update runtime version test
* fix state-machine tests
* TODO
* Use runtime version from storage wasm with fast sync.
* rustfmt
* fmt
* fix test
* revert useless changes.
* clean some unused changes
* fmt
* removing useless trait function.
* remove remaining reference to state_hash
* fix some imports
* Follow chain state version management.
* trie update, fix and constant threshold for trie layouts.
* update deps
* Update to latest trie pr changes.
* fix benches
* Verify proof requires right layout.
* update trie_root
* Update trie deps to latest
* Update to latest trie versioning
* Removing patch
* update lock
* extrinsic for sc-service-test using layout v0.
* Adding RuntimeVersionOf to CallExecutor works.
* fmt
* error when resolving version and no wasm in storage.
* use existing utils to instantiate runtime code.
* Patch to delay runtime switch.
* Revert "Patch to delay runtime switch."
This reverts commit 67e55fee468f1a0cda853f5362b22e0d775786da.
* useless closure
* remove remaining state_hash variables.
* Remove outdated comment
* useless inner hash
* fmt
* fmt and opt-in feature to apply state change.
* feature gate core version, use new test feature for node and test node
* Use a 'State' api version instead of Core one.
* fix merge of test function
* use blake macro.
* Fix state api (require declaring the api in runtime).
* Opt out feature, fix macro for io to select a given version
instead of latest.
* run test nodes on new state.
* fix
* Apply review change (docs and error).
* fmt
* use explicit runtime_interface in doc test
* fix ui test
* fix doc test
* fmt
* use default for path and specname when resolving version.
* small review related changes.
* doc value size requirement.
* rename old_state feature
* Remove macro changes
* feature rename
* state version as host function parameter
* remove flag for client api
* fix tests
* switch storage chain proof to V1
* host functions, pass by state version enum
* use WrappedRuntimeCode
* start
* state_version in runtime version
* rust fmt
* Update storage proof of max size.
* fix runtime version rpc test
* right intent of convert from compat
* fix doc test
* fix doc test
* split proof
* decode without replay, and remove some reexports.
* Decode with compatibility by default.
* switch state_version to u8. And remove RuntimeVersionBasis.
* test
* use api when reading embedded version
* fix decode with apis
* extract core version instead
* test fix
* unused import
* review changes.
Co-authored-by: kianenigma <kian@parity.io>
* Statically register host WASM functions
* Fix `substrate-test-client` compilation
* Move `ExtendedHostFunctions` to `sp-wasm-interface`
* Fix `sp-runtime-interface` tests' compilation
* Fix `sc-executor-wasmtime` tests' compilation
* Use `runtime_interface` macro in `test-runner`
* Fix `sc-executor` tests' compilation
* Reformatting/`rustfmt`
* Add an extra comment regarding the `H` generic arg in `create_runtime`
* Even more `rustfmt`
* Depend on `wasmtime` without default features in `sp-wasm-interface`
* Bump version of `sp-wasm-interface` to 4.0.1
* Bump `sp-wasm-interface` in `Cargo.lock` too
* Bump all of the `sp-wasm-interface` requirements to 4.0.1
Maybe this will appease cargo-unleash?
* Revert "Bump all of the `sp-wasm-interface` requirements to 4.0.1"
This reverts commit 0f7ccf8e0f371542861121b145ab87af6541ac30.
* Make `cargo-unleash` happy (maybe)
* Use `cargo-unleash` to bump the crates' versions
* Align to review comments
* Introduce `SecretUri`
* `inspect-key`: Adds support for `expect-public`
`expect-public` can be used to check that a given secret uri corresponds to the given public key.
This is mainly useful when the secret uri is protected by a password and a new derived account
should be generated. With `--expect-public` the user can pass the public key/account-id of the
"base" secret uri aka the one without any derivation to ensure the correct password was inserted.
* Fixes
* 🤦
* Apply suggestions from code review
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Review feedback
* FMT
* Bump the versions
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* mark template and utils as non-publish
* switch to development version for testing
* activate unleash check
* maybe if I disable all rules...
* Fix isolated compilation of `max-encoded-len-derive` with `syn`
error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
--> src/lib.rs:88:29
|
88 | .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
| --------- ^^ ----------------------------------- _
| |
| syn::Path
error: aborting due to previous error
For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`
* WIP: bump changes crates since v3 tag to next breaking
cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0
FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`
* Update lockfile
* WIP: Bump sp-transaction-pool as well
* WIP: Bump sp-offchain as well
* WIP: Bump frame-system-rpc-runtime-api as well
* WIP: Bump sp-authority-discovery as well
* Manually deactivate dev-deps before `cargo unleash check`
Otherwise we run into `Cycle detected` error.
* Bump sp-consensus-slots
* Add missing Cargo.lock change
* Bump sp-consensus-vrf as well
* Bump sp-keyring as well
* Bump sp-consensus-pow as well
* Try to speed up the `unleash-check` job
Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help
* fixup: Ensure the temp target dir exists for unleash check
* Bump pallet-transaction-payment-rpc-runtime-api as well
Needed for Polkadot
* Bump pallet-transaction-payment-rpc as well
Needed for Polkadot
* Try updating crates after patching in the Polkadot CI job
* Use another approach to update patched Substrate crates
* Try to update all sp-core versions in Polkadot CI job
* Simplify sp-core version checking
* Apply another shellcheck lint
* Just do the simplest thing I guess
* Welp don't do --offline then
* Clean up `unleash-check` job triggers
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* Fix a note in unleash-check cache step
* Add a note about temporary optimization in cargo-unleash
* Pin a newer version of cargo-unleash
Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* bump a bunch of deps in parity-common
* primitive-types 0.10.0
* update Cargo.lock
* downgrade a few more
* this is unlikely to help
* try something
* Checkmate, Atheists!
* Make grandpa work
* Introduce `SharedData`
* Add test and fix bugs
* Switch to `SharedData`
* Make grandpa tests working
* More Babe work
* Make it async
* Fix fix
* Use `async_trait` in sc-consensus-slots
This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.
* Make grandpa tests compile
* More Babe tests work
* Fix network test
* Start fixing service test
* Finish service-test
* Fix sc-consensus-aura
* Fix fix fix
* More fixes
* Make everything compile *yeah*
* Fix build when we have Rust 1.51
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/common/src/shared_data.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update client/consensus/babe/src/tests.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Fix warning
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Init `RuntimeLogger` automatically for each runtime api call
This pr change the runtime api in such a way to always and automatically
enable the `RuntimeLogger`. This enables the user to use `log` or
`tracing` from inside the runtime to create log messages. As logging
introduces some extra code and especially increases the size of the wasm
blob. It is advised to disable all logging completely with
`sp-api/disable-logging` when doing the wasm builds for the on-chain
wasm runtime.
Besides these changes, the pr also brings most of the logging found in
frame to the same format "runtime::*".
* Update frame/im-online/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update test-utils/runtime/Cargo.toml
* Fix test
* Don't use tracing in the runtime, as we don't support it :D
* Fixes
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* upgrade a few dependencies
* make it compile at the expense of duplicate deps
* fix web-wasm and a warning
* introduce activate-wasm-bindgen-features crate
* Revert "introduce activate-wasm-bindgen-features crate"
This reverts commit 5a6e41e683f8a4844c0a735dcd08caabb2313f11.
* add getrandom feature to sc-consensus-aura
* Build every wasm crate in its own project with wasm-builder
Building all wasm crates in one workspace was a nice idea, however it
just introduced problems:
1. We needed to prune old members, but this didn't worked for old git
deps.
2. We locked the whole wasm workspace while building one crate. This
could lead to infinitely locking the workspace on a crash.
Now we just build every crate in its own project, this means we will
build the dependencies multiple times. While building the dependencies
multiple times, we still decrease the build time by around 30 seconds
for Polkadot and Substrate because of the new parallelism ;)
* Remove the requirement on wasm-builder-runner
This removes the requirement on wasm-builder-runner by using the new
`build_dep` feature of cargo. We use nightly anyway and that enables us
to use this feature. This solves the problem of not mixing
build/proc-macro deps with normal deps. By doing this we get rid off
this complicated project structure and can depend directly on
`wasm-builder`. This also removes all the code from wasm-builder-runner
and mentions that it is deprecated.
* Copy the `Cargo.lock` to the correct folder
* Remove wasm-builder-runner
* Update docs
* Fix deterministic check
Modified-by: Bastian Köcher <git@kchr.de>
* Try to make the ui test happy
* Switch to `SKIP_WASM_BUILD`
* Rename `SKIP_WASM_BINARY` to the correct name...
* Update utils/wasm-builder/src/builder.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update utils/wasm-builder/src/builder.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Release new version of wasm-builder-runner
Besides the new version this merges `SKIP_WASM_BUILD` and
`BUILD_DUMMY_WASM_BINARY`, this means a file is generated with
`SKIP_WASM_BUILD` if no file existed before.
* Update utils/wasm-builder-runner/Cargo.toml
* Update utils/wasm-builder-runner/src/lib.rs
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Update versions
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* checkpoint before removing CT from change trie
* before trie backend without tx
* undo
* Started no transaction, but would need using a different root
calculation method, out of the scope of this pr, will roll back.
* Remove NoTransaction.
* partially address review.
dummy stats implementation for no_std.
* Remove ChangeTrieOverlay.
* modified function
* Remove witness_ext
* need noops changes root
* update from cumulus branch
* line break
* remove warning
* line break
* From review: renamings and stats active in no std (except time).
* include cache, exclude change trie cache with individual temporary bad looking
no_std check
* little test
* fuse imports and filter_map prepare_extrinsics_input_inner fold.
* put back ExtInner into Ext, awkward double proto for new function.
* Apply suggestions from code review
* Update primitives/state-machine/Cargo.toml
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Bump version
* update test-utils crates to be ready for publishing
* adding changelog
* Adding automaticly generated READMEs
* fixing versions
* another version mishap
* Upgrade `kvdb-*`, `trie-db` and `memory-db`
The updates of `trie-db` and `memory-db` are important, as they fix the
non-deterministic build of Polkadot/Substrate.
* Change `trie-db` version
* Update test-utils/runtime/Cargo.toml
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update primitives/trie/Cargo.toml
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update `Cargo.lock` and `trie-bench`
* Fix UI tests
* Switch to fixed version of memory-db
Co-authored-by: Andronik Ordian <write@reusable.software>
dependabot[bot]