* Improve transaction submission
Before this pr the transaction pool validated each transaction, even if
the transaction was already known to the pool. This pr changes the
behavior to first check if we are already aware of a transaction and
thus, to only validate them if we don't know them yet. However, there is
still the possibility that a given transaction is validated multiple
times. This can happen if the transaction is added the first time, but
is not yet validated and added to the validated pool.
Besides that, this pr fixes the wrong metrics of gossiped transactions
in the network. It also moves some metrics to the transaction pool api,
to better track when a transaction actually is scheduled for validation.
* Make sure we don't submit the same transaction twice from the network concurrently
* Remove added listener call
* Feedback
* Ignore banned on resubmit
* slots: create primitives crate for consensus slots
* offences: add method to check if an offence is unknown
* babe: initial equivocation reporting implementation
* babe: organize imports
* babe: working equivocation reporting
* babe: add slot number to equivocation proof
* session: move duplicate traits to session primitives
* babe: move equivocation stuff to its own file
* offences: fix test
* session: don't have primitives depend on frame_support
* babe: use opaque type for key owner proof
* babe: cleanup client equivocation reporting
* babe: cleanup equivocation code in pallet
* babe: allow sending signed equivocation reports
* node: fix compilation
* fix test compilation
* babe: return bool on check_equivocation_proof
* babe: add test for equivocation reporting
* babe: add more tests
* babe: add test for validate unsigned
* babe: take slot number in generate_key_ownership_proof API
* babe: add benchmark for equivocation proof checking
* session: add benchmark for membership proof checking
* offences: fix babe benchmark
* babe: add weights based on benchmark results
* babe: adjust weights after benchmarking on reference hardware
* babe: reorder checks in check_and_report_equivocation
* WIP
* Making progress
* Almost ready
* Get service tests compiling
* Fix node screenshot
* Line widths
* Fix node cli tests
* Fix node cli warning
* ChainComponents -> ServiceComponents, fix tests
* make spawn_handle public
* Remove spawnnamed impl for taskmanager
* Move the keep alive stuff to the task manager
* Move the telemetry, base path, rpc keep_alive to the service builder
* Make the task manager keep alive an internal detail
* Rewrite the browser start_client future
* Remove run_node etc
* Revert my personal changes to browser-demo/build.sh
* use |config|
* Add a runtime_version function to SubstrateCli
* Reexport role and runtime version from sc cli
* Update Cargo.lock
* runtime_version -> native_runtime_version
* Pass chain spec to native_runtime_version for polkadot
* Fix line widths
* Traitify ServiceComponents Client
* transaction-pool: expose blocking api for tx submission
* service: separate ServiceBuilder::build for full and light
* service: add ServiceBuilder::build_common
* transaction-pool: extend docs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Use sign_with and stop using `Pair`
* PR feedback
* Remove clone
* Transfer ownership of public to sign_message
* Use Option
* Simplify code
* Fix error message
* Pass keystore as ref
* Pass keystore properly
* Fix tests
The `tree_route` generated by the import notification is only from the
old best block to the new best parent. This means, it does not contain
the new best block in `enacted()`. We need to prune the transactions of
the new best block "manually" to fix this bug.
Besides that, this pr also changed the `id` parameter of the `NewBlock`
chain event to `hash`. The hash of a block is unique in contrast to the
block number. (Block id can either be number or hash)
* Make tx pool aware of retracted fork blocks
* Make it compile
* Update client/transaction-pool/src/lib.rs
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Fix doc test
* Simplify the implementation
* Send tree route as arc to prevent heavy clones
* Switch to use `ExtrinsicHash` to make it more clear
* Fix benchmark
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Add ‘transaction_version’ to the signed transaction
This allows hardware wallets to know which transactions they can safely
sign. To reduce transaction size, I reduced it to a ‘u8’ from a ‘u32’.
Fixes#5951.
* Restore transaction_version to a u32
* Fix comments
`transaction_version` is not part of a tx, but is still signed.
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Fix the test suite
I had forgotten to change the production of transactions in the test
code.
* Fix benchmarks
* Improve docs for `CheckTxVersion` in `frame_system`
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Remove spurious cast
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* grandpa: wire up basic RPC call
* grandpa: make it compile against GRANDPA with expose round state
* grandpa: use shared voter state to expose RPC endpoint
* grandpa: restructure into nested structs
* grandpa: return background rounds too
* grandpa: return error when endpoint not ready
* grandpa: collect grandpa rpc deps
* grandpa: decide to use concrete AuthorityId in finality-grandpa-rpc
* grandpa: remove unncessary type annotation
* grandpa: move error code to const
* grandpa: remove unnecessary WIP comment
* grandpa: remove Id type parameter for SharedVoterState
* grandpa: update tests to add shared_voter_state in parameters
* grandpa: remove old deprecated test
* grandpa: fix getting the correct set_id
* grandpa: make SharedVoterState a struct
* grandpa: wrap shared_voter_state in rpc_setup
* grandpa: replace spaces with tabs
* grandpa: limit RwLock write attempt to 1 sec
* grandpa: add missing doc comments and remove some pub
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-Authored-By: Hernando Castano <HCastano@users.noreply.github.com>
* grandpa: update function name call after change in finality-grandpa
* grandpa: group pub use and only export voter::report
* grandpa: add missing docs
* grandpa: extract out structs used for json serialization
* grandpa: stick to u32 for fields intended for js
* grandpa: move Error type to its own file
* grandpa: group pub use better
* Apply code review suggestion
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* grandpa: use correct version of finality-granpda in rpc crate
* grandpa: add back basic rpc unit test
* grandpa: replace SharedVoterState::new() with empty()
* node: cleanup grandpa::SharedVoterState usage in macro
* grandpa: remove VoterState error variant
* grandpa: enable missing futures compat feature
* grandpa: fix typo in error variant
* grandpa: remove test_utils
* grandpa: allow mocking rpc handler components
* grandpa: rename serialized to report in rpc module
* grandpa: add proper test for RPC
* grandpa: update to finality-grandpa v0.12.1
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Hernando Castano <HCastano@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Expose that `BasicQueue` expects blocking spawn
Up to now `BasicQueue` expected a closure that to spawn a `Future`.
This was expected to be a closure that spawns a blocking future.
However, this wasn't documented anywhere. This pr introduces a new trait
`SpawnBlocking` that exposes this requirement to the outside.
* Feedback
* Drop client from sc-network and sc-client-db, move LongestChain to sc-client-api
* move leaves, cht, in_mem to sc-client-api, drop client from sc-finality-grandpa
* drop sc-service from sc-rpc
* drop sc-service from sc-consensus-aura
* drop sc-client from manual-seal and babe
* drop sc-client from utils/frame/rpc/system and utils/frame/benchmarking-cli
* drop sc-client from bin/node and bin/node-template
* drop sc-client
* fix tests
* remove check -p sc-client from gitlab.yml
* fix warnings
* fixes ui test
* fix light client tests
* adds associated Client type to AbstractService
* adds UsageProvider to Client
* fixed ui test, again
* tried and failed to get node-cli to compile for wasm
* thanks to tomaka for helping me get node-cli to compile for wasmm
* ui test pls pas 🙏🏾
* all tests passing 🪄
* no_run documentation code
* rm -f documentation code
* ClientProvider
* fix mega trait
* move LongestChain to sc-consensus, use adds minimal bounds to AbstractService::Client
* adds license to sc-consensus
Co-authored-by: Benjamin Kampmann <ben@parity.io>
* make new contructor
* add metrics to txpool
* fix review
* fix doc comment
* change to counters
* Update client/transaction-pool/src/metrics.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* Update client/transaction-pool/src/metrics.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* Update client/transaction-pool/src/metrics.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* Update client/transaction-pool/src/lib.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* Update client/transaction-pool/src/lib.rs
Co-Authored-By: Max Inden <mail@max-inden.de>
* use dedicated wrapper
Co-authored-by: Max Inden <mail@max-inden.de>
* client/authority-discovery: Allow to be run by sentry node
When run as a sentry node, the authority discovery module does not
publish any addresses to the dht, but still discovers validators and
sentry nodes of validators.
* client/authority-discovery/src/lib: Wrap lines at 100 characters
* client/authority-discovery: Remove TODO and unused import
* client/authority-discovery: Pass role to new unit tests
* client/authority-discovery: Apply suggestions
Co-Authored-By: André Silva <123550+andresilva@users.noreply.github.com>
* bin/node/cli/src/service: Use expressions instead of statements
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
* Split the Roles bitfield in three
* Forgot to include some changes
* Fix cli test
* More test fixes
* Oh God, merging master broke other tests
* Didn't run the doctests
* Address review
* I'm trying to fix the build blindly because it's taking a good hour to compile on my machine
* Address some review
* Also update the peerset's API to make sense
* Fix peerset tests
* Fix browser node
* client: distinguish between local and network authority
Co-authored-by: André Silva <andre.beat@gmail.com>
* Initial skeleton for offchain phragmen
* Basic compact encoding decoding for results
* add compact files
* Bring back Self::ensure_storage_upgraded();
* Make staking use compact stuff.
* First seemingly working version of reduce, full of todos
* Everything phragmen related works again.
* Signing made easier, still issues.
* Signing from offchain compile fine 😎
* make compact work with staked asssignment
* Evaluation basics are in place.
* Move reduce into crate. Document stuff
* move reduce into no_std
* Add files
* Remove other std deps. Runtime compiles
* Seemingly it is al stable; cycle implemented but not integrated.
* Add fuzzing code.
* Cleanup reduce a bit more.
* a metric ton of tests for staking; wip 🔨
* Implement a lot more of the tests.
* wip getting the unsigned stuff to work
* A bit gleanup for unsigned debug
* Clean and finalize compact code.
* Document reduce.
* Still problems with signing
* We officaly duct taped the transaction submission stuff. 🤓
* Deadlock with keys again
* Runtime builds
* Unsigned test works 🙌
* Some cleanups
* Make all the tests compile and stuff
* Minor cleanup
* fix more merge stuff
* Most tests work again.
* a very nasty bug in reduce
* Fix all integrations
* Fix more todos
* Revamp everything and everything
* Remove bogus test
* Some review grumbles.
* Some fixes
* Fix doc test
* loop for submission
* Fix cli, keyring etc.
* some cleanup
* Fix staking tests again
* fix per-things; bring patches from benchmarking
* better score prediction
* Add fuzzer, more patches.
* Some fixes
* More docs
* Remove unused generics
* Remove max-nominator footgun
* Better fuzzer
* Disable it ❌
* Bump.
* Another round of self-review
* Refactor a lot
* More major fixes in perThing
* Add new fuzz file
* Update lock
* fix fuzzing code.
* Fix nominator retain test
* Add slashing check
* Update frame/staking/src/tests.rs
Co-Authored-By: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
* Some formatting nits
* Review comments.
* Fix cargo file
* Almost all tests work again
* Update frame/staking/src/tests.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Fix review comments
* More review stuff
* Some nits
* Fix new staking / session / babe relation
* Update primitives/phragmen/src/lib.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Update primitives/phragmen/src/lib.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Update primitives/phragmen/compact/src/lib.rs
Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>
* Some doc updates to slashing
* Fix derive
* Remove imports
* Remove unimplemented tests
* nits
* Remove dbg
* Better fuzzing params
* Remove unused pref map
* Deferred Slashing/Offence for offchain Phragmen (#5151)
* Some boilerplate
* Add test
* One more test
* Review comments
* Fix build
* review comments
* fix more
* fix build
* Some cleanups and self-reviews
* More minor self reviews
* Final nits
* Some merge fixes.
* opt comment
* Fix build
* Fix build again.
* Update frame/staking/fuzz/fuzz_targets/submit_solution.rs
Co-Authored-By: Gavin Wood <gavin@parity.io>
* Update frame/staking/src/slashing.rs
Co-Authored-By: Gavin Wood <gavin@parity.io>
* Update frame/staking/src/offchain_election.rs
Co-Authored-By: Gavin Wood <gavin@parity.io>
* Fix review comments
* fix test
* === 🔑 Revamp without staking key.
* final round of changes.
* Fix cargo-deny
* Update frame/staking/src/lib.rs
Co-Authored-By: Gavin Wood <gavin@parity.io>
Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
* fork-tree: prune returns all pruned node data
* epoch-changes: split EpochHeader vs epoch data
* EpochChanges::viable_epoch and add missing comments
* Incoperate the new epoch_changes interface for BABE
* Fix BABE tests
* Fix fork-tree pruning issue
* Fix tests
* Fix pruning algorithm
* fork-tree: implement map function for mapping one value type to another
* Add migration script for new epoch changes scheme
* Update utils/fork-tree/src/lib.rs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* Update client/consensus/slots/src/lib.rs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* Remove authorities_len.is_none check, which is duplicate of unwrap_or(false)
* Update client/consensus/epochs/src/lib.rs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* Update client/consensus/epochs/src/lib.rs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* No trailing ; for return statement
* Use VERSION_KEY for migration
* Fix issues that removed nodes are not properly added into removed list
* Add comments indicating end_slot is non-inclusive
* fork-tree: use &mut F for map type declaration
* Add tests for v0 epoch_changes migration
* Fix babe RPC tests
Co-authored-by: André Silva <andre.beat@gmail.com>
* removes use of sc_client::Client from sc-rpc
* remove Client impl from sc-finality-benches
* remove client impl from sc-finality-grandpa
* read_proof accepts iterator
* remove generic Executor param from ExecutorProvider
* fix long ass line
* code style changes
* merge with master
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
* removes use of sc_client::Client from sc_finality_grandpa
* code formatting
* code formatting
* removes use of sc_client::Client from sc_finality_grandpa
* removes use of sc_client::Client from sc_consensus_babe
* Update client/consensus/babe/src/lib.rs
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* babe_epochAuthorship
remove test-helpers from sp-keyring, bump spec_version, impl_version
* bump Cargo.lock
* add BabeRPC to node-rpc
* rename to BabeApi, remove err_derive
* pass &ServiceBuilder to with_rpc_extensions callback
* sc-consensus-babe-rpc
* Update client/consensus/babe/src/lib.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Better docs, code style chanegs
Co-Authored-By: André Silva <andre.beat@gmail.com>
* new line at the end of Cargo.toml
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
`GossipEngine` spawns two tasks, one for a periodic tick, one to forward
messages from the network to subscribers. These tasks hold an `Arc` to a
`GossipEngineInner`.
To reduce the amount of shared ownership (locking) this patch integrates
the two tasks into a `Future` implementation on the `GossipEngine`
struct. This `Future` implementation can now be called from a single
owner, e.g. the `finality-grandpa` `NetworkBridge`.
As a side effect this removes the requirement on the `network-gossip`
crate to spawn tasks and thereby removes the requirement on the
`finality-grandpa` crate to spawn any tasks.
This is part of a greater effort to reduce the number of owners of
components within `finality-grandpa`, `network` and `network-gossip` as
well as to reduce the amount of unbounded channels. For details see
d9837d7dd, 5f80929dc and 597c0a6c4.
* babe: pass epoch data via intermediates
* Switch to use Box<dyn Any> for intermediates
* Set intermediate.epoch to be Option
* Fix proposer should put out an empty intermediate
* Remove unnecessary encode/decode
* Add EpochData to block_import_params in slot worker
* Fix aura compile
* Fix integration test
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).
This touches a few important things:
- the startup of the async task with tokei:
This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
- increased the version of structopt and tokei
- no more use of structopt internal's API
- less use of generics
Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want
Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
Bastian Köcher