* network: Use "one shot" protocol handler.
Add two new `NetworkBehaviour`s, one handling remote block requests
and another one to handle light client requests (both local and from
remote). The change is motivated by the desire to use multiple
substreams of a single connection for different protocols. To achieve
this, libp2p's `OneShotHandler` is used as a protocol handler in each
behaviour. It will open a fresh substream for the duration of the
request and close it afterwards. For block requests, we currently only
handle incoming requests from remote and tests are missing. For light
client handling we support incoming requests from remote and also
ported a substantial amount of functionality over from
`light_dispatch.rs` (including several tests). However the result lacks
in at least two aspects:
(1) We require external updates w.r.t. the best block per peer and
currently nothing updates this information.
(2) We carry a lot of peer-related state around.
Both aspects could be simplified by externalising peer selection and
just requiring a specific peer ID where the request should be sent to.
We still have to maintain some peer related state due to the way
libp2p's swarm and network behaviour work (e.g. we must make sure to
always issue `NetworkBehaviourAction::SendEvent`s to peers we are
connected to, otherwise the actions die a silent death.
Another change implemented here is the use of protocol buffers as the
encoding for network messages. Certain individual fields of messages
are still SCALE encoded. There has been some discussion about this
in another PR (https://github.com/paritytech/substrate/pull/3452), so
far without resolution.
* Uncomment `Behaviour::light_client_request`.
* Add license headers.
* Add command-line flag to enable yamux flow control.
We never enabled proper flow-control for yamux streams which may cause
stream buffers to exceed their configured limit when the stream
producer outpaces the stream consumer. By switching the window update
mode to on-read, producers will only receive more sending credit when
all data has been consumed from the stream buffer. Using this option
creates backpressure on producers. However depending on the protocol
there is a risk of deadlock, if both endpoints concurrently attempt to
send more data than they have credit for and neither side reads before
finishing their writes. To facilitate proper testing, this PR adds a
command-line flag `use-yamux-flow-control`.
* Replace comment with generic message.
* Pause Kademlia if too many connections
* Fix test
* Update client/network/src/discovery.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Change the limit
Co-authored-by: Toralf Wittner <tw@dtex.org>
* Make WASM browser thing compile
* Fix
* updated exit-future (github repo)
* Switch to broadcast crate
* Migrate client/cli
* Switch exit-future to modernize branch
* Small changes
* Switch to cargo version and fix fg tests
* fix basic-authorship
* Fix crash on grafana macro
* Fix grafana macro
* Switch node python version
* Disable record_metrics_slice in grafana macro on wasm
* Update client/grafana-data-source/src/lib.rs
* Revert "Update client/grafana-data-source/src/lib.rs"
This reverts commit 888009a8e0b7051bd4bfbbfdb0448bcf2e2aae93.
* Add wasm support for state machine
* Switch to my own libp2p version
* Revert "Switch to my own libp2p version"
This reverts commit ce613871b59264b3165b45c37943e6560240daa7.
* Revert "Add wasm support for state machine"
This reverts commit de7eaa0694d9534fc3b164621737968e9a6a7c5f.
* Add sc-browser
* Squash
* remove sc-browser
* Fix keystore on wasm
* stubs for removed functions to make env compatible with old runtimes
* Add test (that doesn't work)
* Fix build scripts
* Revert basic-authorship due to no panics
* Revert cli/informant
* Revert consensus
* revert offchain
* Update utils/browser/Cargo.toml
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* export console functions
* Add new chainspec
* Fix ws in chain spec
* revert chainspec
* Fix chainspec
* Use an Option<PathBuf> in keystore instead of cfg flags
* Remove crud
* Only use wasm-timer for instant and systemtime
* Remove telemetry changes
* Assuming this is ok
* Add a KeystoreConfig
* Add stubs back in
* Update libp2p
* Revert "Add stubs back in"
This reverts commit 4690cf1882aa0f99f7f00a58c4080c8aa9b77c36.
* Remove commented js again
* Bump kvdb-web version
* Fix cli
* Switch branch on futures-timer
* Fix tests
* Remove sc-client test build in check-web-wasm because there isn't a good way to build futures-timer with wasm-bindgen support in the build
* Remove more things ^^
* Switch branch on futures-timer back
* Put DB io stats behind a cfg flag
* Fix things
* Don't timeout transports on wasm
* Update branch of futures-timer and fix bad merge
* Spawn informant
* Fix network test
* Fix delay resets
* Changes
* Fix tests
* use wasm_timer for transaction pool
* Fixes
* Switch futures-timer to crates
* Only diagnose futures on native
* Fix sc-network-test tests
* Select log level in js
* Fix syncing ;^)
* Allow disabling colours in the informant
* Use OutputFormat enum for informant
* MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master
* Update futures-diagnose
* Revert "MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master"
This reverts commit baa4ffc94fd968b6660a2c17ba8113e06af15548.
* Pass whole chain spec in start_client
* Get Instant::now to work in transaction pool again
* Informant dep reordering
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* Remove usage of sc_network::Context trait
* Remove Context::send_consensus
* Pass &mut dyn Network instead of &dyn Network
* Move Validator traits and related to separate module
* Entirely update substrate-telemetry to futures 0.3
* Add a Closed error
* Update to libp2p 0.14
* More work
* More work
* More work
* More work
* Fix warnings
* Remove unwrap()
* Work on tests fixing
* Fix network tests
* Fix external network tests
* Update libp2p and restore Yamux in discovery test
* Ignore DNS if initializatio nfails
* Restore variables ordering
* Forgot browser-utils
* Fix downfall after merge
* Fix tests
* RPC forwarders for adding reserved peers
* implement service side of reserved-peer RPCs
* Clean up unnecessary format! invocation
Co-Authored-By: Niklas Adolfsson <niklasadolfsson1@gmail.com>
* add some tests for the new RPC methods
* remove redundant `data` field
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* In progress, runtime io must switch to future proof root +
child_specific (unique id) + u32 type.
* Switch interface, sr-io seems ok, rpc could use similar interface to
sr-io, genesis json broken if there is child trie in existing encoding
genesis.
* test from previous implementation.
* fix proving test.
* Restore Keyspacedb from other branch, only apply to child trie.
* Removing unneeded child_info from child root (child info are stored
if things changed, otherwhise the root does not change).
* Switch rpc to use same format as ext: more future proof.
* use root from child info for trie backend essence.
* Breaking long lines.
* Update doc and clean pr a bit.
* fix error type
* Restore removed doc on merge and update sr-io doc.
* Switch child storage api to use directly unique id, if managed id
where to be put in place, the api will change at this time.
* Clean deprecated host interface from child.
* Removing assertion on child info (can fail depending on root
memoization).
* merging child info in the overlay when possible.
* child iteration by prefix using child_info.
* Using ChainInfo in frame support. ChainInfo gets redesign to avoid
buffers allocation on every calls.
* Add length of root to the data of child info.
* comments
* Encode compact.
* Remove child info with root.
* Fix try_update condition.
* Comment Ext child root caching.
* Replace tuples by struct with field
* remove StorageTuple alias.
* Fix doc tests, and remove StorageOverlay and ChildStorageOverlay
aliases.
* take test-client off sp-consensus
* use test primitives rather than test client in authority discovery tests
* move runtime-interface tests
* don't forget to remove the dev-dependency
* remove more unneeded dev deps
* add changes_trie_config to test prrimitives
* Separate network crates from its integration tests
* Fix up consensus crates for networking test changes
* remove unnecessary dependencies
* remove unused addition
* remove unnecessary dev-dependencies
* fixing finality grandpa tests
* removing unnecessary executor dependencies
* client/authority-discovery: Add smoke tests for intervall_at
* client/authority-discovery: Fix interval_at to fire on start
* .maintain/sentry-node: Update docker compose file
* client: Introduce --sentry-nodes flag
Enable operators to specify the public addresses of sentry nodes infront
of a validator node so that the validator node can announce the sentry
node addresses instead of its own public addresses on the DHT via the
authority discovery module.
* client/authority-discovery: Break lines at 100 characters
Limit line length to 100 instead of 120 characters.
* Adding script for rename, could be applicable for nodes on top of it, too
* add stderr and gitlab ci features
* apply script
* fix now minor details in expected stderr
* Update the Cargo.lock
* fix name: sc-transaction -> sc-tracing
* fix rename in script, too
* Move transaction pool to primitives
* move backend, errors into primitives
* remove unused client depencies
* Move rpc-api into primitives
* Move peerset back to client
* Move rpc/api back to client, move palette/support/rpc into utils
* move support-rpc into subfolder
* move system-rpc into utils
* move transaction-pool and -graph back into client
* fix broken imports
* Clean up test primitives
* Make support test utils independent of frame
* remove unnecessary node dependencies from service
* Reactivate dependency script:
- only enforce the now achieved status quo will remain
- allow for primitives to depend on /client for now without failing
- more discriptive error message so people understand, what it wants
- minor fix to differentiative between ../client and /client (which may be a subfolder)
- don't allow this to fail anylonger.
* fix doc comment
* 'Should not' rather than 'must not'.
* Revert unwanted dependency changes
* fix faulty import
* fixup derive_more version
* fix wrong import path
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
Pierre Krieger