* Add command-line flag to enable yamux flow control.
We never enabled proper flow-control for yamux streams which may cause
stream buffers to exceed their configured limit when the stream
producer outpaces the stream consumer. By switching the window update
mode to on-read, producers will only receive more sending credit when
all data has been consumed from the stream buffer. Using this option
creates backpressure on producers. However depending on the protocol
there is a risk of deadlock, if both endpoints concurrently attempt to
send more data than they have credit for and neither side reads before
finishing their writes. To facilitate proper testing, this PR adds a
command-line flag `use-yamux-flow-control`.
* Replace comment with generic message.
* Make WASM browser thing compile
* Fix
* updated exit-future (github repo)
* Switch to broadcast crate
* Migrate client/cli
* Switch exit-future to modernize branch
* Small changes
* Switch to cargo version and fix fg tests
* fix basic-authorship
* Fix crash on grafana macro
* Fix grafana macro
* Switch node python version
* Disable record_metrics_slice in grafana macro on wasm
* Update client/grafana-data-source/src/lib.rs
* Revert "Update client/grafana-data-source/src/lib.rs"
This reverts commit 888009a8e0b7051bd4bfbbfdb0448bcf2e2aae93.
* Add wasm support for state machine
* Switch to my own libp2p version
* Revert "Switch to my own libp2p version"
This reverts commit ce613871b59264b3165b45c37943e6560240daa7.
* Revert "Add wasm support for state machine"
This reverts commit de7eaa0694d9534fc3b164621737968e9a6a7c5f.
* Add sc-browser
* Squash
* remove sc-browser
* Fix keystore on wasm
* stubs for removed functions to make env compatible with old runtimes
* Add test (that doesn't work)
* Fix build scripts
* Revert basic-authorship due to no panics
* Revert cli/informant
* Revert consensus
* revert offchain
* Update utils/browser/Cargo.toml
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* export console functions
* Add new chainspec
* Fix ws in chain spec
* revert chainspec
* Fix chainspec
* Use an Option<PathBuf> in keystore instead of cfg flags
* Remove crud
* Only use wasm-timer for instant and systemtime
* Remove telemetry changes
* Assuming this is ok
* Add a KeystoreConfig
* Add stubs back in
* Update libp2p
* Revert "Add stubs back in"
This reverts commit 4690cf1882aa0f99f7f00a58c4080c8aa9b77c36.
* Remove commented js again
* Bump kvdb-web version
* Fix cli
* Switch branch on futures-timer
* Fix tests
* Remove sc-client test build in check-web-wasm because there isn't a good way to build futures-timer with wasm-bindgen support in the build
* Remove more things ^^
* Switch branch on futures-timer back
* Put DB io stats behind a cfg flag
* Fix things
* Don't timeout transports on wasm
* Update branch of futures-timer and fix bad merge
* Spawn informant
* Fix network test
* Fix delay resets
* Changes
* Fix tests
* use wasm_timer for transaction pool
* Fixes
* Switch futures-timer to crates
* Only diagnose futures on native
* Fix sc-network-test tests
* Select log level in js
* Fix syncing ;^)
* Allow disabling colours in the informant
* Use OutputFormat enum for informant
* MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master
* Update futures-diagnose
* Revert "MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master"
This reverts commit baa4ffc94fd968b6660a2c17ba8113e06af15548.
* Pass whole chain spec in start_client
* Get Instant::now to work in transaction pool again
* Informant dep reordering
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
These are a few changes I missed during the refactoring.
1. Initialization issue and boilerplate
Most importantly: part of the `Configuration` initialization was done in `sc_cli::init`. This means the user can not benefit from this initialization boilerplate if they have multiple `Configuration` since `sc_cli::init` can only be called once.
2. Boilerplate for `VersionInfo` and `Configuration`
I'm also answering to the critic of @bkchr on the initialization using version: https://github.com/paritytech/substrate/pull/4692/files/bea809d4c14a2ede953227ac885e3b3f9771c548#r372047238 This will allow initializing a `Configuration` and provide the version by default.
3. Loading the `chain_spec` explicitly
In the past it was done automatically but in some cases we want to delay this. I moved the code to `Configuration.load_spec()` so it can be called later on. `chain_spec` can also be written directly to the `Configuration` without using this `load_spec` helper.
4. [deleted]
5. Fixing issue that prevents the user to override the port
In the refactoring I introduced a bug by mistake that could potentially prevent the CLI user to override the ports if defaults where provided for these ports (only on cumulus).
6. Change task_executor from Box to Arc
This is useful for cumulus where we have 2 nodes with 2 separate Configuration that need to spawn tasks to the same runtime.
7. Renamed TasksExecutorRequired to TaskExecutor
For consistency.
This is related to https://github.com/paritytech/cumulus/issues/24
This is the continuation (and hopefully the end of) #4692
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).
This touches a few important things:
- the startup of the async task with tokei:
This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
- increased the version of structopt and tokei
- no more use of structopt internal's API
- less use of generics
Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want
Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
* WASM runtime switch to import memory
Up to now runtimes have exported their memory. To unify it with
sandboxing, this pr switches runtimes to import memory as well.
From a functional perspective, exporting/importing memory makes no
difference to the runtime.
To provide backwards compatibility, WASM exported memory is still supported.
* Revert debug stuff
* Revert some stuff
* Make debug builds more usable
This pr makes debug builds more usable in terms of `cargo run -- --dev`.
1. `--dev` activates `--execution native`, iff `--execution` is not
given or no sub `--execution-*` is given.
2. It was probably a mistake to compile WASM in debug for a debug build.
So, we now build the WASM binary always as `release` (if not requested
differently by the user). So, we trade compilation time for a better
debug experience.
* Make sure we only overwrite default values
* Make it work
* Apply suggestion
* reset a couple of versions down from v0.2.0
* Unstablise browser-utils (very new) and grafana-data-source (going to be replaced with prometheus sometime)
* unmark a bunch more client crates for stabilization
* fix versions in Cargo.tomls
* Downgrade network-test
* Set frame-metadata version to `10.0.0`
* Hide and documented storage generator as internal api
* Downgrade `sp-externalities`
* Downgrade `sc-cli`
* Downgrade sc-executor et al to 0.8.0
* Downgrade sp-state-machine to 0.8.0
* Downgrade pallet-contracts et al to 0.8.0
* Downgrade sp-sandbox to 0.8.0
* downgrade pallet-evm to 0.8.0
* Downgrade pallet-staking to 0.8.0
2.0 should be implementation of lazy payout alongside all the fixes of
current module
* Downgrade node-transaction-factory to 0.8.0
* frame pallets are 2.0.0
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Wei Tang <accounts@that.world>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
* Switch service to futures03
* Fix tests
* Fix service test and cli
* Re-add Executor trait to SpawnTaskHandle
* Fix node-service
* Update babe
* Fix browser node
* Update aura
* Revert back to tokio-executor to fix runtime panic
* Add todo item
* Fix service tests again
* Timeout test futures
* Fix tests
* nits
* Fix service test
* Remove zstd patch
* Re-add futures01 to aura and babe tests as a dev-dep
* Change failing test to tee
* Fix node
* Upgrade tokio
* fix society
* Start switching grandpa to stable futures
* Revert "Start switching grandpa to stable futures"
This reverts commit 9c1976346237637effc07c13f7d0403daf5e71cf.
* Fix utils
* Revert substrate service test
* Revert gitlab
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
* fork-tree: rebalance fork tree by maximum branch height
* fork-tree: add docs for tree rebalancing
* fork-tree: add test
* babe: rebalance epoch changes tree on deserialization
* fork-tree: fix doc
* fork-tree: fix test
* Extend `Proposer` to optionally generate a proof of the proposal
* Something
* Refactor sr-api to not depend on client anymore
* Fix benches
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Apply suggestions from code review
* Introduce new `into_storage_changes` function
* Switch to runtime api for `execute_block` and don't require `H256`
anywhere in the code
* Put the `StorageChanges` into the `Proposal`
* Move the runtime api error to its own trait
* Adds `StorageTransactionCache` to the runtime api
This requires that we add `type NodeBlock = ` to the
`impl_runtime_apis!` macro to work around some bugs in rustc :(
* Remove `type NodeBlock` and switch to a "better" hack
* Start using the transaction cache from the runtime api
* Make it compile
* Move `InMemory` to its own file
* Make all tests work again
* Return block, storage_changes and proof from Blockbuilder::bake()
* Make sure that we use/set `storage_changes` when possible
* Add test
* Fix deadlock
* Remove accidentally added folders
* Introduce `RecordProof` as argument type to be more explicit
* Update client/src/client.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update primitives/state-machine/src/ext.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Integrates review feedback
* Remove `unsafe` usage
* Update client/block-builder/src/lib.rs
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* Update client/src/call_executor.rs
* Bump versions
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* Entirely update substrate-telemetry to futures 0.3
* Add a Closed error
* Update to libp2p 0.14
* More work
* More work
* More work
* More work
* Fix warnings
* Remove unwrap()
* Work on tests fixing
* Fix network tests
* Fix external network tests
* Update libp2p and restore Yamux in discovery test
* Ignore DNS if initializatio nfails
* Restore variables ordering
* Forgot browser-utils
* Fix downfall after merge
* Fix tests
* Make wasm-builder remove invalid members
Wasm-builder now removes members that point to packages that do not
exist anymore or that were renamed. Up to now, the build would fail and
required manual fixing.
* Fixes build
* clean up cargo.toml syntax
* bumping versions to 2.0
* bump networking to 0.8
* move consensus down to 0.8
* bump consensus pallets to 0.8.0, too
* Upping babe and aura pallets
* add remaining, missing version definitions
* missed some
* Adding script for rename, could be applicable for nodes on top of it, too
* add stderr and gitlab ci features
* apply script
* fix now minor details in expected stderr
* Update the Cargo.lock
* fix name: sc-transaction -> sc-tracing
* fix rename in script, too
* Implement all storage after prefix
* fix test, bump version and fix doc
* bump metadata version
* Update frame/support/procedural/src/storage/storage_struct.rs
* Move transaction pool to primitives
* move backend, errors into primitives
* remove unused client depencies
* Move rpc-api into primitives
* Move peerset back to client
* Move rpc/api back to client, move palette/support/rpc into utils
* move support-rpc into subfolder
* move system-rpc into utils
* move transaction-pool and -graph back into client
* fix broken imports
* Clean up test primitives
* Make support test utils independent of frame
* remove unnecessary node dependencies from service
* Reactivate dependency script:
- only enforce the now achieved status quo will remain
- allow for primitives to depend on /client for now without failing
- more discriptive error message so people understand, what it wants
- minor fix to differentiative between ../client and /client (which may be a subfolder)
- don't allow this to fail anylonger.
* fix doc comment
* 'Should not' rather than 'must not'.
* Revert unwanted dependency changes
* fix faulty import
* fixup derive_more version
* fix wrong import path
* Adding first rough ouline of the repository structure
* Remove old CI stuff
* add title
* formatting fixes
* move node-exits job's script to scripts dir
* Move docs into subdir
* move to bin
* move maintainence scripts, configs and helpers into its own dir
* add .local to ignore
* move core->client
* start up 'test' area
* move test client
* move test runtime
* make test move compile
* Add dependencies rule enforcement.
* Fix indexing.
* Update docs to reflect latest changes
* Moving /srml->/paint
* update docs
* move client/sr-* -> primitives/
* clean old readme
* remove old broken code in rhd
* update lock
* Step 1.
* starting to untangle client
* Fix after merge.
* start splitting out client interfaces
* move children and blockchain interfaces
* Move trie and state-machine to primitives.
* Fix WASM builds.
* fixing broken imports
* more interface moves
* move backend and light to interfaces
* move CallExecutor
* move cli off client
* moving around more interfaces
* re-add consensus crates into the mix
* fix subkey path
* relieve client from executor
* starting to pull out client from grandpa
* move is_decendent_of out of client
* grandpa still depends on client directly
* lemme tests pass
* rename srml->paint
* Make it compile.
* rename interfaces->client-api
* Move keyring to primitives.
* fixup libp2p dep
* fix broken use
* allow dependency enforcement to fail
* move fork-tree
* Moving wasm-builder
* make env
* move build-script-utils
* fixup broken crate depdencies and names
* fix imports for authority discovery
* fix typo
* update cargo.lock
* fixing imports
* Fix paths and add missing crates
* re-add missing crates
Toralf Wittner