* Migrate wasmtime backend to wasmtime-api
* Port to a newer version of wasmtime
* Update to the latest changes.
* Rejig the sandbox module a bit
* Materialze
* Fixes.
* executor wasm_runtime fix
* Refactor everything
* More refactoring
* Even more refactorings
* More cleaning.
* Update to the latest wasmtime
* Reformat
* Renames
* Refactoring and comments.
* Docs
* Rename FunctionExecutor to host.
* Imrpove docs.
* fmt
* Remove panic
* Assert the number of arguments are equal between wasmtime and hostfunc.
* Comment a possible panic if there is no corresponding value variant.
* Check signature of the entrypoint.
* Use git version of wasmtime
* Refine and doc the sandbox code.
* Comment RefCells.
* Update wasmtime to the latest-ish master.
This may solve a problem with segfaults.
* Apply suggestions from code review
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Use full SHA1 hash of wasmtime commit.
* Add a panic message.
* Add some documentation
* Update wasmtime version to include SIGSEGV fix
* Update to crates.io version of wasmtime
* Make it work.
* Move the creation of memory into `InstanceWrapper::new`
* Make `InstanceWrapper` !Send & !Sync
* Avoid using `take_mut`
* Update client/executor/wasmtime/Cargo.toml
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Limit maximum size of memory.
* Rename `init_state` to `with_initialized_state`
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Give perthigns the trait it always deserved.
* Make staking and phragmen work with the new generic per_thing
* Make everything work together 🔨
* a bit of cleanup
* Clean usage
* Bump.
* Fix name
* fix grumbles
* hopefully fix the ui test
* Some grumbles
* revamp traits again
* Better naming again.
* full block import benchmark
* try rocksdb cache
* add profiling helper
* use random keyring instead of zero caching
* update docs
* add more io stats
* remove last sentence
* add ci job to see
* Update primitives/keyring/src/sr25519.rs
Co-Authored-By: Marcio Diaz <marcio.diaz@gmail.com>
* switch to 100tx-block
* remove ci script
Co-authored-by: Marcio Diaz <marcio@parity.io>
`GossipEngine` spawns two tasks, one for a periodic tick, one to forward
messages from the network to subscribers. These tasks hold an `Arc` to a
`GossipEngineInner`.
To reduce the amount of shared ownership (locking) this patch integrates
the two tasks into a `Future` implementation on the `GossipEngine`
struct. This `Future` implementation can now be called from a single
owner, e.g. the `finality-grandpa` `NetworkBridge`.
As a side effect this removes the requirement on the `network-gossip`
crate to spawn tasks and thereby removes the requirement on the
`finality-grandpa` crate to spawn any tasks.
This is part of a greater effort to reduce the number of owners of
components within `finality-grandpa`, `network` and `network-gossip` as
well as to reduce the amount of unbounded channels. For details see
d9837d7dd, 5f80929dc and 597c0a6c4.
* network: Use "one shot" protocol handler.
Add two new `NetworkBehaviour`s, one handling remote block requests
and another one to handle light client requests (both local and from
remote). The change is motivated by the desire to use multiple
substreams of a single connection for different protocols. To achieve
this, libp2p's `OneShotHandler` is used as a protocol handler in each
behaviour. It will open a fresh substream for the duration of the
request and close it afterwards. For block requests, we currently only
handle incoming requests from remote and tests are missing. For light
client handling we support incoming requests from remote and also
ported a substantial amount of functionality over from
`light_dispatch.rs` (including several tests). However the result lacks
in at least two aspects:
(1) We require external updates w.r.t. the best block per peer and
currently nothing updates this information.
(2) We carry a lot of peer-related state around.
Both aspects could be simplified by externalising peer selection and
just requiring a specific peer ID where the request should be sent to.
We still have to maintain some peer related state due to the way
libp2p's swarm and network behaviour work (e.g. we must make sure to
always issue `NetworkBehaviourAction::SendEvent`s to peers we are
connected to, otherwise the actions die a silent death.
Another change implemented here is the use of protocol buffers as the
encoding for network messages. Certain individual fields of messages
are still SCALE encoded. There has been some discussion about this
in another PR (https://github.com/paritytech/substrate/pull/3452), so
far without resolution.
* Uncomment `Behaviour::light_client_request`.
* Add license headers.
* Add command-line flag to enable yamux flow control.
We never enabled proper flow-control for yamux streams which may cause
stream buffers to exceed their configured limit when the stream
producer outpaces the stream consumer. By switching the window update
mode to on-read, producers will only receive more sending credit when
all data has been consumed from the stream buffer. Using this option
creates backpressure on producers. However depending on the protocol
there is a risk of deadlock, if both endpoints concurrently attempt to
send more data than they have credit for and neither side reads before
finishing their writes. To facilitate proper testing, this PR adds a
command-line flag `use-yamux-flow-control`.
* Replace comment with generic message.
* Pause Kademlia if too many connections
* Fix test
* Update client/network/src/discovery.rs
Co-Authored-By: Toralf Wittner <tw@dtex.org>
* Change the limit
Co-authored-by: Toralf Wittner <tw@dtex.org>
* Run offchain workers at particular hash, not number.
* Don't run if not new best.
* Don't run if not new best.
* Update client/service/src/builder.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Update client/service/src/builder.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Update client/service/src/builder.rs
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Make WASM browser thing compile
* Fix
* updated exit-future (github repo)
* Switch to broadcast crate
* Migrate client/cli
* Switch exit-future to modernize branch
* Small changes
* Switch to cargo version and fix fg tests
* fix basic-authorship
* Fix crash on grafana macro
* Fix grafana macro
* Switch node python version
* Disable record_metrics_slice in grafana macro on wasm
* Update client/grafana-data-source/src/lib.rs
* Revert "Update client/grafana-data-source/src/lib.rs"
This reverts commit 888009a8e0b7051bd4bfbbfdb0448bcf2e2aae93.
* Add wasm support for state machine
* Switch to my own libp2p version
* Revert "Switch to my own libp2p version"
This reverts commit ce613871b59264b3165b45c37943e6560240daa7.
* Revert "Add wasm support for state machine"
This reverts commit de7eaa0694d9534fc3b164621737968e9a6a7c5f.
* Add sc-browser
* Squash
* remove sc-browser
* Fix keystore on wasm
* stubs for removed functions to make env compatible with old runtimes
* Add test (that doesn't work)
* Fix build scripts
* Revert basic-authorship due to no panics
* Revert cli/informant
* Revert consensus
* revert offchain
* Update utils/browser/Cargo.toml
Co-Authored-By: Benjamin Kampmann <ben@gnunicorn.org>
* export console functions
* Add new chainspec
* Fix ws in chain spec
* revert chainspec
* Fix chainspec
* Use an Option<PathBuf> in keystore instead of cfg flags
* Remove crud
* Only use wasm-timer for instant and systemtime
* Remove telemetry changes
* Assuming this is ok
* Add a KeystoreConfig
* Add stubs back in
* Update libp2p
* Revert "Add stubs back in"
This reverts commit 4690cf1882aa0f99f7f00a58c4080c8aa9b77c36.
* Remove commented js again
* Bump kvdb-web version
* Fix cli
* Switch branch on futures-timer
* Fix tests
* Remove sc-client test build in check-web-wasm because there isn't a good way to build futures-timer with wasm-bindgen support in the build
* Remove more things ^^
* Switch branch on futures-timer back
* Put DB io stats behind a cfg flag
* Fix things
* Don't timeout transports on wasm
* Update branch of futures-timer and fix bad merge
* Spawn informant
* Fix network test
* Fix delay resets
* Changes
* Fix tests
* use wasm_timer for transaction pool
* Fixes
* Switch futures-timer to crates
* Only diagnose futures on native
* Fix sc-network-test tests
* Select log level in js
* Fix syncing ;^)
* Allow disabling colours in the informant
* Use OutputFormat enum for informant
* MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master
* Update futures-diagnose
* Revert "MallocSizeOf impl on transaction pool broke stuff because wasm_timer::Instant doesnt impl it so just revert the transaction pool to master"
This reverts commit baa4ffc94fd968b6660a2c17ba8113e06af15548.
* Pass whole chain spec in start_client
* Get Instant::now to work in transaction pool again
* Informant dep reordering
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
* Fix CLI setup again
We need to set `config_dir` and `database_path` for almost every
command.
This fixes `purge-chain` and also adds a test to make sure we don't
break it again.
* Adds missing test files
* Split methods
* babe: pass epoch data via intermediates
* Switch to use Box<dyn Any> for intermediates
* Set intermediate.epoch to be Option
* Fix proposer should put out an empty intermediate
* Remove unnecessary encode/decode
* Add EpochData to block_import_params in slot worker
* Fix aura compile
* Fix integration test
* Init epoch changes module
* Initial integration of new epoch changes module for BABE
* Fix all initial compile errors
* rename: digest -> digests
* Fix babe tests
* Bump impl_version
* Fix more test issues
* Remove test flag for tree
It unfortunately won't work for multiple crates.
* Update cargo lock
* Fix duplicate parking_lot version
* Add missing license header
These are a few changes I missed during the refactoring.
1. Initialization issue and boilerplate
Most importantly: part of the `Configuration` initialization was done in `sc_cli::init`. This means the user can not benefit from this initialization boilerplate if they have multiple `Configuration` since `sc_cli::init` can only be called once.
2. Boilerplate for `VersionInfo` and `Configuration`
I'm also answering to the critic of @bkchr on the initialization using version: https://github.com/paritytech/substrate/pull/4692/files/bea809d4c14a2ede953227ac885e3b3f9771c548#r372047238 This will allow initializing a `Configuration` and provide the version by default.
3. Loading the `chain_spec` explicitly
In the past it was done automatically but in some cases we want to delay this. I moved the code to `Configuration.load_spec()` so it can be called later on. `chain_spec` can also be written directly to the `Configuration` without using this `load_spec` helper.
4. [deleted]
5. Fixing issue that prevents the user to override the port
In the refactoring I introduced a bug by mistake that could potentially prevent the CLI user to override the ports if defaults where provided for these ports (only on cumulus).
6. Change task_executor from Box to Arc
This is useful for cumulus where we have 2 nodes with 2 separate Configuration that need to spawn tasks to the same runtime.
7. Renamed TasksExecutorRequired to TaskExecutor
For consistency.
This is related to https://github.com/paritytech/cumulus/issues/24
This is the continuation (and hopefully the end of) #4692
* block_import: switch to Box<dyn Any> for intermediates representation
* Use Cow and return Error instead of Option
* Remove unused error
* Distinguish NoIntermediate/InvalidIntermediate
* client/finality-grandpa: Add regression test observer polling network
Ensure `Future` implementation of `ObserverWork` is polling its
`NetworkBridge`. Regression test for bug introduced in d9837d7dd and
fixed in 504b4e89e.
When polled, `NetworkBridge` forwards reputation change requests from
the `GossipValidator` to the underlying `dyn Network`. This test
triggers a reputation change by calling `GossipValidator::validate` with
an invalid gossip message. After polling the `ObserverWork` which should
poll the `NetworkBridge`, the reputation change should be forwarded to
the test network.
* Nits
Co-authored-by: Max Inden <mail@max-inden.de>
It changes the way we extended the CLI functionalities of substrate to allow more flexibility. (If this was not clear, here is another version: it changes the `sc_cli` API to allow more flexibility).
This touches a few important things:
- the startup of the async task with tokei:
This was in node and node-template and I moved it to substrate. The idea is to have 1 time the code that handles unix signals (SIGTERM and SIGINT) properly. It is however possible to make this more generic to wait for a future instead and provide only a helper for the basic handling of SIGTERM and SIGINT.
- increased the version of structopt and tokei
- no more use of structopt internal's API
- less use of generics
Related to #4643 and https://github.com/paritytech/cumulus/pull/42: the implementation of "into_configuration" and "get_config" are similar but with better flexibility so it is now possible in cumulus to have the command-line arguments only of the run command for polkadot if we want
Related to https://github.com/paritytech/cumulus/issues/24 and https://github.com/paritytech/cumulus/issues/34 : it will now be possible to make a configuration struct for polkadot with some overrides of the default parameters much more easily.
Patch d9837d7dd reintegrated the periodic neighbor packet worker, by
moving its logic into a `Future` implementation on `NetworkBridge` which
needs to be polled by its upper layer.
Polling by the upper layer was implemented within the `Future`
implementation of the `VoterWork` struct but not within the `Future`
implementation of the `ObserverWork` struct. This patch adds polling of
the `NetworkBridge` to the latter.
Sergei Pepyakin