* Use tokio runtime handle instead of TaskExecutor abstraction
Before this pr we had the `TaskExecutor` abstraction which theoretically
allowed that any futures executor could have been used. However, this
was never tested and is currently not really required. Anyone running a
node currently only used tokio and nothing else (because this was hard
coded in CLI). So, this pr removes the `TaskExecutor` abstraction and
relies directly on the tokio runtime handle.
Besides this changes, this pr also makes sure that the http and ws rpc
server use the same tokio runtime. This fixes a panic that occurred when
you drop the rpc servers inside an async function (tokio doesn't like
that a tokio runtime is dropped in the async context of another tokio
runtime).
As we don't use any custom runtime in the http rpc server anymore, this
pr also removes the `rpc-http-threads` cli argument. If external parties
complain that there aren't enough threads for the rpc server, we could
bring support for increasing the thread count of the tokio runtime.
* FMT
* Fix try runtime
* Fix integration tests and some other optimizations
* Remove warnings
* Implemented `seal_ecdsa_recovery` function in the contract pallet.
Added benchmark and unit test.
* Run `cargo fmt`
* Skip fmt for slices
* Changes according comments in pull request.
* Fix build without `unstable-interface` feature
* Applied suggestion from the review
* Apply suggestions from code review
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
* Apply suggestions from code review
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
* Changed RecoveryFailed to EcdsaRecoverFailed
* Manually updated weights.rs
* Apply suggestions from code review
Co-authored-by: Michael Müller <mich@elmueller.net>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Michael Müller <mich@elmueller.net>
* Improve sandbox internal api
This improves the internal sandbox api for the executor implementations.
The main point is to hide the tls in the internal api and not having it
exposed to the outside.
This is especially needed for wasmtime 0.29.0
* Fmt
* Make it nicer
* Store the database in a role specific subdirectory
This is a cleaned up version of #8658 fixing #6880
polkadot companion: paritytech/polkadot#2923
* Disable prometheus in tests
* Also change p2p port
* Fix migration logic
* Use different identification file for rocks and parity db
Add tests for paritydb migration
* sp-utils => sc-utils
* cargo fmt
* These files are now in the client so should be licensed as GPL3
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Support multiple, mergable vesting schedules
* Update node runtime
* Remove some TODO design questions and put them as commennts
* Update frame/vesting/src/benchmarking.rs
* Syntax and comment clean up
* Create filter enum for removing schedules
* Dry vesting calls with do_vest
* Improve old benchmarks to account for max schedules
* Update WeightInfo trait and make dummy fns
* Add merge_schedule weights
* Explicitly test multiple vesting scheudles
* Make new vesting tests more more clear
* Apply suggestions from code review
* Update remove_vesting_schedule to error with no index
* Try reduce spacing diff
* Apply suggestions from code review
* Use get on vesting for bounds check; check origin first
* No filter tuple; various simplifications
* unwrap or default when getting user schedules
* spaces be gone
* ReadMe fixes
* Update frame/vesting/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* address some comments for docs
* merge sched docs
* Apply suggestions from code review
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* log error when trying to push to vesting vec
* use let Some, not is_some
* remove_vesting_schedule u32, not optin
* new not try_new, create validate builder; VestingInfo
* Merge prep: break out tests and mock
* Add files forgot to include in merge
* revert some accidental changes to merged files
* Revert remaining accidental file changes
* More revert of accidental file change
* Try to reduce diff on tests
* namespace Vesting; check key when key should not exist;
* ending_block throws error on per_block of 0
* Try improve merge vesting info comment
* Update frame/vesting/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* add validate + correct; handle duration > blocknumber
* Move vesting_info module to its own file
* Seperate Vesting/locks updates from writing
* Add can_add_vesting schedule
* Adjust min vested transfer to be greater than all ED
* Initial integrity test impl
* merge_finished_and_yet_to_be_started_schedules
* Make sure to assert storage items are cleaned up
* Migration initial impl (not tested)
* Correct try-runtime hooks
* Apply suggestions from code review
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* header
* WIP: improve benchmarks
* Benchmarking working
* benchmarking: step over max schedules
* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Simplify APIs by accepting vec; convert to bounded on write
* Test: build_genesis_has_storage_version_v1
* Test more error cases
* Hack to get polkadot weights to work; should revert later
* Improve benchmarking; works on polkadot
* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* WIP override storage
* Set storage not working example
* Remove unused tests
* VestingInfo: make public, derive MaxEndcodedLen
* Rename ending_block to ending_block_as_balance
* Superificial improvements
* Check for end block infinite, not just duration
* More superficial update
* Update tests
* Test vest with multi schedule
* Don't use half max balance in benchmarks
* Use debug_assert when locked is unexpected 0
* Implement exec_action
* Simplify per_block calc in vesting_info
* VestingInfo.validate in add_vesting_schedule & can_add_vesting_schedule
* Simplify post migrate check
* Remove merge event
* Minor benchmarking updates
* Remove VestingInfo.correct
* per_block accesor max with 1
* Improve comment
* Remoe debug
* Fix add schedule comment
* Apply suggestions from code review
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* no ref for should_remove param
* Remove unused vestingaction derive
* Asserts to show balance unlock in merge benchmark
* Remove unused imports
* trivial
* Fix benchmark asserts to handle non-multiple of 20 locked
* Add generate_storage_info
* migration :facepalm
* Remove per_block 0 logic
* Update frame/vesting/src/lib.rs
* Do not check for ending later than greatest block
* Apply suggestions from code review
* Benchmarks: simplify vesting schedule creation
* Add log back for migration
* Add note in ext docs explaining that all schedules will vest
* Make integrity test work
* Improve integrity test
* Remove unnescary type param from VestingInfo::new
* Remove unnescary resut for ending_block_as_balance
* Remove T param from ending_block_as_balance
* Reduce visibility of raw_per_block
* Remove unused type param for validate
* update old comment
* Make log a dep; log warn in migrate
* VestingInfo.validate returns Err(()), no T type param
* Try improve report_schedule_updates
* is_valid, not validate
* revert node runtime reorg;
* change schedule validity check to just warning
* Simplify merge_vesting_info return type
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Add warning for migration
* Fix indentation
* Delete duplicate warnings
* Reduce diff in node runtime
* Fix benchmark build
* Upgrade cargo.toml to use 4.0.0-dev
* Cleanup
* MaxVestingSchedulesGetter initial impl
* MinVestedTransfer getter inintial impl
* Test MaxVestingSchedules & MinVestedTransfer getters; use getters in benchmarks
* Run cargo fmt
* Revert MinVestedTransfer & MaxVestingSchedules getters; Add integrity test
* Make MAX_VESTING_SCHEDULES a const
* fmt
* WIP: benchmark improvements
* Finish benchmark update
* Add test for transfer to account with less than ed
* Rm min_new_account_transfer; move sp-io to dev-dep
* Reduce cargo.toml diff
* Explain MAX_VESTING_SCHEDULES choice
* Fix after merge
* Try fix CI complaints
* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* fmt
* trigger
* fmt
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>
* Add comments and refactor Sandbox module
* Adds some comments
* Add wasmtime instance to the sandbox and delegate calls
* Adds module imports stub
* WIP state holder via *mut
* My take at the problem
* Brings back invoke and instantiate implementation details
* Removes redundant bound
* Code cleanup
* Fixes invoke closure
* Refactors FunctionExecutor to eliminate lifetime
* Wraps `FunctionExecutor::sandbox_store` in `RefCell`
* Renames `FunctionExecutor::heap` to `allocator`
* Wraps `FunctionExecutor::allocator` in `RefCell`
* Refactors FunctionExecutor to `Rc<Inner>` pattern
* Implements scoped TLS for FunctionExecutor
* Fixes wasmi instancing
* Fixes sandbox asserts
* Makes sandbox compile after wasmtime API change
* Uses Vurich/wasmtime for the Lightbeam backend
* Uses wasmtime instead of wasmi for sandbox API results
* Refactors sandbox to use one of the execution backends at a time
* Fixes wasmtime module instantiation
* TEMP vurich branch stuff
* Adds wasmer impl stub
* Adds get global
* Fixes warnings
* Adds wasmer invoke impl
* Implements host function interface for wasmer
* Fixes wasmer instantiation result
* Adds workaround to remove debug_assert
* Fixes import object generation for wasmer
* Attempt to propagate wasmer::Store through sandbox::Store
* Wraps `sandbox::Store::memories` in `RefCell`
* Moves `sandbox::instantiate` to `sandbox::Store`
* Eliminate `RefCell<memories>`
* Implements `HostState::memory_get/set`, removes accidental `borrow_mut`
* Fixes sandbox memory handling for wasmi
* Fix memory allocation
* Resets Cargo.lock to match master
* Fixes compilation
* Refactors sandbox to use TLS for dispatch_thunk propagation to wasmer
* Pass dispatch thunk to the sandbox as a TLS
* Initialize dispatch thunk holder in `SandboxInstance`
* Comment out Wasmtime/Lightbeam sandbox backend
* Revert wasmtime back to mainstream
* Adds SandboxExecutionMethod enum for cli param
* Cleanup sandbox code
* Allow wasmi to access wasmer memory regions
* More cleanup
* Remove debug logging, replace asserts with runtime errors
* Revert "Adds SandboxExecutionMethod enum for cli param"
This reverts commit dcb2b1d3b54145ab51ad2e3fef0d980ba215b596.
* Fixes warnings
* Fixes indentation and line width
* Fix return types condition
* Puts everything related under the `wasmer-sandbox` feature flag
* Fixes warnings
* Address grumbles
* Split instantiate per backend
* More splits
* Refacmemory allocation
* Nitpicks
* Attempt to wrap wasmer memory in protoco enforcing type
* Revert renaming
* WIP wasm buffer proxy API
* Reimplement util::wasmer::MemoryRef to use buffers instead of memory slices
* Adds WasmiMemoryWrapper and MemoryTransfer trait
* Refactor naming
* Perform all memory transfers using MemoryTransfer
* Adds allocating `read`
* Adds comments
* Removes unused imports
* Removes now unused function
* Pulls Cargo.lock from origin/master
* Fix rustdoc
* Removes unused `TransferError`
* Update Cargo.lock
* Removes unused import
* cargo fmt
* Fix feature dependency graph
* Feature should flow from the top level crate
* We should not assume a specific workspace structure
* sc-executor-wasmi does not use the feature
* sc-executor-wasmtime should not know about the feature
* Fix doc typo
* Enable wasmer-sandbox by default (for now)
It will be removed before merge. It is so that the benchbot
uses the wasmer sandbox.
* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Revert "cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_contracts --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/contracts/src/weights.rs --template=./.maintain/frame-weight-template.hbs"
This reverts commit d713590ba45387c4204b2ad97c8bd6f6ebabda4e.
* cargo fmt
* Add ci-check to prevent wasmer sandbox build breaking
* Run tests with wasmer-sandbox enabled
* Revert "Run tests with wasmer-sandbox enabled"
This reverts commit cff63156a162f9ffdab23e7cb94a30f44e320f8a.
Co-authored-by: Sergei Shulepov <s.pepyakin@gmail.com>
Co-authored-by: Andrew Jones <ascjones@gmail.com>
Co-authored-by: Alexander Theißen <alex.theissen@me.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
* Split native executor stuff from wasm executor stuff
* Remove `native_runtime_version` in places
* Fix warning
* Fix test warning
* Remove redundant NativeRuntimeInfo trait
* Add a warning for use_native
* Run cargo fmt
* Revert "Add a warning for use_native"
This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.
* Make choosing an executor (native/wasm) an explicit part of service construction
* Add Cargo.lock
* Rename Executor to ExecutorDispatch
* Update bin/node/executor/src/lib.rs
Co-authored-by: Squirrel <gilescope@gmail.com>
* Fix tests
* Fix minor node-executor error
* Fix node cli command thing
Co-authored-by: Squirrel <gilescope@gmail.com>
* Comment out browser stuff
* Remove browser stuff
* Remove more wasm transport code
* Remove ExtTransport and rework how telemetry initialises.
* Change (most) wasm-timer using code to use std::time
* Rename CI-job
* Aura does not compile for wasm
* Remove testing in the browser on CI
* Update README
* Leave `StreamSink` be
* fmt
* RUSTSEC-2021-0076 bump libsecp256k1
libsecp256k1 allows overflowing signatures
https://rustsec.org/advisories/RUSTSEC-2021-0076
Changes were made to conform to libsecp256k1 version differences.
Closes#9356
* parse_standard_slice() -> parse_overflowing_slice()
* Added v2 host function for ecdsa_verify
* Add feature tag over helpers
* Added ecdsa_verify v2 to test runner
* PR feedback
- Spaces -> tabs
- renamed two helper functions
* Fixed imports after rebasing
* Bump rest of libsecp256k1 (and libp2p)
libp2p also uses libsecp256k1 so it is required to be bumped too, along
with all the version difference changes.
* Add version2 for ecdsa pubkey recovery
* libp2p rebase master fixes
* Fix test panic when non Behaviour event is returned
* Update bin/node/browser-testing/Cargo.toml
* Update primitives/core/src/ecdsa.rs
* Update primitives/core/src/ecdsa.rs
* Update Cargo.lock
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.
Updated code related to deprecated warning on tracing-subscriber `scope()`
( See https://github.com/tokio-rs/tracing/issues/1429 )
* Partially applied
* Everything builds, need to implement compact encoding as well.
* Fix some tests, add a ui test as well.
* Fix everything and everything.
* small nits
* a bunch more rename
* more reorg
* more reorg
* last nit of self-review
* Seemingly fixed the build now
* Fix build
* make it work again
* Update primitives/npos-elections/solution-type/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update primitives/npos-elections/solution-type/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* nits
* factor out double type
* fix try-build
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* implement "auto" database backend in client/db, in progress, #9201
* move fn supports_ref_counting from DatabaseSource enum to Database trait to make it work correctly for all types of dbs
* update kvdb_rocksdb to 0.13 and use it's new config feature to properly auto start existing database
* tests for auto database reopening
* introduce OpenDbError to cleanup opening database error handling and handle case when database is not enabled at the compile time
* cargo fmt strings again
* cargo fmt strings again
* rename DataSettingsSrc to fix test compilation
* fix the call to the new kvdb-rocksdb interdace in tests to fix compilation
* simplify OpenDbError and make it compile even when paritydb and rocksdb are disabled
* cargo fmt
* fix compilation without flag with-parity-db
* fix unused var compilation warning
* support different paths for rocksdb and paritydb in DatabaseSouce::Auto
* support "auto" database option in substrate cli
* enable Lz4 compression for some of the parity-db colums as per review suggestion
* applied review suggestions
* Transaction storage guide and fixes
* Apply suggestions from code review
Co-authored-by: Bruno Škvorc <bruno@skvorc.me>
* Update frame/transaction-storage/README.md
Co-authored-by: Bruno Škvorc <bruno@skvorc.me>
* Extended example
Co-authored-by: Bruno Škvorc <bruno@skvorc.me>
* Decouples light-sync state from chain spec
This decouples the light-sync state from chain spec. First, the
light-sync state currently only works with BABE+Grandpa, so not
all *Substrate* based chains can use this feature. The next problem was
also that this pulled the `sc-consensus-babe` and `sc-finality-grandpa`
crate into `sc-chain-spec`.
If a chain now wants to support the light-sync state, it needs to add
the `LightSyncStateExtension` to the chain spec as an extension. This is
documented in the crate level docs of `sc-sync-state-rpc`. If this
extension is not available, `SyncStateRpc` fails at initialization.
* Fix compilation for browser
* Fmt
* Emit error when Config part is imported but without the std feature
* Add UI test for missing std feature on GenesisConfig
* Update frame/support/test/Cargo.toml
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Remove unused imports
* Unify all dummy party checker macros
* Fix
* Dispaly pallet_path::GenesisConfig instead of PalletConfig in error message
* Revert changes to construct_runtime_ui.rs
* Add additional parameter for dummy part checker macro
* Apply suggestions from code review
* fix master merge: update version
* update Cargo.lock
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Kian Paimani