* Use tokio runtime handle instead of TaskExecutor abstraction
Before this pr we had the `TaskExecutor` abstraction which theoretically
allowed that any futures executor could have been used. However, this
was never tested and is currently not really required. Anyone running a
node currently only used tokio and nothing else (because this was hard
coded in CLI). So, this pr removes the `TaskExecutor` abstraction and
relies directly on the tokio runtime handle.
Besides this changes, this pr also makes sure that the http and ws rpc
server use the same tokio runtime. This fixes a panic that occurred when
you drop the rpc servers inside an async function (tokio doesn't like
that a tokio runtime is dropped in the async context of another tokio
runtime).
As we don't use any custom runtime in the http rpc server anymore, this
pr also removes the `rpc-http-threads` cli argument. If external parties
complain that there aren't enough threads for the rpc server, we could
bring support for increasing the thread count of the tokio runtime.
* FMT
* Fix try runtime
* Fix integration tests and some other optimizations
* Remove warnings
* build the template, hand it over to zeke now.
* Tests working
* save wip
* Some updates
* Some cleanup
* mo cleanin
* Link to issue
* Apply suggestions from code review
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Bound accuracy for prepare_election_result
* Use npos_election::Error for phragmms
* save
* Apply suggestions from code review
* Simplify test to use Balancing::set
* Cargo.lock after build
* Revert "Cargo.lock after build"
This reverts commit 7d726c8efa687c09e4f377196b106eb9e9760487.
* Try reduce cargo.lock diff
* Update bin/node/runtime/src/lib.rs
* Comment
* Apply suggestions from code review
* Set balancing directly
* Document som pub items
* Update frame/election-provider-multi-phase/src/unsigned.rs
* Apply suggestions from code review
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Improve some comments
* Revert accidental change to random file
* tiney
* revert
Co-authored-by: kianenigma <kian@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* add query types to generate_storage_alias
* adjust comment
* use ValueQuery explicitly for generate_storage_alias with generic value type
* bump impl_version
* adjust line width and add import
* more compilation and formatting fixes
* formatting
* Store the database in a role specific subdirectory
This is a cleaned up version of #8658 fixing #6880
polkadot companion: paritytech/polkadot#2923
* Disable prometheus in tests
* Also change p2p port
* Fix migration logic
* Use different identification file for rocks and parity db
Add tests for paritydb migration
* doc: convert the adoc to md
* add ref to UncheckedExtrinsic
* Add references to the SS58 format
* Add details about the inspect command
* removing command that is no longer available
* reorder display so ss58 representations show up next to each other
* remove deprecated section
* Add doc about the password option
* fix fmt
* minor fixes
fix#6613
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Squirrel <gilescope@gmail.com>
Co-authored-by: David <dvdplm@gmail.com>
* Migrate Aura pallet to BoundedVec
Implementing issue #8629
* Fixed aura tests after BoundedVec change
* Moved Vec to BoundedVec in authority-discovery
* Merging into the main branch
* Added MaxEncodedLen to crypto
Need this without full_crypto to be able to add generate_store_info
* Add generate_store_info for aura
* Adding changes to Slot to add MaxEncodedLen
* Adding generate_store_info to authority discovery
* fmt
* removing panics in runtime if vec size too large
* authority-discovery: Remove panics in runtime
Can happen if vec size is too large, so truncate the vec in that case
* Adding logging when I truncate Vecs
* Got the sign the other way around
* Reverting pallet_aura changes
This is already being addressed by PR #9371
* Change BoundedVec to WeakBoundedVec
More robust implementation following @thiolliere recommendation.
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Support multiple, mergable vesting schedules
* Update node runtime
* Remove some TODO design questions and put them as commennts
* Update frame/vesting/src/benchmarking.rs
* Syntax and comment clean up
* Create filter enum for removing schedules
* Dry vesting calls with do_vest
* Improve old benchmarks to account for max schedules
* Update WeightInfo trait and make dummy fns
* Add merge_schedule weights
* Explicitly test multiple vesting scheudles
* Make new vesting tests more more clear
* Apply suggestions from code review
* Update remove_vesting_schedule to error with no index
* Try reduce spacing diff
* Apply suggestions from code review
* Use get on vesting for bounds check; check origin first
* No filter tuple; various simplifications
* unwrap or default when getting user schedules
* spaces be gone
* ReadMe fixes
* Update frame/vesting/src/lib.rs
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* address some comments for docs
* merge sched docs
* Apply suggestions from code review
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* log error when trying to push to vesting vec
* use let Some, not is_some
* remove_vesting_schedule u32, not optin
* new not try_new, create validate builder; VestingInfo
* Merge prep: break out tests and mock
* Add files forgot to include in merge
* revert some accidental changes to merged files
* Revert remaining accidental file changes
* More revert of accidental file change
* Try to reduce diff on tests
* namespace Vesting; check key when key should not exist;
* ending_block throws error on per_block of 0
* Try improve merge vesting info comment
* Update frame/vesting/src/lib.rs
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* add validate + correct; handle duration > blocknumber
* Move vesting_info module to its own file
* Seperate Vesting/locks updates from writing
* Add can_add_vesting schedule
* Adjust min vested transfer to be greater than all ED
* Initial integrity test impl
* merge_finished_and_yet_to_be_started_schedules
* Make sure to assert storage items are cleaned up
* Migration initial impl (not tested)
* Correct try-runtime hooks
* Apply suggestions from code review
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* header
* WIP: improve benchmarks
* Benchmarking working
* benchmarking: step over max schedules
* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* Simplify APIs by accepting vec; convert to bounded on write
* Test: build_genesis_has_storage_version_v1
* Test more error cases
* Hack to get polkadot weights to work; should revert later
* Improve benchmarking; works on polkadot
* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* WIP override storage
* Set storage not working example
* Remove unused tests
* VestingInfo: make public, derive MaxEndcodedLen
* Rename ending_block to ending_block_as_balance
* Superificial improvements
* Check for end block infinite, not just duration
* More superficial update
* Update tests
* Test vest with multi schedule
* Don't use half max balance in benchmarks
* Use debug_assert when locked is unexpected 0
* Implement exec_action
* Simplify per_block calc in vesting_info
* VestingInfo.validate in add_vesting_schedule & can_add_vesting_schedule
* Simplify post migrate check
* Remove merge event
* Minor benchmarking updates
* Remove VestingInfo.correct
* per_block accesor max with 1
* Improve comment
* Remoe debug
* Fix add schedule comment
* Apply suggestions from code review
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
* no ref for should_remove param
* Remove unused vestingaction derive
* Asserts to show balance unlock in merge benchmark
* Remove unused imports
* trivial
* Fix benchmark asserts to handle non-multiple of 20 locked
* Add generate_storage_info
* migration :facepalm
* Remove per_block 0 logic
* Update frame/vesting/src/lib.rs
* Do not check for ending later than greatest block
* Apply suggestions from code review
* Benchmarks: simplify vesting schedule creation
* Add log back for migration
* Add note in ext docs explaining that all schedules will vest
* Make integrity test work
* Improve integrity test
* Remove unnescary type param from VestingInfo::new
* Remove unnescary resut for ending_block_as_balance
* Remove T param from ending_block_as_balance
* Reduce visibility of raw_per_block
* Remove unused type param for validate
* update old comment
* Make log a dep; log warn in migrate
* VestingInfo.validate returns Err(()), no T type param
* Try improve report_schedule_updates
* is_valid, not validate
* revert node runtime reorg;
* change schedule validity check to just warning
* Simplify merge_vesting_info return type
* Apply suggestions from code review
* Apply suggestions from code review
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Add warning for migration
* Fix indentation
* Delete duplicate warnings
* Reduce diff in node runtime
* Fix benchmark build
* Upgrade cargo.toml to use 4.0.0-dev
* Cleanup
* MaxVestingSchedulesGetter initial impl
* MinVestedTransfer getter inintial impl
* Test MaxVestingSchedules & MinVestedTransfer getters; use getters in benchmarks
* Run cargo fmt
* Revert MinVestedTransfer & MaxVestingSchedules getters; Add integrity test
* Make MAX_VESTING_SCHEDULES a const
* fmt
* WIP: benchmark improvements
* Finish benchmark update
* Add test for transfer to account with less than ed
* Rm min_new_account_transfer; move sp-io to dev-dep
* Reduce cargo.toml diff
* Explain MAX_VESTING_SCHEDULES choice
* Fix after merge
* Try fix CI complaints
* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* cargo run --quiet --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_vesting --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/vesting/src/weights.rs --template=./.maintain/frame-weight-template.hbs
* fmt
* trigger
* fmt
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: kianenigma <kian@parity.io>
* Split native executor stuff from wasm executor stuff
* Remove `native_runtime_version` in places
* Fix warning
* Fix test warning
* Remove redundant NativeRuntimeInfo trait
* Add a warning for use_native
* Run cargo fmt
* Revert "Add a warning for use_native"
This reverts commit 9494f765a06037e991dd60524f2ed1b14649bfd6.
* Make choosing an executor (native/wasm) an explicit part of service construction
* Add Cargo.lock
* Rename Executor to ExecutorDispatch
* Update bin/node/executor/src/lib.rs
Co-authored-by: Squirrel <gilescope@gmail.com>
* Fix tests
* Fix minor node-executor error
* Fix node cli command thing
Co-authored-by: Squirrel <gilescope@gmail.com>
* Comment out browser stuff
* Remove browser stuff
* Remove more wasm transport code
* Remove ExtTransport and rework how telemetry initialises.
* Change (most) wasm-timer using code to use std::time
* Rename CI-job
* Aura does not compile for wasm
* Remove testing in the browser on CI
* Update README
* Leave `StreamSink` be
* fmt
* RUSTSEC-2021-0076 bump libsecp256k1
libsecp256k1 allows overflowing signatures
https://rustsec.org/advisories/RUSTSEC-2021-0076
Changes were made to conform to libsecp256k1 version differences.
Closes#9356
* parse_standard_slice() -> parse_overflowing_slice()
* Added v2 host function for ecdsa_verify
* Add feature tag over helpers
* Added ecdsa_verify v2 to test runner
* PR feedback
- Spaces -> tabs
- renamed two helper functions
* Fixed imports after rebasing
* Bump rest of libsecp256k1 (and libp2p)
libp2p also uses libsecp256k1 so it is required to be bumped too, along
with all the version difference changes.
* Add version2 for ecdsa pubkey recovery
* libp2p rebase master fixes
* Fix test panic when non Behaviour event is returned
* Update bin/node/browser-testing/Cargo.toml
* Update primitives/core/src/ecdsa.rs
* Update primitives/core/src/ecdsa.rs
* Update Cargo.lock
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Remove unneeded dependencies and dev-dependencies.
Made self_destruct test not dependent on wasm bin size.
Updated code related to deprecated warning on tracing-subscriber `scope()`
( See https://github.com/tokio-rs/tracing/issues/1429 )
* Partially applied
* Everything builds, need to implement compact encoding as well.
* Fix some tests, add a ui test as well.
* Fix everything and everything.
* small nits
* a bunch more rename
* more reorg
* more reorg
* last nit of self-review
* Seemingly fixed the build now
* Fix build
* make it work again
* Update primitives/npos-elections/solution-type/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* Update primitives/npos-elections/solution-type/src/lib.rs
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* nits
* factor out double type
* fix try-build
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* implement "auto" database backend in client/db, in progress, #9201
* move fn supports_ref_counting from DatabaseSource enum to Database trait to make it work correctly for all types of dbs
* update kvdb_rocksdb to 0.13 and use it's new config feature to properly auto start existing database
* tests for auto database reopening
* introduce OpenDbError to cleanup opening database error handling and handle case when database is not enabled at the compile time
* cargo fmt strings again
* cargo fmt strings again
* rename DataSettingsSrc to fix test compilation
* fix the call to the new kvdb-rocksdb interdace in tests to fix compilation
* simplify OpenDbError and make it compile even when paritydb and rocksdb are disabled
* cargo fmt
* fix compilation without flag with-parity-db
* fix unused var compilation warning
* support different paths for rocksdb and paritydb in DatabaseSouce::Auto
* support "auto" database option in substrate cli
* enable Lz4 compression for some of the parity-db colums as per review suggestion
* applied review suggestions
* Transaction storage guide and fixes
* Apply suggestions from code review
Co-authored-by: Bruno Škvorc <bruno@skvorc.me>
* Update frame/transaction-storage/README.md
Co-authored-by: Bruno Škvorc <bruno@skvorc.me>
* Extended example
Co-authored-by: Bruno Škvorc <bruno@skvorc.me>
* Decouples light-sync state from chain spec
This decouples the light-sync state from chain spec. First, the
light-sync state currently only works with BABE+Grandpa, so not
all *Substrate* based chains can use this feature. The next problem was
also that this pulled the `sc-consensus-babe` and `sc-finality-grandpa`
crate into `sc-chain-spec`.
If a chain now wants to support the light-sync state, it needs to add
the `LightSyncStateExtension` to the chain spec as an extension. This is
documented in the crate level docs of `sc-sync-state-rpc`. If this
extension is not available, `SyncStateRpc` fails at initialization.
* Fix compilation for browser
* Fmt
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* mark template and utils as non-publish
* switch to development version for testing
* activate unleash check
* maybe if I disable all rules...
* Fix isolated compilation of `max-encoded-len-derive` with `syn`
error[E0369]: binary operation `==` cannot be applied to type `syn::Path`
--> src/lib.rs:88:29
|
88 | .filter(|attr| attr.path == parse_quote!(max_encoded_len_crate))
| --------- ^^ ----------------------------------- _
| |
| syn::Path
error: aborting due to previous error
For more information about this error, try `rustc --explain E0369`.
Error: could not compile `max-encoded-len-derive`
* WIP: bump changes crates since v3 tag to next breaking
cargo unleash version bump-breaking --changed-since v3.0.0
cargo unleash version set-pre dev --changed-since v3.0.0
FIXME: Don't modify crates that are not yet released, e.g.
`max-encoded-len-derive`
* Update lockfile
* WIP: Bump sp-transaction-pool as well
* WIP: Bump sp-offchain as well
* WIP: Bump frame-system-rpc-runtime-api as well
* WIP: Bump sp-authority-discovery as well
* Manually deactivate dev-deps before `cargo unleash check`
Otherwise we run into `Cycle detected` error.
* Bump sp-consensus-slots
* Add missing Cargo.lock change
* Bump sp-consensus-vrf as well
* Bump sp-keyring as well
* Bump sp-consensus-pow as well
* Try to speed up the `unleash-check` job
Previously, the job took 106 minutes - let's see if explicitly
specifying a `CARGO_TARGET_DIR` will help
* fixup: Ensure the temp target dir exists for unleash check
* Bump pallet-transaction-payment-rpc-runtime-api as well
Needed for Polkadot
* Bump pallet-transaction-payment-rpc as well
Needed for Polkadot
* Try updating crates after patching in the Polkadot CI job
* Use another approach to update patched Substrate crates
* Try to update all sp-core versions in Polkadot CI job
* Simplify sp-core version checking
* Apply another shellcheck lint
* Just do the simplest thing I guess
* Welp don't do --offline then
* Clean up `unleash-check` job triggers
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* Fix a note in unleash-check cache step
* Add a note about temporary optimization in cargo-unleash
* Pin a newer version of cargo-unleash
Co-authored-by: Igor Matuszewski <xanewok@gmail.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
* Update impl-codec to use new upstream MaxEncodedLen trait
* Adapt crates to use the updated codec crate for `MaxEncodedLen`
* Remove max-encoded-len crate altogether
* Fix test compilation in `pallet-proxy`
* reorganize import (#9186)
* Fix remaining `MaxEncodedLen` imports
* Fix remaining old usages of max-encoded-len crate
* Fix UI test
* Manually depend on new impl-codec to fix Polkadot companion build
* Use newly released primitive-types v0.9.1 that has new codec impls
* Make sure codec deps are up-to-date in crates that use them
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
* bump a bunch of deps in parity-common
* primitive-types 0.10.0
* update Cargo.lock
* downgrade a few more
* this is unlikely to help
* try something
* Checkmate, Atheists!
* Do not call `initialize_block` before any runtime api
Before this change we always called `initialize_block` before calling
into the runtime. There was already support with `skip_initialize` to skip
the initialization. Almost no runtime_api requires that
`initialize_block` is called before. Actually this only leads to higher
execution times most of the time, because all runtime modules are
initialized and this is especially expensive when the block contained a
runtime upgrade.
TLDR: Do not call `initialize_block` before calling a runtime api.
* Change `validate_transaction` interface
* Fix rpc test
* Fixes and comments
* Some docs
Andrew Jones