Before we only updated the chain info of sync when we have imported
something using the import queue. However, if you import your own
blocks, this is not done using the import queue and so sync is not
updated. If we don't do this, it can lead to sync switching to "major
sync" mode because sync is not informed about new blocks. This
especially happens on Cumulus, where a collator is selected multiple
times to include its block into the relay chain and thus, sync switches
to major sync mode while the node is still building blocks.
This implements tracking of the last seen blocks in informant display
to prevent printing the import message twice. In Cumulus we first import
blocks as part of the block building with `new_best == false` and set
the best block after we know which one was included by the relay chain.
This leads to printing the import messages two times. This pr solves the
problem by track the latest seen blocks to not print the message twice.
* Print bad mandatory error
This prints the error that leads to bad mandatory.
* Update frame/system/src/lib.rs
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* Adds missing trait import
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* `decl_module!` print better error on duplicate reserved keyword
This prints a better error message on duplicated reserved keywords,
instead of complaining because of missing `origin`.
* Review feedback
This pr fixes a bug with the transaction pool not sending certain events
like finalized and also fixes the order of events. The problem with the
finalized event was that we did not extracted pruned extrinsics if there
were not ready transactions in the pool. However this is wrong, if we
have a re-org, a tx is clearly not ready anymore and we still need to
send a pruned event for it because it is in a new block included. This
also lead to sending "ready" events and tx being re-validated. The
listener also only send the "finalized" event if it has seen a block as
being included, which did not happen before with the old code.
The second fix of the pr is the order of events. If we prune and retract the
same transaction in the same block, we first need to send the "retract"
event and after that the "pruned" event, because finalization takes
longer and this would lead to the UI showing "retract" while it actually
is included.
* Adds support for storage parameter types
This pr adds a new parameter types type, the storage parameter types.
This parameter type supports loading the value from the storage or
returning the given default value.
* Use twox_128
* Update docs
* Update frame/support/src/lib.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
The `tree_route` generated by the import notification is only from the
old best block to the new best parent. This means, it does not contain
the new best block in `enacted()`. We need to prune the transactions of
the new best block "manually" to fix this bug.
Besides that, this pr also changed the `id` parameter of the `NewBlock`
chain event to `hash`. The hash of a block is unique in contrast to the
block number. (Block id can either be number or hash)
* Make tx pool aware of retracted fork blocks
* Make it compile
* Update client/transaction-pool/src/lib.rs
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Fix doc test
* Simplify the implementation
* Send tree route as arc to prevent heavy clones
* Switch to use `ExtrinsicHash` to make it more clear
* Fix benchmark
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Instead of having the tx pool and offchain worker being feed from the
same import notification stream, this pr splits them to use two
different streams. The first advantage of this split is that the tx pool
will not be spawned anymore in another task and instead will directly
process the notification in the same task. This has the advantage of
being faster when the system is being under load, as the tx pool will
not be waiting for being scheduled to handle the notification.
* Make `Proposer` consume its reference on `propose`
A proposer must be created per new round, so it makes sense to have the
proposer consume its own reference.
* Remove `ProposerInner`
* Enable the `runtime-wasm` for wasm builds
This enables the `runtime-wasm` feature for wasm builds. The feature is
not mandatory and will only be activated if it exists in the
`Cargo.toml`.
* Fix compilation
* Update docs
* Uprgade version
* Apply suggestions from code review
If we are running `--dev` chain, we should forbid the `--chain`
argument. The `--dev` chain is always special by only having one
authority etc and some other chain spec is probably not setup for this
correctly. In the end `--dev` is just a shortcut for `--validator --alice`.
There was a bug which related in required files not being tracked of
being modified. This pr fixes this bug by making sure we ignore version
requirements for path dependencies and git dependencies. This also
ensures that we only track `.rs` or `.toml` files. Another improvement
is that we only include paths which don't contain a `Cargo.toml` if this
`Cargo.toml` does not belongs to the package being processed. This
prevents that sub-crates are added to the tracked files, while not being
part of the dependencies.
* Don't clone values when calculating storage root
Instead of cloning all the keys and values of the overlay when
calculating the storage root, we pass all the values by reference. This
should probably bring some performance improvements when calculating the
storage root.
* no cow version (#6113)
Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Instead of resetting `SIGPIPE` to the system default handler, we now
handle a clap error manually and ignore any error when writing to
`stdout`/`stderr`. In this way, the node does not silently stops when it
encounters a `SIGPIPE` in normal mode and we still support piping of the
cli output.
Instead of informing the telemetry about each block that is finalized,
we only need to send the last finalized block. This removes log spam on
initial sync.
* Optimize `decode_len`
Instead of reading the full storage value into the runtime, we only read
at maximum `5bytes` from the storage into the runtime. Furthermore this
drops any handling with regards to set default values in
`decl_storage!`. If the value does not exists or the decoding of the
length fails, it will return `None`. To prevent people from messing
stuff up, this feature relies on the `StorageDecodeLength` trait that is
sealed by `frame-support` (aka only implementable inside this crate).
* Some clean ups
* Update frame/support/src/storage/mod.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* Make `Digest` support `StorageAppend`
This adds support for `StorageAppend` to `Digest`. Digest is just a
wrapper around a `Vec` and we abuse the fact that SCALE does not puts
any special marker into the encoding for structs. So, we can just append
to the encoded Digest. A test is added that ensures, if the `Digest`
format ever changes, we remove this optimization.
* Update weight
* Update frame/support/src/storage/mod.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* Update frame/system/src/lib.rs
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
* Expose that `BasicQueue` expects blocking spawn
Up to now `BasicQueue` expected a closure that to spawn a `Future`.
This was expected to be a closure that spawns a blocking future.
However, this wasn't documented anywhere. This pr introduces a new trait
`SpawnBlocking` that exposes this requirement to the outside.
* Feedback
* Require `fn` token in `decl_storage` `get`
The `fn` token was already for quite some time as an optional parameter.
It was introduced to make these functions better findable. This pr makes
the `fn` token required.
* Remove `GetterNoFnkeyword`
Instead of just listening on ipv4 by default, this enables listening on
ipv6 as well. While doing this, it also fixes a bug which lead to always
listen on all sockets, even if we specify listen addresses explicitly.
* `decl_runtime_apis!` - check that a method without `changed_in` exists
This adds another check to the macro that ensures that not only methods
with `changed_in` exists, but there are also the default methods exist.
* Update primitives/api/proc-macro/src/decl_runtime_apis.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Fix test
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Add new RPC method to get the chain type
This adds a new RPC method to get the chain type of the running chain.
The chain type needs to be specified in the chain spec. This should make
it easier for tools/UI to display extra information without needing to
rely on parsing the chain name.
* Update client/rpc-api/src/system/mod.rs
Co-Authored-By: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Primitive crate
* Feedback
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
There was a bug in determining the correct git files that should be
tracked. The ref extracted from `.git/HEAD` pointed to a relative path
and thus could not be found by cargo and resulted in always recompiling
the binary.
* Make `application-crypto` `std` feature internal
The macros should not generate code that requires that the calling crate
has a feature with the name `std` defined.
* Use a proper panic!
This pr ensures that we don't panic because of a deadline that is
already reached in basic authorship. This is mainly useful for debug
builds, as release builds normally don't have debug assertions enabled.
* Implements mocking of runtime apis
This pr adds support for easily mock runtime api implementations for
tests by using the `mock_impl_runtime_apis!` macro. The syntax is
similar to `impl_runtime_apis!`. The mocked implementation automatically
implements `ApiExt`, `ApiErrorExt` and `Core` as these are required by
the runtime api traits, but not required in tests or only a subset of them.
* Fix warnings
* Update primitives/api/proc-macro/src/utils.rs
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
* Review feedback
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
The signal handler will be reset to the default system one. This means
the program will be exited instead of panicking. This is required to
support piping in the console.
* Enable `wasmtime` only for `x86_64` in substrate node
When using the new feature of cargo `cargo build -Z features=itarget`,
the `wasmtime` feature will not bleed into the build for other
architectures.
* Use new cargo feature in CI
Currently when we connect to a bootnode and it returns a different peer
id, than the one we provided, we disconnect silently. This pr changes
the behavior to printing an error when a bootnode returns a different
peer id than the provided one.
* Adds new event phase `Initialization`
Every event that was deposited inside of `on_initialize` was assigned to
the `ApplyExtrinsic(0)` phase which wasn't correct. This pr introduces a
new phase `Initialization`. This is the new phase while initializing
a block. After initialization we switch to `ApplyExtrinsic(N)` and at
the end to `Finalization` as before.
* Set `ExecutionPhase` in `initialize`
* Increment `spec_version`
* Make sure frame examples compile for wasm
This makes sure that `frame-example` and `frame-example-offchain-worker`
compile for wasm.
This also fixes compilation for these crates. The offchain worker
example doesn't use serde-json anymore as that is too heavy and breaks
`no_std` compilation.
* Apply suggestions from code review
Co-Authored-By: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
* Move runtime upgrade to `frame-executive`
Instead of storing the runtime upgraded in the space of `frame-system`,
this moves it to `frame-executive`. We also start storing the
`spec_version` and `impl_version` the last runtime upgrade was applied.
This scheme has multiple advantages:
- We don't need to make sure that runtime upgrade was set.
- This will work in the future when we want to test a runtime upgrade,
because the `on_runtime_upgrade` will be executed automatically when the
runtime changes.
* Move `LAST_RUNTIME_UPGRADE` key to `frame-executive`
* Add some more documentation
* Make sure `spec_version` always increases between runtime upgrades
* Fix test
* Upgrade `parity-multiaddr` to fix compilation
* Also check if the `spec_name` changed
* Remove `RuntimeUpgraded` storage entry
* Don't run on genesis
Up to now `wasm-builder` iterated over all packages that were not part
of the wasm workspace and added the files to the `rerun-if-changed`
list. However, this included to many files and resulted in needless
re-compilations in `Cumulus` for example.
* Don't include `:code` by default in storage proofs (#5060)
* Adds test to verify that the runtime currently is always contained in
the proof
* Start passing the runtime wasm code from the outside
* Fix compilation
* More build fixes
* Make the test work as expected now :)
* Last fixes
* Fixes benchmarks
* Review feedback
* Apply suggestions from code review
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* Review feedback
* Fix compilation
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
* Fix compilation and change the way `RuntimeCode` works
* Fix tests
* Switch to `Cow`
Co-authored-by: Benjamin Kampmann <ben@gnunicorn.org>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.
* Adds documentation for `wipe` and `commit`
This adds documentation to `wipe` and `commit` of `Externalities`.
Besides that it removes the default implementation that would just panic
and requires that all implementers of the trait implement the functions.
* Update primitives/externalities/src/lib.rs
Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.
This prs cleans up some of the frame benchmarking stuff:
- Move CLI into `frame-benchmarking-cli`. No frame related CLI should
exists in the default Substrate CLI.
- Move all traits and types related to frame benchmarking into the
`frame-benchmarking` trait. Frame types should be isolated in Frame.
This removes the following syntactic sugar from `construct_runtime!`:
- Expansion of `default` to the default set of module parts
- Expansion of `System: system` to the default set of module parts
The macro now requires the user to provide all the module parts of a pallet.
* Adds `with_pair!` macro to application-crypto
This macro will "generate" the given code only when the crypto pair is
available. So, when either the `std` or the `full_crypto` feature is
enabled.
* Fix example
* Adds a test to ensure that we clear the heap between calls into runtime
The tests shows that we currently not clearing the heap in wasmtime.
For now we don't run the test for wasmtime.
* Fix compilation
* Fix CLI setup again
We need to set `config_dir` and `database_path` for almost every
command.
This fixes `purge-chain` and also adds a test to make sure we don't
break it again.
* Adds missing test files
* Split methods
* FreeingBumpAllocator: Initialize the heads to `u32::max_value()`
`self.heads` can point to an element with the index `0` in the heap.
This would make the allocator fail to reuse this element.
* Simplify the `PREFIX_SIZE` handling
* Fix memory leak in runtime interface
We used `slice::from_raw_parts` in runtime-interface which did not free
the memory afterwards. This pr changes it to `Vec::from_raw_parts` to
make sure `drop` is called properly and the values are freed.
* Check that `len` is non-zero
* Adds comment
When messing with the signature, we need to make sure that we acutally
mess-up the signature. As the generated private/public key is random,
the signature is random as well. It can happen that `bytes[0] ==
bytes[2]` which makes the test fail. We fix this problem by just
inverting the bytes at `0` and `2`.
* Expose information about the extrinsic in the metadata
This pr exposes some information about the extrinsic used in the runtime
via metadata. The following information are exposed:
- Version of the extrinsic
- List of all signed extensions used by the extrinsic.
* Increment `spec_version`
Bastian Köcher