Commit Graph
100 Commits
Author SHA1 Message Date
Bastian Köcher 86ab838b17 Keep current block randomness in state (#9294)
* Keep current block randomness in state

Instead of killing it at the end of the block, it stays in the block for
inspection. This is required by parachains to get access to this
randomness of the relay chain.

* Fix tests
2021-07-07 11:44:05 +00:00
Bastian Köcher 7bbf949296 Clean up sc-allocator (#9295) 2021-07-07 13:11:31 +02:00
Bastian Köcher f781dcaf2c Add function for embedding the runtime version in a wasm blob (#9277)
* Add function for embedding the runtime version in a wasm blob

This function can be used to add the custom section to a wasm blob with
the runtime version in it.

* Review nitpick
2021-07-06 10:41:27 +00:00
Bastian Köcher d489bd70b5 Do not call initialize_block before any runtime api (#8953)
* Do not call `initialize_block` before any runtime api

Before this change we always called `initialize_block` before calling
into the runtime. There was already support with `skip_initialize` to skip
the initialization. Almost no runtime_api requires that
`initialize_block` is called before. Actually this only leads to higher
execution times most of the time, because all runtime modules are
initialized and this is especially expensive when the block contained a
runtime upgrade.

TLDR: Do not call `initialize_block` before calling a runtime api.

* Change `validate_transaction` interface

* Fix rpc test

* Fixes and comments

* Some docs
2021-07-01 15:50:42 +00:00
Bastian Köcher eac38de198 Remove RandomNumberGenerator (#9198)
* Remove RandomNumberGenerator

This is not used in Substrate/Polkadot. If someone else needs it, they
can copy the code or use chacha.

* 🤦
2021-06-24 14:19:36 +00:00
Bastian Köcher 1278b9a6d4 Aura: Skip initialize block & remove cache (#9132)
This instructs the Aura runtime api to skip initialize block, when
requesting the authorities. This is important, as we don't want to use
the new authorities that should be used from the next block on.
Besides that, it removes the caching stuff. The cache is not available
on full nodes anyway. In the future we should store the authorities
probably in the aux store.
2021-06-17 07:27:53 +00:00
Bastian Köcher 45f8b6e91f pallet-authorship: Fixing some nitpicks (#9095)
As reviewing the pallet yesterday, I have found some nitpicks that I
fixed.
2021-06-13 11:41:13 +00:00
Bastian Köcher e360cff92e pallet-offences: Switch to partition_point (#9049)
This changes the code to use `partition_point` instead of
`binary_search_by_key`, because this was very likely the problematic
pallet 2 weeks ago on polkadot.
2021-06-12 02:38:17 +02:00
Bastian Köcher 68833498c6 Transaction pool: Remove futures-diagnose and thread pool (#9074)
* Transaction pool: Remove futures-diagnose and thread pool

This pr removes `futures-diagnose` as this isn't used anymore. Besides
that the pr also removes the thread pool that was used to validate the
transactions in the background. Instead of this thread pool we now spawn
two separate long running tasks that we use to validate the
transactions. All tasks of the transaction pool are now also spawned as
essential tasks. This means, if any of these tasks is stopping, the node
will stop as well.

* Update client/transaction-pool/src/api.rs
2021-06-11 17:24:30 +00:00
Bastian Köcher 11037ac35f Try fix ui tests (#9082)
* Try fix ui tests

* More
2021-06-11 15:12:57 +00:00
Bastian Köcher 258c1a86f6 Transaction pool: Ensure that we prune transactions properly (#8963)
* Transaction pool: Ensure that we prune transactions properly

There was a bug in the transaction pool that we didn't pruned
transactions properly because we called `prune_known`, instead of `prune`.

This bug was introduced by:
https://github.com/paritytech/substrate/pull/4629

This is required to have stale extrinsics being removed properly, so
that they don't fill up the tx pool.

* Fix compilation

* Fix benches

* ...
2021-06-03 16:04:29 +02:00
Bastian Köcher 2ab5f9aeca Transactionpool: Make ready_at return earlier (#8995)
`ready_at` returns when we have processed the requested block. However,
on startup we already have processed the best block and there
are no transactions in the pool on startup anyway. So, we can set `updated_at`
to the best block on startup.

Besides that `ready_at` now returns early when there are no ready nor
any future transactions in the pool.
2021-06-02 18:13:47 +00:00
Bastian Köcher ba52404a27 Dependabot use correct label (#8973) 2021-06-01 11:57:35 +00:00
Bastian Köcher fa22eb7361 Optimize next_storage_key (#8956)
* Optimize `next_storage_key`

- Do not rely on recursion
- Use an iterator over the overlay to not always call the same method

* Fix bug
2021-05-31 20:17:15 +02:00
Bastian Köcher 53e28b98e9 Introduce code_substitute (#8898)
This introduces a new field `code_substitute` into the chain spec. This
can be used to substitute the on-chain wasm starting from a given block
until there is another wasm on chain (determined through the
`spec_version`). This can be used to fix broken on chain wasm runtimes.
2021-05-25 14:07:36 +00:00
Bastian Köcher 4dc8f3a7e5 Aura improvements (#8881)
* Aura: Expose function to build the verifier

* Use best block to initialize the authorities cache

* Use best block when determining the slot duration

* Remove `AuraBlockImport`

* Some cleanups

* Fix build error
2021-05-22 21:01:11 +00:00
Bastian Köcher ee726ed55d Improve impl_opaque_keys! (#8856)
The macro should assume less about the scope where it is being used in.
In this case it is about not assuming that the crate where the macro is
called in provides a `std` feature.
2021-05-19 08:37:42 +00:00
Bastian KöcherandKian Paimani 6a6f330412 Make wasmtime the default when the feature is enabled (#8855)
* Make wasmtime the default when the feature is enabled

* Update client/cli/src/arg_enums.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2021-05-19 08:29:32 +00:00
Bastian Köcher 9b4f9c9e42 Remove unneeded trait bounds (#8765)
Before we required these trait bounds because of some bug in rustc, but
now as this bug is fixed they can be removed.
2021-05-09 19:17:42 +00:00
Bastian KöcherandAndré Silva e1caa2979f Fix the calculation of the time until the next slot (#8753)
* Fix the calculation of the time until the next slot

* Update client/consensus/slots/src/slots.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-05-07 00:15:08 +02:00
Bastian Köcher 37ea3958d9 Let the clone wars begin! (#8731)
Sprinkle some `Clone` onto the cli commands.
2021-05-04 21:57:18 +00:00
Bastian Köcher 1f21d77ad1 Set max log level when changing the log directives via RPC (#8721)
Before this pr changing the log directives would not change the max log
level. This means that if the node was started with `info` logging and
some `trace` logging was enabled, this `trace` wouldn't be logged. To
fix this we also need to update the max log level. This max log level is
used by the log macros to early return.
2021-05-04 11:34:52 +00:00
Bastian KöcherandAndré Silva 2675741a09 Rework inherent data client side (#8526)
* Lol

* Yeah

* Moare

* adaasda

* Convert AURA to new pallet macro

* AURA: Switch to `CurrentSlot` instead of `LastTimestamp`

This switches AURA to use `CurrentSlot` instead of `LastTimestamp`.

* Add missing file

* Update frame/aura/src/migrations.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Remove the runtime side provide inherent code

* Use correct weight

* Add TODO

* Remove the Inherent from AURA

* 🤦

* Remove unused stuff

* Update primitives authorship

* Fix babe inherent data provider

* Fix consensus-uncles

* Fix BABE

* Do some further changes to authorship primitives... :D

* More work

* Make it compile the happy path

* Make it async!

* Take hash

* More stuff

* Hacks

* Revert "Hacks"

This reverts commit cfffad88668cfdebf632a59c4fbfada001ef8251.

* Fix

* Make `execute_block` return the final block header

* Move Aura digest stuff

* Make it possible to disable equivocation checking

* Fix fix fix

* Some refactorings

* Comment

* Fixes fixes fixes

* More cleanups

* Some love

* Better love

* Make slot duration being exposed as `Duration` to the outside

* Some slot info love

* Add `build_aura_worker` utility function

* Copy copy copy

* Some stuff

* Start fixing pow

* Fix pow

* Remove some bounds

* More work

* Make grandpa work

* Make slots use `async_trait`

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make grandpa work

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make it async

* Fix fix

* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* Make grandpa tests compile

* More Babe tests work

* Fix network test

* Start fixing service test

* Finish service-test

* Fix sc-consensus-aura

* Fix fix fix

* More fixes

* Make everything compile *yeah*

* Make manual-seal compile

* More fixes

* Start fixing Aura

* Fix Aura tests

* Fix Babe tests

* Make everything compile

* Move code around and switch to async_trait

* Fix Babe

* Docs docs docs

* Move to FRAME

* Fix fix fix

* Make everything compile

* Last cleanups

* Fix integration test

* Change slot usage of the timestamp

* We really need to switch to `impl-trait-for-tuples`

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Some extra logging

* Remove dbg!

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-05-03 14:39:25 +00:00
Bastian Köcher 541692c4a8 Use host max log level when initializing the RuntimeLogger (#8655)
* Use host max log level when initializing the `RuntimeLogger`

This should fix performance problems introduced by logging under certain
circumstances. Before we always called into the host and the host was
doing the log filtering, now as the correct max log level is set, we
don't call into the host for every log line to check if it should be
logged. However, we would still call into the host to determine if
something should be logged when `something=trace` is given as we don't
forward the log targets that are enabled.

* Finish the pr
2021-04-23 13:22:39 +00:00
Bastian Köcherandthiolliere c8136bd1df Remove serde requirement from FRAME macros (#8628)
* Remove `serde` requirement from FRAME macros

Currently there is some implicit requirement on `serde` being present in
the `Cargo.toml` of a pallet when `GenesisConfig` is used. This pr
removes this requirement by using the serde attribute `serde(crate = "..")`.

* build a unique reexport of serde in impl_opaque_keys, by abusing paste doc concatenation

* Optimize

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2021-04-16 12:42:37 +02:00
14b5acab86 Introduce a "dynamic" block size limit for proposing (#8588)
* Introduce a "dynamic" block size limit for proposing

This adds support for using a dynamic block size limit per call to
`propose`. This is required for Cumulus/Parachains to always use stay in
the limits of the maximum allowed PoV size.

As described in the docs, the block limit is only checked in the process
of pushing transactions. As we normally do some other operations in
`on_finalize`, it can happen that the block size still grows when there
is some proof being collected (as we do for parachains). This means,
that the given block limit needs to be rather conservative on the actual
value and should not be the upper limit.

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* More future proof encoded size updating

* Use `ProofRecorderInner`

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/slots/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/slots/src/slots.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-04-14 17:56:22 +00:00
Bastian Köcher 6378a4ee78 Pallet macro support frame_system::Config with args (#8606) 2021-04-12 19:11:23 +00:00
Bastian Köcher 59b900bae3 Fix parameter_types! macro (#8594)
Make it work with different kinds of parameter types when
`static` is one of them.
2021-04-12 19:37:35 +02:00
Bastian Köcher ec5c7762a0 Require FullCodec on frame_system::Call (#8540) 2021-04-06 15:08:04 +02:00
Bastian Köcher 4e334f59d6 Use log::error! for bad mandatory (#8521)
To make the life of people easier ;)
2021-04-04 13:42:11 +02:00
4186274ddd Fixes storage_hash caching issue and enables better caching for Cumulus (#8518)
* Fixes `storage_hash` caching issue and enables better caching for Cumulus

There was a caching issue with `storage_hash` that resulted in not
reverting cached storage hashes when required. In Cumulus this resulted
in nodes failing to import new blocks after a runtime upgrade, because
they were using the old runtime version.

Besides that, this pr optimizes for the Cumulus use case. In particular
that we always import blocks first as non-best blocks and enact them
later. In current version of the caching that would mean we would always
throw away the complete cache of the latest imported block. Now, we
always update the cache for the first block of a new block height. This
enables us to use the cache if this block will enacted as best block
later. If there is a fork and that is enacted as best, we revert all the
changes to the cache.

* Apply suggestions from code review

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>

* Indentation

* Update client/db/src/storage_cache.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-04-02 23:58:30 +02:00
Bastian Köcher dcf29a8523 Make SetCode::set_code return a result (#8515) 2021-04-02 01:09:41 +02:00
Bastian KöcherandAndré Silva 217c4be226 Make BlockImport and Verifier async (#8472)
* Make grandpa work

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make it async

* Fix fix

* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* Make grandpa tests compile

* More Babe tests work

* Fix network test

* Start fixing service test

* Finish service-test

* Fix sc-consensus-aura

* Fix fix fix

* More fixes

* Make everything compile *yeah*

* Fix build when we have Rust 1.51

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix warning

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-03-30 09:19:49 +00:00
Bastian Köcher 2998b42311 Get rid of test-helpers feature in sc-consensus-babe (#8486)
This is not really required and having a special `test-helpers` feature
is a bad idea anyway.
2021-03-30 08:10:54 +02:00
Bastian KöcherandDenis P 22139a3c03 Prepare UI tests for Rust 1.51 & new CI image (#8474)
* Prepare UI tests for Rust 1.51 & new CI image

* another stderr to be fixed

Co-authored-by: Denis P <denis.pisarev@parity.io>
2021-03-29 17:05:06 +02:00
Bastian KöcherandAndré Silva 49e79967c8 Use async_trait in sc-consensus-slots (#8461)
* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* slots: remove mutex around BlockImport in SlotWorker

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-03-27 22:40:28 +01:00
Bastian Köcher 3e64293bd4 Fix companion check (#8464) 2021-03-26 14:59:05 +00:00
Bastian Köcher 6adf24ca0c Change companion check order (#8460)
* Change companion check order

Apparently if the pr isn't approved it counts as not mergable. However,
this is rahter confusing. To fix this, we just change the order.

* Move exit
2021-03-26 14:21:00 +01:00
Bastian KöcherandGuillaume Thiolliere c727799531 Better description for assert in frame-session genesis (#8399)
* Better description for assert in frame-session genesis

There is an assert that checks that an account exists, after setting a
key. However, this assert isn't very self-descriptive.

* Update frame/session/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-03-18 20:24:05 +01:00
Bastian KöcherandPeter Goodspeed-Niklaus f69f79cc20 Improve logging in network gossip (#8389)
* Improve logging in network gossip

This adds some more information to the logging output to get a better
understanding when something fails.

* Update client/network-gossip/src/state_machine.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
2021-03-18 12:16:42 +01:00
Bastian Köcher 15e15e7d8e Aura and Slots refactoring (#8386)
* Make slot duration being exposed as `Duration` to the outside

* Some slot info love

* Add `build_aura_worker` utility function

* Copy copy copy
2021-03-17 23:25:58 +00:00
Bastian Köcher 413acc5ddb Fix CI benchmark check (#8380) 2021-03-16 19:37:58 +01:00
Bastian Köcherandkianenigma 2cc9329b09 Update diener to enable polkadot companions that use new crates (#8371)
* Update diener to enable polkadot companions that use new crates

Diener will be updated in the CI image to make it possible to have
companions that add new crates from Substrate.

Besides that the wasm-bindgen is updated, because `0.2.70` is not
available anymore on crates.io.

* update lock

Co-authored-by: kianenigma <kian@parity.io>
2021-03-16 15:42:56 +01:00
Bastian Köcher 3743cec9bd Do not ban peers for sending multiple valid requests (#8325)
We introduced banning of peers who spam us with the same request (more
than 2 times). However, we missed that it is completely legal to send
the same request multiple times as long as we did not provide any
answer. An example for that is the justification request. This request
is send multiple times until we could fetch the justification from one
of our peers. So, the solution to this problem is to tag requests as
fulfilled and to start counting these fulfilled requests. If the number
is higher than what we allow, the peer should be banned.
2021-03-12 11:13:44 +00:00
Bastian Köcher 5d73e960da Convert timestamp to unit type wrapper (#8333)
The timestamp inherent type was up to now just a simple `u64`. This
worked, but doesn't give you that much guarantees at compile time about
the type. This pr changes that by converting this type to a unit type
wrapper, similar to what we have done for `Slot`.

This is required for some future pr that touches quite a lot of the
inherents stuff :)

Besides this unit wrapper type, this pr also moves the `OnTimestampSet`
trait to `frame_support::traits`.
2021-03-11 23:33:34 +01:00
Bastian Köcher c33235ac5b Update to latest proc-macro-crate version (#8294) 2021-03-09 22:22:25 +00:00
Bastian Köcher 7599e0d6e8 Introduce new concept of "slot portion for proposing" (#8280)
* Introduce new concept of "slot portion for proposing"

Currently when building a block we actually give the proposer all of the
time in the slot, while this is wrong. The slot is actually split in at
least two phases proposing and propagation or in the polkadot case into
three phases validating pov's, proposing and propagation. As we don't
want to bring that much polkadot concepts into Substrate, we only
support splitting the slot into proposing and propagation. The portion
can now be passed as parameter to AuRa and BABE to configure this value.
However, this slot portion for propagation doesn't mean that the
proposer can not go over this limit. When we miss slots we still apply
the lenience factor to increase the proposing time, so that we have
enough time to build a heavy block.

Besides all what was said above, this is especially required for
parachains. Parachains have a much more constraint proposing window.
Currently the slot duration is at minimum 12 seconds, but we only have
around 500ms for proposing. So, this slot portion for proposing is
really required to make it working without hacks.

* Offgit feedback

* Cast cast cast
2021-03-09 19:14:54 +00:00
Bastian Köcher 85f32e8813 Decrease the peer reputation on invalid block requests (#8260)
* Decrease the peer reputation on invalid block requests

This pr changes the block request handler to decrease the reputation of
peers when they send the same request multiple times or they send us an
invalid block request.

* Review feedback

* Change log target

* Remove unused code
2021-03-09 14:43:19 +01:00
Bastian Köcher b0ebf6498f Do not use Option to wrap GenesisConfig fields (#8275)
Currently we wrap every `GenesisConfig` field in an `Option`, while
we require `Default` being implemented for all pallet genesisconfigs.
Passing `None` also results in the genesis not being initialized, which
is a bug as seen from the perspective of a pallet developer?

This pr changes the fields of the `GenesisConfig` to non `Option` types.
2021-03-06 13:42:21 +00:00
Bastian KöcherandAndré Silva 1680422f1f AuRa improvements (#8255)
* AuRa improvements

Hot and fresh AuRa improvements. This pr does the following:

- Move code belonging to the import queue etc to import_queue.rs
- Introduce `ImportQueueParams` and `StartAuraParams` structs to make
it more easier to understand what parameters we pass to AuRa.
- Introduce `CheckForEquivocation` to tell AuRa if it should check for
equivocation on block import. This is required for parachains, because
they are allowed to equivocate when they build two blocks for the same
slot, but for different relay chain parents.

* Update client/consensus/aura/src/import_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix compilation

* AAA

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-03-05 17:03:30 +01:00
Bastian Köcher d0530d0f2a Fix warnings related to panic and assert (#8272)
We were using the wrong syntax and that will be dropped with Rust 2021.
The compiler already starts to hint the wrong syntax with warnings. So,
we fix this here.
2021-03-05 10:56:33 +01:00
Bastian Köcher 274e7f0652 Do not return the Header from execute_block (#8256)
That was actually a bad idea by me, because it should essentially be the
same as block.header.

Ty @kianenigma
2021-03-03 21:59:28 +01:00
Bastian Köcher 4de4662480 Make ExecuteBlock::execute_block return the final block header (#8244)
This pr changes the `ExecuteBlock` trait to return the final header that
results from executing the given block.
2021-03-03 08:44:34 +00:00
Bastian KöcherandAndré Silva dc190a69f2 Move AuRa digest from client to primitives (#8245)
* Move AuRa digest from client to primitives

This makes the digest stuff usable from inside the runtime ;)

* Update primitives/runtime/src/generic/digest.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Review feedback

* Make BABE use the new functionality

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-03-03 09:41:27 +01:00
Bastian KöcherandGuillaume Thiolliere 68390d4085 Init RuntimeLogger automatically for each runtime api call (#8128)
* Init `RuntimeLogger` automatically for each runtime api call

This pr change the runtime api in such a way to always and automatically
enable the `RuntimeLogger`. This enables the user to use `log` or
`tracing` from inside the runtime to create log messages. As logging
introduces some extra code and especially increases the size of the wasm
blob. It is advised to disable all logging completely with
`sp-api/disable-logging` when doing the wasm builds for the on-chain
wasm runtime.

Besides these changes, the pr also brings most of the logging found in
frame to the same format "runtime::*".

* Update frame/im-online/src/lib.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update test-utils/runtime/Cargo.toml

* Fix test

* Don't use tracing in the runtime, as we don't support it :D

* Fixes

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2021-03-01 15:29:17 +01:00
Bastian KöcherandAndré Silva 8a0e8ea9a6 Move proof generation to the type system level (#8185)
* Start

* Finish!!!!

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Review comments

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-02-24 20:43:50 +00:00
Bastian KöcherandAndré Silva 76d3814e90 Make on_slot return the block with the post header (#8188)
* Make `on_slot` return the block with the post header

Before this pr `on_slot` returned the pre block. However this is wrong,
because adding some post digest changes the hash of the header. Thus,
we need to make sure to return the correct block that uses the post
header.

* Update primitives/consensus/common/src/block_import.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-02-24 08:03:05 +01:00
Bastian KöcherandAndré Silva 4f4a0c5b38 Make keystore return None when a key doesn't exist (#8163)
* Make keystore return `None` when a key doesn't exist

* Fixes

* More fixes

* Update comment

* Update primitives/keystore/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/keystore/src/local.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Address comments

* Update client/keystore/src/local.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-02-22 14:24:09 +00:00
Bastian Köcher 821e018d75 Ensure we spawn the block import worker as an essential task (#8155)
* Ensure we spawn the block import worker as an essential task

This pr ensures that we spawn the block import worker as an essential
task. This is quite important as we need to bring down the node when the
block import is done. Besides that it adds some debug output to the
block import worker.

* Don't be stupid :D
2021-02-19 16:31:03 +00:00
Bastian KöcherandDenis P 20527aec0e Fix warning in rustdoc job (#8159)
* Fix warning in rustdoc job

* More fixes

* Remove `build-rust-doc` job

Remove this job until upstream is fixed:
https://github.com/rust-lang/rust/issues/82284

* CI: temp. remove of the publishing job, no use of it w/o build

Co-authored-by: Denis P <denis.pisarev@parity.io>
2021-02-19 15:04:23 +01:00
Bastian Köcher cbe35b9bd4 Remove OnSlot associated type (#8156)
Currently we always use a boxed future everywhere anyway. This also
enables us to use a boxed `SlotWorker` (which is required for Cumulus).
2021-02-19 00:15:26 +01:00
Bastian KöcherandAndré Silva b2021030cc Make it clear in CLI that paritydb is experimental (#8152)
* Make it clear in CLI that paritydb is experimental

Sadly this is a breaking change for the CLI.

* Update client/cli/src/params/database_params.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-02-18 14:58:01 +01:00
Bastian Köcher 33f9becf41 Simplify runtime api error handling (#8114)
* Ahh

* Work work work

* Fix all the compilation errors

* Fix test

* More fixes...
2021-02-15 11:55:40 +00:00
Bastian Köcher fd5fbd6252 Switch to use diener patch for companion build (#8073)
This switch to the new `diener patch` command to patch all Substrate
crates in Polkadot. This should remove the requirement to manually merge
Substrate master to make the companion build, as we now would use the
already with master merged code from this build job local checkout.
2021-02-10 12:38:29 +01:00
Bastian Köcher f174e14793 Switch to latest impl-trait-for-tuples (#8082)
Switches to the latest version everywhere now, as I fixed the problems
in the crate ;)
2021-02-09 13:31:44 +01:00
Bastian KöcherandAndré Silva 5569313bd6 AURA: Switch to CurrentSlot instead of LastTimestamp (#8023)
* Convert AURA to new pallet macro

* AURA: Switch to `CurrentSlot` instead of `LastTimestamp`

This switches AURA to use `CurrentSlot` instead of `LastTimestamp`.

* Add missing file

* Update frame/aura/src/migrations.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Remove the runtime side provide inherent code

* Use correct weight

* Add TODO

* Remove the Inherent from AURA

* 🤦

* Remove unused stuff

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-02-03 13:47:13 +01:00
Bastian Köcher 7d65b5aa59 Convert AURA to new pallet macro (#8020) 2021-02-02 14:19:40 +00:00
Bastian KöcherandDavid 5410486947 Fix tracing tests (#8022)
* Fix tracing tests

The tests were not working properly.

1. Some test was setting a global subscriber, this could lead to racy
conditions with other tests.

2. A logging test called `process::exit` which is completly wrong.

* Update client/tracing/src/lib.rs

Co-authored-by: David <dvdplm@gmail.com>

* Review comments

Co-authored-by: David <dvdplm@gmail.com>
2021-02-02 12:19:35 +01:00
Bastian Köcher c42d756fb7 Fix tracing spans are not being forwarded to spawned task (#8009)
* Fix tracing spans are not being forwarded to spawned task

There is a bug that tracing spans are not forwarded to spawned task. The
problem was that only the telemetry span was forwarded. The solution to
this is to use the tracing provided `in_current_span` to capture the
current active span and pass the telemetry span explictely. We will now
always enter the span when the future is polled. This is essentially the
same strategy as tracing is doing with its `Instrumented`, but now
extended for our use case with having multiple spans active.

* More tests
2021-02-01 14:54:21 +00:00
Bastian Köcher d40f9c166f Sync: Fix issue of not freeing a block announcement slot (#8006)
* Sync: Fix issue of not freeing a block announcement slot

There was a bug that when the block announcement validation returned an
error, the slot reserved for this validation wasn't freed. This could
lead to a situation where we rejected any block announcement from such a
peer for that the block announcement returned an error multiple times.

* Better logging

* Fuck I'm dumb

* 🤦
2021-01-29 16:29:25 +00:00
Bastian KöcherandAndré Silva b6294418f8 Introduce a Slot type (#7997)
* Introduce a `Slot` type

Instead of having some type definition that only was used in half of the
code or directly using `u64`, this adds a new unit type wrapper `Slot`.
This makes it especially easy for the outside api to know what type is
expected/returned.

* Change epoch duratioC

* rename all instances of slot number to slot

* Make the constructor private

Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-01-28 19:44:22 +00:00
Bastian KöcherandAndré Silva e21a61eac8 Rewrite the async code in BasicQueue (#7988)
* Rewrite the async code in `BasicQueue`

This is some smaller change to rewrite the async code in `BasicQueue`. I
require this for some other pr I'm working on ;)

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Hmm :D

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-01-26 16:38:57 +01:00
Bastian Köcher 97bb9ab35a Detect conflicting module names in construct_runtime! (#7968) 2021-01-25 10:37:20 +01:00
Bastian Köcherandcheme cd0ad4805d Make offchain indexing work (#7940)
* Make offchain indexing work

This fixes some bugs with offchain indexing to make it actually working ;)

* Fix tests

* Fix browser build

* Update client/db/src/offchain.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Remove seperation between prefix and key

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
2021-01-21 13:12:42 +01:00
Bastian Köcher 450b96c50d Sync: Propagate block announcement data (#7903)
* Sync: Propagate block announcement data

This pr adds a feature to the sync protocol to propagate the data that
we received alongside a block announcement. This is done by adding a
cache that caches the last X block announcement data where X is set to
the number of `in_peers` (giving every peer the chance to send us a
different block). This will be required by parachains to ensure that
even peers who are not connected to a collator receive the data
alongside the block announcement to properly validate it and request the
block.

* Review comment

* Bring back the code and add new variant to ensure we don't insert block
announce data when something wasn't checked

* Also use out_peers
2021-01-19 17:01:11 +01:00
Bastian Köcher 1560c8a704 Log target before prefix for more consistent logging (#7897)
* Log target before prefix for more consistent logging

As requested, this moves the target before the prefix to have consistent
logging between logs with and without a prefix.

* Add a space
2021-01-14 17:04:41 +01:00
Bastian Köcher 2d4b7fc6ff Fix max log level (#7851)
With the switch to tracing we did not set the `max_log_level` anymore.
This resulted in a performance degradation as logging did not early
exited and thus, `trace` logs were at least resolved every time.

This pr fixes it by ensuring that we set the correct max log level.
2021-01-08 19:32:49 +00:00
Bastian Köcher 94bb119ef9 Subkey should not import the entire world. (#7845)
There is no reason for subkey to import the default Substrate node to
support a feature that would only be usable for the Substrate node.
Subkey itself should be more the default key management binary for
Substrate related chains. If certain chains require some special
functionality, they can easily stick together their own "my-chain-key".
2021-01-07 13:14:42 +01:00
Bastian Köcher 8cdb5c01b6 Fix incorrect use of syn::exports (#7838)
* Fix incorrect use of syn::exports

Instead of using `syn::exports` we should import the trait from the
quote crate directly.

* Use own macro for test cases to fix compilation with latest syn

* Fix test
2021-01-06 22:22:23 +01:00
Bastian Köcher 452244514a Fix master build (#7837)
* Fix master build

* Use correct copyright year
2021-01-06 09:45:56 +00:00
Bastian Köcher e3e651f72c Happy new year (#7814)
* Happy new year

Updates the copyright years and fixes wrong license headers.

* Fix the template

* Split HEADER into HEADER-APACHE & HEADER-GPL
2021-01-04 09:03:13 +00:00
Bastian Köcher 672a2912b8 Fix ss58check test when executed with other tests (#7815)
There was a bug that could make other ss58 tests fail when being
executed with this one in parallel. This test changes the default ss58
version and if other tests are run at the time the default version is
changed, they would fail. To fix this problem, we now run the actual
test as a new process.
2021-01-03 21:17:31 +00:00
bd48ff310d Add proper commit_all to TestExternalities (#7808)
* Add proper `commit_all` to `TestExternalities`

This pr adds a propoer `commit_all` function to `TestExternalities` to
commit all changes from the overlay to the internal backend. Besides
that it fixes some bugs with handling empty dbs when calculating a delta
storage root. It also changes the way data is added to the in memory
backend.

* Update primitives/state-machine/src/testing.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Don't allow self proxies (#7803)

* Allow council to slash treasury tip (#7753)

* wk2051 | D4 |Allow council to slash treasury tip | p1

* Update frame/tips/src/lib.rs

Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>

* wk2051 | D5 |Allow council to slash treasury tip | p2

* wk2051 | D5 |Allow council to slash treasury tip | p3

* wk2051 | D5 |Allow council to slash treasury tip | p4

* wk2051 | D5 |Allow council to slash treasury tip | p5

* random change

* cargo run --release --features=runtime-benchmarks --manifest-path=bin/node/cli/Cargo.toml -- benchmark --chain=dev --steps=50 --repeat=20 --pallet=pallet_tips --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --output=./frame/tips/src/weights.rs --template=./.maintain/frame-weight-template.hbs

* fix typo

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/tips/src/tests.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* wk2052 | D1 | Allow council to slash treasury tip | p6

Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Review feedback

* Review feedback

* Update docs

* More docs

* Make it private

* Use `None`

* Use apply transaction

* Update primitives/state-machine/src/testing.rs

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: RK <r.raajey@gmail.com>
Co-authored-by: Xiliang Chen <xlchen1291@gmail.com>
Co-authored-by: Parity Benchmarking Bot <admin@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
2020-12-30 13:33:05 +00:00
Bastian Köcher 256871f553 Add CLI argument to disable log color output (#7795) 2020-12-29 23:08:58 +00:00
Bastian Köcher 05ae24d90a Fix base-path handling in key insert (#7775)
This fixes the handling of base-path when using `key insert`. Before
the base-path wasn't setup correctly, as done when starting a node. This
resulted in putting the keys into the wrong directory. This pr fixes
this by creating the correct base-path/config dir for the keystore.
Besides that it also removes the insert command from `subkey` as it
doesn't make that much sense. If requested, we could bring it back later.
2020-12-28 14:08:30 +00:00
Bastian Köcher a6c392d0cc Adds try_get for StorageMap and StorageDoubleMap (#7774)
* Adds `try_get` for `StorageMap` and `StorageDoubleMap`

* Switch to Value as return type
2020-12-22 14:41:49 +00:00
Bastian Köcher ce97b6e5c4 Make it possible to calculate the storage root as often as you want (#7714)
* Make it possible to calculate the storage as often as you want

So, until now each Substrate based blockchain has calculated the storage
root once, at the end of the block. Now there is Frontier that wants to
calculate some intermediate storage root. However this failed on block
import. The problem with that was the extrinsics root. When building the
block we stored `Default::default()` as extrinsics root, because yeah,
we don't know the extrinsics root before finishing the block. At the end
this extrinsics root was then calculated. But on block import we passed
the already known extrinsics root. This was no problem, as we removed
this value at the end of the block. However when you all the storage
root in between, that changes the storage root between block building
and block import.

This pr changes this behavior. It removes the `ExtrinsicsRoot` storage
entry and also doesn't pass it anymore to `System::initialize`. By doing
it, we remove the difference in the storage and fix the storage root mismatch.

* Fix bug with incorrectly calculating the extrinscs root

* Review feedback
2020-12-21 18:08:22 +01:00
Bastian Köcher 5b61b7361c Switch back to the crates.io release of trybuild (#7764)
* Switch back to the crates.io release of trybuild

My fix was merged on upstream and this release contains it. So, no
more reason to keep the git dependency.

* The lock file...
2020-12-19 10:22:08 +01:00
Bastian Köcher 104747daa0 Remove unnecessary parameter from sync (#7761) 2020-12-18 15:02:08 +01:00
Bastian KöcherandAndré Silva d3c7a99d3c Update common block in sync after importing blocks of a peer, please read UPDATE (#7733)
* Update common block in sync after importing blocks of a peer

This updates the sync code to update the common block of a peer, after
we have imported blocks from this peer. This fixes a bug for when we are
connected to one or more nodes that are doing a full sync as our node.
Nodes in full sync will not announce new blocks, as we don't send import
notifications on full sync. The problem as now that we were connected to
some peer that reported some low number as its best and we tried to sync
these blocks. But, as we did not update the common block of this peer,
we would sync these blocks over and over again. Being captured in some
time warp.
The solution to this problem is that we increase the common number as we
import blocks from this peer.

* Test

* Test name..

* Fix test

* Cleanup some code and write some new regression test

* Implement the ancestor search

* Check that the common number is smaller than the last finalized block

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/network/src/protocol/sync.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Change the way we build the status messages

* Start some new test...

* Finish test

* Rename test

* Update client/network/src/protocol.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2020-12-18 12:04:17 +01:00
Bastian Köcher 5a19ece7e3 Fix polkadot companion test (#7732) 2020-12-15 10:28:40 +01:00
Bastian Köcher 059746183d Default block announce validation should reject additional data (#7726)
If we are using the default block announce validator and receive
additional data alongside the block announcement, we should disconnect
this peer.
2020-12-14 18:47:08 +01:00
Bastian Köcher 588461f52a Make sure the companion CI test fails on any error (#7683) 2020-12-07 16:48:54 +01:00
Bastian Köcher d9b1b14769 Adds support for tuples in runtime-interface (#7672) 2020-12-06 15:14:19 +01:00
Bastian Köcher 947b82df68 Don't panic on a sink error (#7666) 2020-12-03 17:11:49 +01:00
Bastian Köcher 2877c038cf Use block requests to check if block responses are correct (#7653)
* Use block requests to check if block responses are correct

Before this pr sync relied on recently announced blocks to check if a
given peer response is correct. However this could lead to situations
where we requested a block from a peer and it gave us the requested, but
we rejected the response because this peer never send us an announcement
for the given block. See the added tests for a reproduction of the
problem.
With this pr, we now take the block request to check if a given response
matches the request. A node should not send us a block response
without a request anyway.

Essentially there is still a bug, because as you see in the test, we are
requesting block 2, while we already have this block imported. It even
happens that we request a block from the network that we have authored.
However a fix for this would require some more refactoring of the sync code.

* Revert change

* Give the test a proper name

* Add moar logging

* Move cheaper checks

* Move checks to common place
2020-12-03 15:49:23 +01:00
Bastian Köcher 9d43568e4e Use unreleased trybuild to fix CI (#7656) 2020-12-02 16:53:44 +01:00
Bastian Köcher 6722a83ba6 Forward storage changes in manual seal (#7614)
This prevents nodes from executing the same block 2 times.
2020-11-27 17:36:58 +01:00
Bastian Köcher 4f97481da7 Remove RpcMetrics weirdness (#7608)
* Remove `RpcMetrics` weirdness

The metrics was returning an error when prometheus was not given. This
was a really weird setup, especially when compared to all other metrics
that just do nothing if there is no registry.

* Fix browser build
2020-11-27 10:24:34 +01:00
Bastian Köcher d698d01378 Inform sync explicitly about new best block (#7604)
* Inform sync explicitly about new best block

Instead of "fishing" the new best block out of the processed blocks, we
now tell sync directly that there is a new best block. It also makes
sure that we update the corresponding sync handshake to the new best
block. This is required for parachains as they first import blocks and
declare the new best block after being made aware of it by the relay chain.

* Adds test

* Make sure async stuff had time to run
2020-11-26 16:33:17 +00:00