* Make it possible to calculate the storage as often as you want
So, until now each Substrate based blockchain has calculated the storage
root once, at the end of the block. Now there is Frontier that wants to
calculate some intermediate storage root. However this failed on block
import. The problem with that was the extrinsics root. When building the
block we stored `Default::default()` as extrinsics root, because yeah,
we don't know the extrinsics root before finishing the block. At the end
this extrinsics root was then calculated. But on block import we passed
the already known extrinsics root. This was no problem, as we removed
this value at the end of the block. However when you all the storage
root in between, that changes the storage root between block building
and block import.
This pr changes this behavior. It removes the `ExtrinsicsRoot` storage
entry and also doesn't pass it anymore to `System::initialize`. By doing
it, we remove the difference in the storage and fix the storage root mismatch.
* Fix bug with incorrectly calculating the extrinscs root
* Review feedback
* Leave some system data for inspection
There is not much benefit in being active when removing this data. It's
actively harmful when one tries to read the block number in runtime
APIs in the context of a block.
* Update the expected root hash
This is excepted since now we persist new members.
* Revert extrinsics_root to `take`
It's going away in one of the following PRs anyway
* Update the state root once again
* Update the comment on the storage items that are left in the storage
Excluding ExtrinsicsRoot since it's going away
We call `offchain_worker` with the state of the imported block and pass
the header of this block. However in the runtime we call all
`offchain_worker` functions with the number of the parent block. Besides
that we also pass all digests and not only the pre runtime digests. In
the context where the offchain worker is executed we have all digests, so
there is no real reason to only pass pre runtime digests. Another fix is
that we also insert the hash of the current header into the block hash map.
* Rename `ModuleToIndex` to `PalletRuntimeSetup`
Besides the renaming it also adds support getting the name of a pallet
as configured in the runtime.
* Rename it to `PalletInfo`
* Remove accidentally added files
* implement events handling, implement parent_id for spans & events
* add events to sp_io::storage
* update test
* add tests
* adjust limit
* let tracing crate handle parent_ids
* re-enable current-id tracking
* add test for threads with CurrentSpan
* fix log level
* remove redundant check for non wasm traces
* remove duplicate definition in test
* Adding conditional events API
* prefer explicit parent_id over current,
enhance test
* limit changes to client::tracing event implementation
* remove From impl due to fallback required on parent_id
* make tracing codecable
* replace with global tracing
* new tracing interface
* impl TracingSubscriber in client
* implement access to global TracingSubscriber from primitives
* span for wasm
* increment towards Wasm Tracing Subscriber implementation
* increment, remove sp-tracing from runtime-interface
* increment, it compiles
* attained original functionality with new mechanism
* implement remaining TracingSubscriber functions
* remove spans from decl_module
* add handling for encoded values
* Revert "replace with global tracing"
This reverts commit 8824a60deea54d9b437407a21c8ceaf6a1902ee5.
* Wasm Side Tracing
* tracing on wasm
* enable tracing wasm on node-runtime
* export all the macros in std
* tracing subscriber on wasm-side only
* pass spans and events over and record them
* reactivate previous code and cleanup
* further cleaning up
* extend the span macros, activate through executive
* tracking the actual extrinsic, too
* style
* fixing tests
* spaces -> tabs
* attempting to reactivate params
* activate our tests in CI
* some passing
* tests passing
* with core lazy
* global tracer for wasm side with pass over
* fixing metadata referencing
* remove const_fn feature requirement
* reenable dispatch traces
* reset client tracing
* further cleaning up
* fixing runtime-test
* move tracing-build setup into runtime-test
* Merge DebugWriter from tracing and frame-support, move to sp-std
* remove dangling fixme
* Docs for tracing primitives
* cleaning up a bit more
* Wasm interface docs
* optimise docs.rs setup
* adding tracing flags to uncomment
* remove brace
* fixing imports
* fixing broken syntax
* add required modules
* nicer formatting
* better target management
* adding low level storage tracing events into frame
* add custom Debug impl for WasmMetadata
* cloning profiler
* adding info about cloning profiler
* using in-scope for within calls
* proper time tracing, cleaning up println
* allow to disable tracing on runtime_interface-macro
* disable tracing for wasm-tracing-interface
* simplify wasm-tracing-api
* update client to new interface
* fixing docs and tests for sp-tracing
* update integration tests
* re-activating enter_span
* dropping FIXME, it's documented
* fix formatting
* fix formatting
* fix imports
* more debug info
* inform wasm about it being disabled by returning 1
* only one tracer, but enabled multi-all support
* make trait pub again for tests
* Apply suggestions from code review
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
* fixing wasm doc tests for proper usage
* remove unnecessary import
* fixing formatting
* minor style fixes
* downgrading wabt
* update error message for UI
* Fix interface test
* next attempt to fix macros
* geee
* revert tracing on hashed for future PR
* remove local macros, use originals
* we are able to convert to static items
* implement more WasmValue types
* adding support to convert str, debug and encoded values
* more minor fixes
* revert unsafe 'static making
* fix indentation
* remove commented lines
* bump all them tracing versions
* cleaning up docs and info
* document new flag
* the new layered system handles span cloning better
* Apply suggestions from code review
Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Matt Rutherford <mattrutherford@users.noreply.github.com>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: David <dvdplm@gmail.com>
It was a mistake to use the `*_verify` methods for signature batching.
This pr move the signature batching into their own functions. This is
required, because otherwise transaction signature verification infers
with other signature verifications.
This pr also temporarily disables signature batching. The functionality
stays, but we need to make sure that all nodes have the new runtime
interface, before we can bring back signature batching.
* Emit a PaymentParameters event once per block
This contains per-block paramaters need to calculate
fees off-chain.
* Add WeightToFee trait
* Add documentation to polynomial types
* Ignore pseudo code snippet for doc tests
* Use `Mul` implementation of Perbill
* Add tests for WeightToFeePolynomial
* Revert "Emit a PaymentParameters event once per block"
This reverts commit 6c4763baff3d8179676a3c1660fe7063fd56a8ca.
Co-authored-by: Gavin Wood <gavin@parity.io>
* Only check single extrinsics weight limit in validate_transaction.
* Add missing parameter to all pallets.
* Add tests, fix default configuration.
* Bump spec version.
* Use AvailableBlockRation to calculate MaxExtrinsicWeight
* Include post dispatch corrected weight in extrinsic events
* Drop the 'Post' from ApplyExtrinsicResultWithPostInfo to make it less verbose
* Apply suggestions from code review
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Use proper Event type in pallet_system tests
* Add test that the actual weight is returned by events
* Make fn extract_actual_weight cap at pre dispatch weight
* Bump spec version
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
* Split operational and normal weight
* Update other parts of the code
* relatively minor changes to add/fix behavior
* Limit reported block weight for fee adjustment
* fix test
* explicit match
* Move common logic out
* Update frame/system/src/lib.rs
Co-authored-by: Gavin Wood <gavin@parity.io>
* API improvement @bkchr
Co-authored-by: Gavin Wood <gavin@parity.io>
* Introduce `BlockExectionWeight` and `ExtrinsicBaseWeight`
* Add new traits everywhere
* Missed one update
* fix tests
* Update `check_weight` logic
* introduce `max_extrinsic_weight` function
* fix + add tests
* format nits
* remove println
* make test a bit more clear
* Remove minimum weight
* newlines left over from find/replace
* Fix test, improve clarity
* Fix executor tests
* Extrinsic base weight same as old `MINIMUM_WEIGHT`
* fix example test
* Expose constants
* Add test for full block with operational and normal
* Initiate test environment with `BlockExecutionWeight` weight
* format nit
* Update frame/system/src/lib.rs
Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Replace `TransactionBaseFee` with `ExtrinsicBaseWeight` (#5761)
* Replace `TransactionBaseFee` with `ExtrinsicBaseFee`
* Fix stuff
* Fix and make tests better
* Forgot to update this test
* Fix priority number in test
* Remove minimum weight from merge
* Fix weight in contracts
* remove `TransactionBaseFee` from contract tests
* Let `register_extra_weight_unchecked` go past `MaximumBlockWeight`
* address feedback
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
* Adds new event phase `Initialization`
Every event that was deposited inside of `on_initialize` was assigned to
the `ApplyExtrinsic(0)` phase which wasn't correct. This pr introduces a
new phase `Initialization`. This is the new phase while initializing
a block. After initialization we switch to `ApplyExtrinsic(N)` and at
the end to `Finalization` as before.
* Set `ExecutionPhase` in `initialize`
* Increment `spec_version`
* Move runtime upgrade to `frame-executive`
Instead of storing the runtime upgraded in the space of `frame-system`,
this moves it to `frame-executive`. We also start storing the
`spec_version` and `impl_version` the last runtime upgrade was applied.
This scheme has multiple advantages:
- We don't need to make sure that runtime upgrade was set.
- This will work in the future when we want to test a runtime upgrade,
because the `on_runtime_upgrade` will be executed automatically when the
runtime changes.
* Move `LAST_RUNTIME_UPGRADE` key to `frame-executive`
* Add some more documentation
* Make sure `spec_version` always increases between runtime upgrades
* Fix test
* Upgrade `parity-multiaddr` to fix compilation
* Also check if the `spec_name` changed
* Remove `RuntimeUpgraded` storage entry
* Don't run on genesis
* Initial idea of `on_runtime_upgrade`
* Runtime storage for module version
* Gui shawntabrizi runtime upgrade (#5118)
* adding unleash to ci (#5020)
* adding unleash to ci
* fixing formatting
* with a dot please
* alpha.3 now
* do not publish testing helpers
* remove old test-helpers cruft
* fix cargo.lock
* with alpha 4
* do not publish runtime-interface-test either
* disable more test crates from publishing
* switch to alpha.5
* replace tempdir with tempfile
* update lru
* switch to bytes 0.5
* release script fixes
* switch on and to latest alpha
* BUT THE SPACES
* Fix: CI failing for some CLI tests (#5043)
* Initial commit
Forked at: ad90ab7ec9
Parent branch: origin/master
* Increase killing grace period of CLI tests and display more info
* Use --dev everywhere possible
* Put pruning mode to its own params struct
* Add pruning params to export-blocks command
* Added missing file
* Removed not-dev mode in tests
* Add pruning mode to the revert command
* Decrease killing grace period again
* Move back unsafe_pruning to import_params
* Applied proposed changes
* aura: remove unused tx pool (#5046)
* aura: remove unused transaction pool parameter
* node-template: remove transaction pool from aura
* aura: fix tests
* Extend rust nightly detection in `wasm-builder` (#5021)
Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.
* Add steps setting to benchmarking CLI (#5033)
* Add steps setting to CLI, use max value to hit worst case.
* Bump impl_version.
* Apply review suggestion.
* Remove indices from node-template (#5025)
* Remove indices from node-template
* Use identity lookup instead
* Bump impl
* clean cargo.toml
* Fix documentation for "BlockBuilder::push_trusted" (#5051)
* fix doc
* rephrase
* do not check unleash on every PR, only master and tags (#5054)
* do not check unleash on every PR, only master and tags
* move scripts folder
* add signed-tag check to CI
* remove publish-to-crates-io dependencies
Co-authored-by: s3krit <pugh@s3kr.it>
* prepare version to alpha.1 (#5055)
bump version to -alpha.1
* Sync: validate block responses for required data (#5052)
* Less verbose state-db logging
* Validate block responses for block bodies
* Update client/network/src/protocol.rs
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
* Added validation test
* Disconnect on missing header as well
* Typo
Co-Authored-By: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
* Make these chainspecs fields private (#5031)
* Fix dockerfile (#5059)
* Adds documentation for `wipe` and `commit` (#5053)
* Adds documentation for `wipe` and `commit`
This adds documentation to `wipe` and `commit` of `Externalities`.
Besides that it removes the default implementation that would just panic
and requires that all implementers of the trait implement the functions.
* Update primitives/externalities/src/lib.rs
Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
* Fix the issue with `trybuild`'s `ui` tests (#4992)
* repro ui bug
* fix the tests
* test with the new image
* test without CARGO_HOME
* test without fixes
* test again
* fix trybuild old versions
* bump CArgo.lock
* fix trybuild newest versions
* bump Cargo.lock
* trying on the latest image
* bump Cargo.lock
* run with the old image
* ci will be green on the image from 2020-02-19 [skip ci]
* bump Cargo.lock
* Activate publishing of draft releases... (#5062)
* Activate publishing of draft releases...
... And fix the message sending (missing parameter).
* publish_draft_release.sh now checks latest...
... release on github rather than just a tag
* Fix/div by zero (#5041)
* Handle gas_price being zero separately
* Bump spec_version
* Add a unit & integration tests for gas price = 0
* set missing metadata fields, prepping alpha.2 (#5067)
* setting first batch of descriptions
* fix what I just broke
* next batch
* and pallets, too
* last batch
* set cargo.lock
* keep'em dev-deps
* bump version to alpha.2
* Fix revalidation not revalidating multiple times (#5065)
* removes use of sc_client::Client from sc_finality_grandpa (#5030)
* removes use of sc_client::Client from sc_finality_grandpa
* code formatting
* code formatting
* removes use of sc_client::Client from sc_finality_grandpa
* Remove deprecated host functions (#5038)
Sadly we need to keep one function `ext_blake2_256`. This function is
manually defined in `sp-core`.
* removes use of sc_client::Client from sc_basic_authorship (#5050)
* removes use of sc-client from sc-basic-authorship
* refactor use of ProposerFactory
* correct dep path
* pallet-transaction-payment clean up (#5070)
* Formatting clean up
* Introduce separate setters for the fees.
* *: Rename prometheus-exporter crate to substrate-prometheus-end… (#5076)
This patch renames the crate for the following two reasons:
1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".
2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.
* rename `browser-utils` to `substrate-browser-utils` (#5079)
* prepping for Alpha.3 (#5080)
* Bump to alpha.3
* update gitlab-ci
* Propagate DispatchError for benchmarks. (#5075)
* Propagate DispatchError for benchmarks.
* Apply review suggestions.
* Use RuntimeString.
* fix expect
Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
* Add options to overwrite range bounds in benchmark command. (#5072)
* Add --mins --maxs to benchmark command.
* Apply review suggestions.
* Update yamux to version 0.4.4. (#5086)
* Remove more instances of futures01 (#4633)
* Start removing last few instances of futures01
* Use to_poll on wasm
* Revert "Use to_poll on wasm"
This reverts commit 1c61728f10d520df5f9b28c415a0db68e478b9c7.
* Fix fg test
* Upgrade network test futures
* Update offchain hyper version
* Update service test
* bump tokio to 0.2.10
* Removed some unneeded tokios
* fixes
* fix run_until_all_full
* Make service test debuggable
* Update client/offchain/src/api/http.rs
Co-Authored-By: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>
* Add service_test to test-int output
* nitpicking
* Finally fix test
* Give up and revert client/serviec/test
* Revert gitlab ci too
Co-authored-by: Demi Obenour <demi@parity.io>
* Make export blocks default to json on stdout (#5090)
* Make export blocks default to json on stdout
* Multiline instead of single line to stay under 100 cols
* Change --json flag to --binary, defaulting to json
* Offence reporting returns a result (#5082)
* Offence reporting returns a result
* Bump spec_version
* Use unwrap instead of assertions
* Fix more review grumbles
* Update to libp2p 0.16.2 (#5088)
* Remove request ID from the new protocol (#5049)
* Make sure we remove a peer on disconnect in gossip (#5104)
* Make sure we remove peers on disconnect in gossip state machine
* Clear up the code
* Add a comment
* Expose `state-db` memory info (#5110)
This exposes memory statistics from the state-db.
* Change extrinsic_count to extrinsic_index in pallet-utility (#5044)
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
* Add more metrics to prometheus (#5034)
* Add a few things
* Add finality_grandpa_round
* fix fg tests
* Nitpicks
* Nitpicks
* Fix name of prometheus crate
* Update to SCALE 1.2.0 (#5113)
This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.
* Lazy payouts (#4474)
* TODOs
* Remove superfluous:
* partial implementation
* full implementation
* fix preferences
* update comments
* upgrade test WIP
* fix more tests
* fix cutoff
* fix saturation
* comment
* upgrade mock
* upgrade test
* WIP migration
* WIP migration
* remove slot stake stuff
* fix merge
* migration of ledger
* remove equalize from test
* add test
* fix
* update doc
* fix compilation
* improve test readibility
* improve doc
* fix most todo
* fix migration and test
* remove println
* WIP
* add test and spec
* weight
* update doc
* safer end_era
* fix exposure of conversion
* Revert "safer end_era"
This reverts commit 72ff737d27be67d87308514b13e2574bc5f09fce.
* fix useless put
* exposure clipped
* doc
* fix payout with clipped
* fix node runtime
* add doc
* pluggable and generalized staking module
* remove print
* update doc
* refactor
* improve documentation and implementation
* fix test
* Fix test
* fix test
* fix test
* fix remove lowest stake from exposure, not biggest.
* nomination index arguments in nominator_payout
* add test
* try to fix offence
* apply slashed and bond eras until active era
* doc
* update spec version
* add test upgrade from previous test environment
* Apply suggestions from code review
Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>
* nominators upgrade has been cleaned
* dynamic history depth implementation
* make current_era - history_depth included
* Change equality check to start era to less than or equal
* Use era specific validator prefs
* Add print statement and comment about start era if <
* fix next_reward overflow
* make more check for bad era claim for zero cost
* small refactor
* code refactor + fix use of deprecated storage
* fix wasm build
* add comment
* Fix tests
* remove outdated comment
* Apply suggestions from code review
Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>
* gather active era information into one storage
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
* impl on_runtime_upgrade
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
* make compile
* Add some tests
* docs
* Remove "useless" code
* Fix merge and use n + 1 block number
* Fix tests
* unfix ui tests
* Update on_initialize.stderr
* fix test
* Fix test
* Bump spec
* Remove `on_finalise` and `on_initialise`
* Use bool for tracking runtime upgraded
* typo
* Support runtime upgrade with `set_storage`
* Refactor migration code location
* add trailing newlines
* Remove old `IsUpgraded` flag
* Update state root
* Exhaustive match statement
* Apply suggestions from code review
Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
Bastian Köcher