Commit Graph

757 Commits

Author SHA1 Message Date
Parth 77eba1351a add ExternalitiesExt for TestExternalities (#8877) 2021-05-21 10:21:12 +00:00
Andreas Doerr a183031eef Fix doc for CryptoStore::ecdsa_sign_prehashed() (#8876)
* fix ecdsa_sign_prehashed() doc

* document both instances
2021-05-21 09:52:20 +02:00
Bastian Köcher ee726ed55d Improve impl_opaque_keys! (#8856)
The macro should assume less about the scope where it is being used in.
In this case it is about not assuming that the crate where the macro is
called in provides a `std` feature.
2021-05-19 08:37:42 +00:00
Andreas Doerr ed39290f91 Add CryptoStore::ecdsa_sign_prehashed() (#8838)
* Pair::sign_prehashed()

* add CryptoStore::ecdsa_sign_prehashed()

* add test for testing keystore

* address review comments
2021-05-18 16:07:45 +02:00
Kian Paimani b5f23bfd1c Ensure election offchain workers don't overlap (#8828)
* Initial version, well tested, should work fine.

* Add one last log line

* Update frame/election-provider-multi-phase/src/unsigned.rs

Co-authored-by: Gavin Wood <gavin@parity.io>

* Update frame/election-provider-multi-phase/src/unsigned.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Update frame/election-provider-multi-phase/src/unsigned.rs

Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>

* Fix a few more things

* fix build

* rewrite the whole thing with a proper lock

* clean

* clean some nits

* Add unit tests.

* Update primitives/runtime/src/offchain/storage_lock.rs

Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Fix test

* Fix tests

Co-authored-by: Gavin Wood <gavin@parity.io>
Co-authored-by: Guillaume Thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Peter Goodspeed-Niklaus <coriolinus@users.noreply.github.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-05-18 12:22:06 +00:00
Sergei Shulepov bee5c2dd71 Emit wasm sections only when compiling to wasm (#8845) 2021-05-18 12:06:35 +00:00
Andreas Doerr a68ed7c3d8 [clippy] Fix clippy issues for crate sp-core (#8809)
* Fix clippy issues for crate sp-core

* Update primitives/core/benches/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update primitives/core/src/ed25519.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Remove clippy attributes

* Missed a clippy attribute

* remove clippy attributes for bechmarks as well

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-05-18 09:38:56 +02:00
Ayush Mishra c28ab1fcac Improve match statement (#8817) 2021-05-15 21:43:58 +02:00
Sergei Shulepov 0849bcce0e Embed runtime version as a custom section (#8688)
* emit a custom section from impl_runtime_apis!

This change emits a custom section from the impl_runtime_apis! proc macro.

Each implemented API will result to emitting a link section `runtime_apis`.
During linking all sections with this name will be concatenated and
placed into the final wasm binary under the same name.

* Introduce `runtime_version` proc macro

This macro takes an existing `RuntimeVersion` const declaration, parses
it and emits the version information in form of a linking section.
Ultimately such a linking section will result into a custom wasm
section.

* Parse custom wasm section for runtime version

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Fix sc-executor integration tests

* Nits

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Refactor apis section deserialization

* Fix version decoding

* Reuse uncompressed value for CallInWasm

* Log on decompression error

* Simplify if

* Reexport proc-macro from sp_version

* Merge ReadRuntimeVersionExt

* Export `read_embedded_version`

* Fix test

* Simplify searching for custom section

Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-05-12 13:39:08 +00:00
Nikolay Volf 31c5ddc1ce update environmental (#8789) 2021-05-12 14:48:01 +02:00
Sergei Shulepov 6f9bc19972 Make Allocator trait pub (#8777)
We need this in order to be able to assemble more fine grained host
function sets.

E.g. we don't want to use `SubstrateHostFunctions` for PVF. We would
better whitelist certain host functions. However, we cannot do that
because we cannot refer to the `Allocator` runtime interface.

I have not been able to find the reason why it wasn't made `pub` in the
first place, but do not see any reason why not to.
2021-05-10 17:42:55 +00:00
Shaun Wang 2a38b23062 Add arithmetic dispatch errors. (#8726)
* Add arithmetic dispatch errors.

* Replace custom overflow errors.

* Replace custom underflow and division by zero errors.

* Replace overflow/underflow in token error.

* Add token and arithmetic errors in dispatch error equality test.

* Trigger CI.
2021-05-10 10:14:02 +02:00
Bastian Köcher 9b4f9c9e42 Remove unneeded trait bounds (#8765)
Before we required these trait bounds because of some bug in rustc, but
now as this bug is fixed they can be removed.
2021-05-09 19:17:42 +00:00
Veniamin 221aa2b865 equilibrium added ss58 prefix (#8342)
Co-authored-by: Veniamin <benjamin@eosdt.com>
2021-05-06 14:16:30 +02:00
Alexander Popiak 6eb091c736 derive Encode and Decode for BigUint (#8744) 2021-05-06 10:56:24 +00:00
Roman Proskuryakov 7c3a3f79ba Add a readme about feature metered for sp-utils (#8736) 2021-05-05 15:33:52 +00:00
Aten 3d259fdee5 remove patract ss58version for it's useless now (#8738) 2021-05-05 14:41:30 +00:00
mattrutherford 6e8957b095 RPC to trace execution of specified block (#7780)
* Add filter reload handle

* add RPC, move logging module from cli to tracing

* remove dup fn

* working example

* Update client/rpc-api/src/system/mod.rs

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Prefer "set" to "reload"

* Re-enable the commented out features of the logger

* Remove duplicate code

* cleanup

* unneeded lvar

* Bump to latest patch release

* Add new CLI option to disable log filter reloading,

Move profiling CLI options to SharedParams

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Applied suggestions from reviews

* Fix calls to init_logger()

* Handle errors when parsing logging directives

* Deny `system_setLogFilter` RPC by default

* One more time

* Don't ignore parse errors for log directives set via CLI or RPC

* Improve docs

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/config.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fix merge errors

* include default directives with system_setLogFilter RPC,

implement system_rawSetLogFilter RPC to exclude defaults

* docs etc...

* update test

* refactor: rename fn

* Add a test for system_set_log_filter – NOTE: the code should likely change to return an error when bad directives are passed

* Update client/cli/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Address review grumbles

* Add doc note on panicking behaviour

* print all invalid directives before panic

* change RPCs to: addLogFilter and resetLogFilter

* make CLI log directives default

* add comments

* restore previous behaviour to panic when hard-coded directives are invalid

* change/refactor directive parsing

* fix line width

* add test for log filter reloading

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* finish up suggestions from code review

* improve test

* change expect message

* change fn name

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* add docs, remove unused fn

* propagate Err on invalid log directive

* Update tracing-subscriber version

* Improve docs for `disable_log_reloading` CLI param

* WIP implementation: RPC and trace capturing

* WIP

* fix incorrect number of digest items

* return errors

* add From impl for Values, rename structs

* fixes

* implement option to choose targets for traces

* rename fn

* fix EnvFilter and add root span

* fix root span

* add docs, remove unnecessary traits

* fix regression on parent_id introduced in a9c73113a8

* fix line width

* remove unused

* include block hash, parent hash & targets in response

* move types from sp-tracing into sp-rpc

move block and parent hash into root of BlockTrace

* switch from log::trace to tracing::trace in state-machine

* use unsigned integer type to represent Ext::id in traces

* ensure id is unique by implementing Subscriber

tracing_subscriber::FmtSubscriber does not guarantee
unique ids

* indentation

* fix typo

* update types

* add sp_io::storage events

* Change response format

- update types
- record distinct timestamps
- sort spans by first entered

* convert to HexDisplay, refactor

* Sort out fallout from merge

* Update client/rpc-api/src/state/mod.rs

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Exit early unless the node runs with --rpc-methods=Unsafe

* Better error handling

* Use wasm-timer

* revert trace alteration in `state-machine` and remove events in `sp_io::storage`

Resolve in follow-up PR

* Review feedback: less collects

* Without Arcs

* Fix span exit

* typo

* cleanup

* Add a few debug messages to tracing module

* Structure traces state-machine/ext; Dispatchable extrinsics spans not working

* Correctly encode Option storage values

* Remove test field for Put and Get

* Try out some changes to dispatch macro

* Add various log messages in dispatch

* Add span dispatch span to new proc macro

* Remove debug messages in dispatch

* Trivial clean up

* Structure remaining state-machine traces (ChangesRoot*)

* Removed unnesecary tracing targets

* Remove log

* New cargo.lock post merge

* Add logging for wasm_overrides

* remove temp logs

* remove temp logs

* remove unused dep

* remove temp logs

* add logging to wasm_overrides

* add logging to state_tracing

* add logging for spans to substrate (includes timings)

* Skip serializing some event fields; Remove most storage traces

* Bring back all ext.rs traces

* Do not skip bool values in events

* Skip serializing span values

* Serialize span values; remove some trace events in ext

* Remove more trace events

* Delete commented out traces

* Remove all unused traces

* Add event filtering

* Fix typo

* wip - change response types to be more efficient

missing import

type

* Serialize struct fields as camelCase

* Add back in event filtering

* Remove name field from event

* Sort spans by time entered

* Sort spans in ASCending order

* Add storage keys target param to rpc

* Limit payload size; improve hash fields; include storage keys

- cleanup event_key_filter
- better block hash representation
- limit payload size
- cleanup based on andrews comments

* Error when serialized payload is to big

* Import MAX_PAYLOAD from rpc-servers

* Clean up ext.rs

* Misc. cleaning and comments

* Strict ordering span Id; no span sort; adjust for rpc base payload

* Add RPC docs to rpc-api/src/state/mod

* Make params bullet points

* Update primitives/rpc/src/tracing.rs

* Put all tracing logic within trace

* Remove attr.record in new_span

* Add back value record in new_span

* restore result collection in ext

* Revert "Add back value record in new_span"

This reverts commit baf1a735f23e5eef1bf6264adfabb788910fa661.

* 🤦

* more 🤦

* Update docs; Try fix line width issues

* Improve docs

* Improve docs

* Remove default key filters + add key recs to docs

* Try restore old traces

* Add back doc comment

* Clean up newlines in ext.rs

* More new line remova;
l

* Use FxHashMap

* Try use EnvFilter directives for event filtering

* Remove directive, filter events by fields

* Use trace metadata correctly

* Try EnvFilter directive with all default targets

* Revert "Try EnvFilter directive with all default targets"

This reverts commit 4cc6ebc721d207c3b846444174f89d45038525ac.

* Clean up clippy warning

* Incorporate Niklas feedback

* Update trace/log macro calls to have better syntx

* Use Ordering::Relaxed

* Improve patch and filter doc comment

* Clean up `BlockSubscriber::new`

* Try optimize `BlockSubscriber::enabled`

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Apply suggestions from code review

Co-authored-by: David <dvdplm@gmail.com>

* Use contains_key

* use heuristic for payload size

* Add error tupe for client::tracing::block

* Minor tweaks

* Make a note about `--features with-tracing`

* Add CURL example to RPC docs

* Link to substrate-archibe wasm

* Trivial doc clean up based on David feedback

* Explicit result type name

* Respect line length

* Use the error

* Don't print timings when spans close

* Fix failing sc-rpc-api

* Update  sp-tracing inner-line doc

* Update client/tracing/src/block/mod.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/call_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/call_executor.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/tracing/src/block/mod.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/tracing/src/block/mod.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Address some review grumbles

* Update primitives/state-machine/src/ext.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Use result_encoded structure fields in ext.rs

* Use value key for ext put

* Add notes about tracing key names matter

Co-authored-by: Matt <mattrutherford@users.noreply.github.com>
Co-authored-by: David <dvdplm@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
2021-05-05 07:44:36 +00:00
André Silva 122cb6038a primitives: remove random_seed from BlockBuilder API (#8718)
* primitives: remove random_seed from BlockBuilderApi

* node: remove random_seed

* primitives: bump BlockBuilderApi version

* client: rpc: fix test
2021-05-04 09:29:44 +00:00
Bastian Köcher 2675741a09 Rework inherent data client side (#8526)
* Lol

* Yeah

* Moare

* adaasda

* Convert AURA to new pallet macro

* AURA: Switch to `CurrentSlot` instead of `LastTimestamp`

This switches AURA to use `CurrentSlot` instead of `LastTimestamp`.

* Add missing file

* Update frame/aura/src/migrations.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Remove the runtime side provide inherent code

* Use correct weight

* Add TODO

* Remove the Inherent from AURA

* 🤦

* Remove unused stuff

* Update primitives authorship

* Fix babe inherent data provider

* Fix consensus-uncles

* Fix BABE

* Do some further changes to authorship primitives... :D

* More work

* Make it compile the happy path

* Make it async!

* Take hash

* More stuff

* Hacks

* Revert "Hacks"

This reverts commit cfffad88668cfdebf632a59c4fbfada001ef8251.

* Fix

* Make `execute_block` return the final block header

* Move Aura digest stuff

* Make it possible to disable equivocation checking

* Fix fix fix

* Some refactorings

* Comment

* Fixes fixes fixes

* More cleanups

* Some love

* Better love

* Make slot duration being exposed as `Duration` to the outside

* Some slot info love

* Add `build_aura_worker` utility function

* Copy copy copy

* Some stuff

* Start fixing pow

* Fix pow

* Remove some bounds

* More work

* Make grandpa work

* Make slots use `async_trait`

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make grandpa work

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make it async

* Fix fix

* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* Make grandpa tests compile

* More Babe tests work

* Fix network test

* Start fixing service test

* Finish service-test

* Fix sc-consensus-aura

* Fix fix fix

* More fixes

* Make everything compile *yeah*

* Make manual-seal compile

* More fixes

* Start fixing Aura

* Fix Aura tests

* Fix Babe tests

* Make everything compile

* Move code around and switch to async_trait

* Fix Babe

* Docs docs docs

* Move to FRAME

* Fix fix fix

* Make everything compile

* Last cleanups

* Fix integration test

* Change slot usage of the timestamp

* We really need to switch to `impl-trait-for-tuples`

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update primitives/inherents/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Some extra logging

* Remove dbg!

* Update primitives/consensus/common/src/import_queue/basic_queue.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-05-03 14:39:25 +00:00
Lohann Paterno Coutinho Ferreira ffca28ba59 Remove Offence delay (#8414)
* Removed can_report api from OnOffenceHandler

* Removed DeferredOffences and create a storage migration

* Removed missing comments

* Mock set_deferred_offences and deferred_offences methods

* OnOffenceHandler::on_offence always succeed

* Fix benchmark tests

* Fix runtime-benchmark cfg methods

* Removed 'applied' attribute from Offence event

* refactor deprecated deferred offences getter

* Validate if offences are submited after on_runtime_upgrade

* update changelog

* Remove empty lines

* Fix remove_deferred_storage weights

* Remove Offence::on_runtime_upgrade benchmark

* Revert CHANGELOG.md update

* Deprecate DeferredOffenceOf type

* Update copyright

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Add migration logs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fix migration log

* Remove unused import

* Add migration tests

* rustfmt

* use generate_storage_alias! macro

* Refactor should_resubmit_deferred_offences test

* Replace spaces by tabs

* Refactor should_resubmit_deferred_offences test

* Removed WeightSoftLimit

* Removed WeightSoftLimit from tests and mocks

* Remove unused imports

* Apply suggestions from code review

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2021-05-03 07:53:09 +00:00
Peter Goodspeed-Niklaus c786fb21a0 Refactor election solution trimming for efficiency (#8614)
* Refactor election solution trimming for efficiency

The previous version always trimmed the `CompactOf<T>` instance,
which was intrinsically inefficient: that's a packed data structure,
which is naturally expensive to edit. It's much easier to edit
the unpacked data structures: the `voters` and `assignments` lists.

* rework length-trim tests to work with the new interface

Test suite now compiles. Tests still don't pass because the macro
generating the compact structure still generates `unimplemented!()`
for the actual `compact_length_of` implementation.

* simplify

* add a fuzzer which can validate `Compact::encoded_size_for`

The `Compact` solution type is generated distinctly for each runtime,
and has both three type parameters and a built-in limit to the number
of candidates that each voter can vote for. Finally, they have an
optional `#[compact]` attribute which changes the encoding behavior.

The assignment truncation algorithm we're using depends on the ability
to efficiently and accurately determine how much space a `Compact`
solution will take once encoded.

Together, these two facts imply that simple unit tests are not
sufficient to validate the behavior of `Compact::encoded_size_for`.
This commit adds such a fuzzer. It is designed such that it is possible
to add a new fuzzer to the family by simply adjusting the
`generate_solution_type` macro invocation as desired, and making a
few minor documentation edits.

Of course, the fuzzer still fails for now: the generated implementation
for `encoded_size_for` is still `unimplemented!()`. However, once
the macro is updated appropriately, this fuzzer family should allow
us to gain confidence in the correctness of the generated code.

* Revert "add a fuzzer which can validate `Compact::encoded_size_for`"

This reverts commit 916038790887e64217c6a46e9a6d281386762bfb.

The design of `Compact::encoded_size_for` is flawed. When `#[compact]`
mode is enabled, every integer in the dataset is encoded using run-
length encoding. This means that it is impossible to compute the final
length faster than actually encoding the data structure, because the
encoded length of every field varies with the actual value stored.

Given that we won't be adding that method to the trait, we won't be
needing a fuzzer to validate its performance.

* revert changes to `trait CompactSolution`

If `CompactSolution::encoded_size_for` can't be implemented in the
way that we wanted, there's no point in adding it.

* WIP: restructure trim_assignments_length by actually encoding

This is not as efficient as what we'd hoped for, but it should still
be better than what it's replacing. Overall efficiency of
`fn trim_assignments_length` is now `O(edges * lg assignments.len())`.

* fix compiler errors

* don't sort voters, just assignments

Sorting the `voters` list causes lots of problems; an invariant that
we need to maintain is that an index into the voters list has a stable
meaning.

Luckily, it turns out that there is no need for the assignments list
to correspond to the voters list. That isn't an invariant, though previously
I'd thought that it was.

This simplifies things; we can just leave the voters list alone,
and sort the assignments list the way that is convenient.

* WIP: add `IndexAssignment` type to speed up repeatedly creating `Compact`

Next up: `impl<'a, T> From<&'a [IndexAssignmentOf<T>]> for Compact`,
in the proc-macro which makes `Compact`. Should be a pretty straightforward
adaptation of `from_assignment`.

* Add IndexAssignment and conversion method to CompactSolution

This involves a bit of duplication of types from
`election-provider-multi-phase`; we'll clean those up shortly.

I'm not entirely happy that we had to add a `from_index_assignments`
method to `CompactSolution`, but we couldn't define
`trait CompactSolution: TryFrom<&'a [Self::IndexAssignment]` because
that made trait lookup recursive, and I didn't want to propagate
`CompactSolutionOf<T> + TryFrom<&[IndexAssignmentOf<T>]>` everywhere
that compact solutions are specified.

* use `CompactSolution::from_index_assignment` and clean up dead code

* get rid of `from_index_assignments` in favor of `TryFrom`

* cause `pallet-election-provider-multi-phase` tests to compile successfully

Mostly that's just updating the various test functions to keep track of
refactorings elsewhere, though in a few places we needed to refactor some
test-only helpers as well.

* fix infinite binary search loop

Turns out that moving `low` and `high` into an averager function is a
bad idea, because the averager gets copies of those values, which
of course are never updated. Can't use mutable references, because
we want to read them elsewhere in the code. Just compute the average
directly; life is better that way.

* fix a test failure

* fix the rest of test failures

* remove unguarded subtraction

* fix npos-elections tests compilation

* ensure we use sp_std::vec::Vec in assignments

* add IndexAssignmentOf to sp_npos_elections

* move miner types to `unsigned`

* use stable sort

* rewrap some long comments

* use existing cache instead of building a dedicated stake map

* generalize the TryFrom bound on CompactSolution

* undo adding sp-core dependency

* consume assignments to produce index_assignments

* Add a test of Assignment -> IndexAssignment -> Compact

* fix `IndexAssignmentOf` doc

* move compact test from sp-npos-elections-compact to sp-npos-elections

This means that we can put the mocking parts of that into a proper
mock package, put the test into a test package among other tests.

Having the mocking parts in a mock package enables us to create a
benchmark (which is treated as a separate crate) import them.

* rename assignments -> sorted_assignments

* sort after reducing to avoid potential re-sort issues

* add runtime benchmark, fix critical binary search error

"Why don't you add a benchmark?", he said. "It'll be good practice,
and can help demonstrate that this isn't blowing up the runtime."

He was absolutely right.

The biggest discovery is that adding a parametric benchmark means that
you get a bunch of new test cases, for free. This is excellent, because
those test cases uncovered a binary search bug. Fixing that simplified
that part of the code nicely.

The other nice thing you get from a parametric benchmark is data about
what each parameter does. In this case, `f` is the size factor: what
percent of the votes (by size) should be removed. 0 means that we should
keep everything, 95 means that we should trim down to 5% of original size
or less.

```
Median Slopes Analysis
========
-- Extrinsic Time --

Model:
Time ~=     3846
    + v    0.015
    + t        0
    + a    0.192
    + d        0
    + f        0
              µs

Min Squares Analysis
========
-- Extrinsic Time --

Data points distribution:
    v     t     a     d     f   mean µs  sigma µs       %
<snip>
 6000  1600  3000   800     0      4385     75.87    1.7%
 6000  1600  3000   800     9      4089     46.28    1.1%
 6000  1600  3000   800    18      3793     36.45    0.9%
 6000  1600  3000   800    27      3365     41.13    1.2%
 6000  1600  3000   800    36      3096     7.498    0.2%
 6000  1600  3000   800    45      2774     17.96    0.6%
 6000  1600  3000   800    54      2057     37.94    1.8%
 6000  1600  3000   800    63      1885     2.515    0.1%
 6000  1600  3000   800    72      1591     3.203    0.2%
 6000  1600  3000   800    81      1219     25.72    2.1%
 6000  1600  3000   800    90       859     5.295    0.6%
 6000  1600  3000   800    95     684.6     2.969    0.4%

Quality and confidence:
param     error
v         0.008
t         0.029
a         0.008
d         0.044
f         0.185

Model:
Time ~=     3957
    + v    0.009
    + t        0
    + a    0.185
    + d        0
    + f        0
              µs
```

What's nice about this is the clear negative correlation between
amount removed and total time. The more we remove, the less total
time things take.
2021-05-03 07:26:35 +00:00
François Garillot 052be8bbef Simplify some Option / Result / ? operator patterns (#8653)
* Simplify some Option / Result / ? operator patterns

When those match a combinator exactly.

Tool-aided by [comby-rust](https://github.com/huitseeker/comby-rust).

* adjust after review

* adjust post-review
2021-04-23 13:36:10 +00:00
Bastian Köcher 541692c4a8 Use host max log level when initializing the RuntimeLogger (#8655)
* Use host max log level when initializing the `RuntimeLogger`

This should fix performance problems introduced by logging under certain
circumstances. Before we always called into the host and the host was
doing the log filtering, now as the correct max log level is set, we
don't call into the host for every log line to check if it should be
logged. However, we would still call into the host to determine if
something should be logged when `something=trace` is given as we don't
forward the log targets that are enabled.

* Finish the pr
2021-04-23 13:22:39 +00:00
Xiliang Chen 327934c19e impl Zero and One for fixed point numbers (#8647)
* impl Zero and One for fixed point numbers

* update librocksdb-sys to support M1

* fix build

* trigger CI
2021-04-22 22:02:54 +00:00
Pierre Krieger d64f79924a Update to libp2p 0.37 (#8625)
* Update to libp2p 0.37

* Line widths

* Fix tests
2021-04-18 08:04:45 +00:00
Guillaume Thiolliere bb343c7c26 fix doc and dyn (#8631) 2021-04-17 17:01:31 +02:00
Bastian Köcher c8136bd1df Remove serde requirement from FRAME macros (#8628)
* Remove `serde` requirement from FRAME macros

Currently there is some implicit requirement on `serde` being present in
the `Cargo.toml` of a pallet when `GenesisConfig` is used. This pr
removes this requirement by using the serde attribute `serde(crate = "..")`.

* build a unique reexport of serde in impl_opaque_keys, by abusing paste doc concatenation

* Optimize

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2021-04-16 12:42:37 +02:00
Bastian Köcher 14b5acab86 Introduce a "dynamic" block size limit for proposing (#8588)
* Introduce a "dynamic" block size limit for proposing

This adds support for using a dynamic block size limit per call to
`propose`. This is required for Cumulus/Parachains to always use stay in
the limits of the maximum allowed PoV size.

As described in the docs, the block limit is only checked in the process
of pushing transactions. As we normally do some other operations in
`on_finalize`, it can happen that the block size still grows when there
is some proof being collected (as we do for parachains). This means,
that the given block limit needs to be rather conservative on the actual
value and should not be the upper limit.

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: Andronik Ordian <write@reusable.software>

* More future proof encoded size updating

* Use `ProofRecorderInner`

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/slots/src/lib.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/slots/src/slots.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/basic-authorship/src/basic_authorship.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-04-14 17:56:22 +00:00
Vladislav d0933c4634 Add Sora network SS58 Address (#8261)
* Add Sora network SS58 Address

Signed-off-by: Vladislav Markushin <negigic@gmail.com>

* Update Sora network SS58 Address to `69`

Signed-off-by: Vladislav Markushin <negigic@gmail.com>

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2021-04-14 09:16:12 +00:00
Alexander Theißen f854194139 contracts: Add RPC that allows instantiating of a contract (#8451)
* contracts: Add RPC that allows instantiating of a contract

* Encode `debug_message` as bytes because usage of `String` is forbidden

* Remove erroneous derive attribute

* Fix rpc tests for new `debug_message` encoding

* Fix typo

Co-authored-by: Andrew Jones <ascjones@gmail.com>

Co-authored-by: Andrew Jones <ascjones@gmail.com>
2021-04-13 11:26:52 +00:00
Guillaume Thiolliere a4ed9bb9b2 Ensure inherent are first (#8173)
* impl

* fix tests

* impl in execute_block

* fix tests

* add a test in frame-executive

* fix some panic warning

* use trait to get call from extrinsic

* remove unused

* fix test

* fix testing

* fix tests

* return index of extrinsic on error

* fix test

* Update primitives/inherents/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* address comments

rename trait, and refactor

* refactor + doc improvment

* fix tests

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2021-04-13 09:30:13 +00:00
Kian Paimani 373e3a4ddc clean arithmetic and unify names with the new api (#8581) 2021-04-09 16:36:06 +02:00
Xavier Lau d6f0ce0551 ModuleId to PalletId - part of #8372 (#8477)
* `ModuleId` to `PalletId` - part of #8372

* fix doc

* move `PalletId` to `frame-support`

* fix compile

* fix tests

* `ModuleId` to `PalletId`

* subcommand `moduleid` to `palletid`
2021-04-09 09:15:40 +00:00
Shawn Tabrizi c04b44b0a3 Add PoV Tracking to Benchmarking Pipeline (#8559)
* Added a function to estimate proof size for benchmarking

* integrate proof_size into benchmarking pipeline

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* fix tests

* one more test

* Update bench.rs

* Update utils/frame/benchmarking-cli/src/writer.rs

Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>

* Update utils/frame/benchmarking-cli/src/command.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2021-04-08 12:54:46 +02:00
Robert Habermeier a600e278ed Support code blobs compressed with zstd (#8549)
* begin maybe-compressed-blob

* fix build

* implement blob compression / decompression

* add some tests

* decode -> decompress

* decompress code if compressed

* make API of compresseed blob crate take limit as parameter

* use new API in sc-executro

* wasm-builder: compress wasm

* fix typo

* simplify

* address review

* fix wasm_project.rs

* Update primitives/maybe-compressed-blob/Cargo.toml

Co-authored-by: Andronik Ordian <write@reusable.software>

Co-authored-by: Andronik Ordian <write@reusable.software>
2021-04-07 20:44:45 +00:00
Liu-Cheng Xu 00432a5ab7 Add some trivial improvements to primitives runtime (#8528)
* Add some trivial improvements

* Finish primitives/runtime
2021-04-05 20:20:36 +02:00
Bastian Köcher 4e334f59d6 Use log::error! for bad mandatory (#8521)
To make the life of people easier ;)
2021-04-04 13:42:11 +02:00
Bastian Köcher 217c4be226 Make BlockImport and Verifier async (#8472)
* Make grandpa work

* Introduce `SharedData`

* Add test and fix bugs

* Switch to `SharedData`

* Make grandpa tests working

* More Babe work

* Make it async

* Fix fix

* Use `async_trait` in sc-consensus-slots

This makes the code a little bit easier to read and also expresses that
there can always only be one call at a time to `on_slot`.

* Make grandpa tests compile

* More Babe tests work

* Fix network test

* Start fixing service test

* Finish service-test

* Fix sc-consensus-aura

* Fix fix fix

* More fixes

* Make everything compile *yeah*

* Fix build when we have Rust 1.51

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/common/src/shared_data.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Update client/consensus/babe/src/tests.rs

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* Fix warning

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
2021-03-30 09:19:49 +00:00
Sukhveer Sanghera a946c3343e Add Social Network (#8065)
* Add Social Network

* Remove TNET

* Update ss58-registry.json

* Update ss58-registry.json

* Update ss58-registry.json

* Add back comment

Co-authored-by: everhusk <sanghera@everhusk.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Github Actions <action@github.com>
2021-03-29 17:18:29 +00:00
Peter Goodspeed-Niklaus c2dd5e21a4 make types within generate_solution_type macro explicit (#8447)
* make types within `generate_solution_type` macro explicit

Closes #8444.

Just changes the parsing logic for that macro; does not change any
emitted code. The associated types associated with the macro now
require explicit, keyword-style declaration.

**Old**:

```rust
sp_npos_elections::generate_solution_type!(
	#[compact]
	pub struct TestCompact::<VoterIndex, TargetIndex, PerU16>(16)
);
```

**New**:

```rust
sp_npos_elections::generate_solution_type!(
	#[compact]
	pub struct TestCompact::<VoterIndex = VoterIndex, CandidateIndex = TargetIndex, Accuracy = PerU16>(16)
);
```

* un-ignore doc-tests

* use new form in bin/node/runtime/

* rename CandidateIndex -> TargetIndex

* add tests demonstrating some potential compile failures
2021-03-28 08:21:06 +00:00
Gavin Wood ff5765eac3 Repot frame_support::traits; introduce some new currency stuff (#8435)
* Reservable, Transferrable Fungible(s), plus adapters.

* Repot into new dir

* Imbalances for Fungibles

* Repot and balanced fungible.

* Clean up names and bridge-over Imbalanced.

* Repot frame_support::trait. Finally.

* Make build.

* Docs

* Good errors

* Fix tests. Implement fungible::Inspect for Balances.

* Implement additional traits for Balances.

* Revert UI test "fixes"

* Fix UI error

* Fix UI test

* Fixes

* Update lock

* Grumbles

* Grumbles

* Fixes

Co-authored-by: Bastian Köcher <info@kchr.de>
2021-03-27 14:37:13 +01:00
Roman Borschel e1570b9ab4 Update to libp2p-0.36 (#8420)
* Update to libp2p-0.36

* Some more Cargo.lock updates.
2021-03-24 18:29:35 +00:00
Shaun Wang 4fd69de38e Implement PartialEq for DispatchError. (#8407) 2021-03-20 20:50:53 -04:00
Arkadiy Paronyan 4a0d6d9490 Storage chains: indexing, renewals and reference counting (#8265)
* Transaction indexing

* Tests and fixes

* Fixed a comment

* Style

* Build

* Style

* Apply suggestions from code review

Co-authored-by: cheme <emericchevalier.pro@gmail.com>

* Code review suggestions

* Add missing impl

* Apply suggestions from code review

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* impl JoinInput

* Don't store empty slices

* JoinInput operates on slices

Co-authored-by: cheme <emericchevalier.pro@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
2021-03-18 11:46:27 +00:00
Shaun Wang 2e5522444a Replace 'Module' with 'Pallet' in construct_runtime macro (#8372)
* Use 'Pallet' struct in construct_runtime.

* Fix genesis and metadata macro.

* Fix 'Pallet' type alias.

* Replace 'Module' with 'Pallet' for all construct_runtime use cases.

* Replace more deprecated 'Module' struct.

* Bring back AllModules and AllPalletsWithSystem type, but deprecate them.

* Replace deprecated 'Module' struct from merge master.

* Minor fix.

* Fix UI tests.

* Revert UI override in derive_no_bound.

* Fix more deprecated 'Module' use from master branch.

* Fix more deprecated 'Module' use from master branch.
2021-03-18 08:50:08 +00:00
Bastian Köcher 15e15e7d8e Aura and Slots refactoring (#8386)
* Make slot duration being exposed as `Duration` to the outside

* Some slot info love

* Add `build_aura_worker` utility function

* Copy copy copy
2021-03-17 23:25:58 +00:00
Jon Häggblad 0d6884b919 Storing multiple Justifications per block (#7640)
* primitives/runtime: initial changes on supporting multiple Justifications

* primitives/runtime: make Justifications strongly typed

* Encode/decode Justifications

* primitives/runtime: add Justification type

* backend: apply_finality and finalize_block takes a single Justification

* manual-seal: create engine id and let rpc take encoded justification

* backend: skeleton functions for appending justifications

* backend: initial implementation append_justification

Initial implementation of append_justification on the Backend trait, and also remove unused skeleton
functions for append_justificaton on Finaziler trait.
k

* backend: guard against duplicate consensus engine id

* client/db: add check for block finality

* client/api: add append_justification to in_mem db

* client/light: add no-op append_justification

* network: fix decode call for Justification

* network: only send a single Justification in BlockData

* network: minor comment update

* protocol: update field names to distinguish single justification

* client: further field renames to plural

* client: update function names to plural justifications

* client/db: upgrade existing database for new format

* network: remove dependency on grandpa crate

* db: fix check for finalized block

* grandpa: check for multiple grandpa justifications hwne importing

* backend: update Finalizer trait to take multiple Justifications

* db: remove debugging statements in migration code

* manual-seal: update note about engine id

* db: fix check for finalized block

* client: update variable name to reflect it is now plural

* grandpa: fix incorrect empty Justications in test

* primitives: make Justifications opaque to avoid being empty

* network: fix detecting empty Justification

* runtime: doc strings for Justifications functions

* runtime: add into_justifications

* primitives: check for duplicates in when adding to Justifications

* network/test: use real grandpa engine id in test

* client: fix reviewer comments

* primitives: rename Justifications::push to append

* backend: revert changes to Finalizer trait

* backend: revert mark_finalized

* backend: revert changes to finalize_block

* backend: revert finalized_blocks

* db: add a quick early return for performance

* client: minor reviewer comments

* service/test: use local ConsensusEngineId

* network: add link to issue for sending multiple Justifications

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* network: tweaks to review suggestions

* network: revert change to BlockData for backwards compatibility

* Apply suggestion from code review

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>

* Apply suggestions from code review

Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>

* primitives: update doc comment for Justifications

* client/db/upgrade: avoid grandpa crate dependency

* consensus: revert to single Justification for import_justification

* primitives: improve justifications docs

* style cleanups

* use and_then

* client: rename JUSTIFICATIONS db column

* network: revert to using FRNK in network-test

Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: André Silva <123550+andresilva@users.noreply.github.com>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
2021-03-17 21:18:16 +00:00
Peter Goodspeed-Niklaus 23b32e7543 Improve complexity of CompactAssignments::unique_targets (#8314)
* Improve complexity of CompactAssignments::unique_targets

Original implementation was O(n**2). Current impl is O(n log n).

Avoided the original proposed mitigation because it does not retain
the de-duplicating property present in the original implementation.
This implementation does a little more work, but retains that property.

* Explicitly choose sp_std Vec and BTreeSet

Ensures that the macro still works if someone uses it in a context
in which sp_std is not imported or is renamed.

* explicitly use sp_std vectors throughout compact macro
2021-03-17 08:13:33 +00:00
Gavin Wood 363db4f086 staking: Flexible generation of reward curve and associated tweaks (#8327)
* Initial abstraction

* Alter rest of APIs

* Fixes

* Some extra getters in Gilt pallet.

* Refactor Gilt to avoid u128 conversions

* Simplify and improve pow in per_things

* Add scalar division to per_things

* Renaming from_fraction -> from_float, drop _approximation

* Fixes

* Fixes

* Fixes

* Fixes

* Make stuff build

* Fixes

* Fixes

* Fixes

* Fixes

* Update .gitignore

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/gilt/src/lib.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Update frame/gilt/src/mock.rs

Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

* Fixes

* Fixes

* Fixes

Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
2021-03-16 12:03:58 +00:00