Commit Graph

14 Commits

Author SHA1 Message Date
Bigna Härdi 0e49ed72aa Add serde feature flag to primitives (#13027)
* add serde_full feature flag

add serde_full to sp_runtime

add space to toml

add serde_full to application-crypto

add serde_full to arithmetic

fix arithmetic

add serde full to beefy

add serde full to consensus

add serde_full to core

add serdefull to finality grandpa

add serde_full to several primitives crates

make rpc no_std compatible

add scale info to runtime

make serializer no_std compatible

add serde full to storage

add full serde to version

add serde full to weights

add all serde_full features

add . to comment

add missing impl-serde

fix no-std build

fix build

add full_crypto to serde_full

serde_full also implements crypto

full_serde does not work with full_crytpo. needs std

no no_std serde impl possible

also for crypto std is necessary

no serde full for application crypto

fix arithmetic

fix tomls

fix some things

impl fmt for Signature

add serialize to Public

add impl_maybe_marker_serde_full

fix sp-application-crypto toml

add serde feature flag

fix clippy

fix toml grandpa

fix grandpa

rename if_std to if_serde

keystore is not no_std compatible

make keystore vrf no_std compatible

fix nopos-elections

fix rpc

fix serializer

fix test-primitives

fix version

add comment

add serde full only import for format string

remove all(serde_full and full_crypot) as serde_full enforces full_crypto

make comment better readable

even better comment

clean up rpc toml

clean up toml

clean up serializer toml

clean up storage toml

fix std build

update .lock

fix sp-version

move sp_std import

test extern crate alloc

replace sp_std with core

add missing core

sp_core: serde feature do not enforce full crypto

application-crypto: serde feature do not enforce full crypto

rename serde_full to serde

add dep:serde and alloc to default feature

add full_crypto and remove unnecessary debu/fmt impls for serde

update comment

remove obolsete change in display AccountId32

remove extra changes

minimize diff

revert keystore changes

remove std from keystore

remove full-crypto feature

fix serde import

fix comment

fix feature = serde

* rename serde_full to serde

* move #[doc(hidden)] back

* remove feature = full crypto require frm MultiSigner

* reorder serde and scale_info import

* fix bs58 missing alloc import in serde feature

* add `from_string` to serde feature and add unimplemented

* remove serde feature from fixed_point display

* Remove serde/alloc

Co-authored-by: Davide Galassi <davxy@datawok.net>

* Update primitives/consensus/babe/Cargo.toml

Co-authored-by: Bastian Köcher <git@kchr.de>

* Update primitives/arithmetic/src/fixed_point.rs

Co-authored-by: Bastian Köcher <git@kchr.de>

* revert `from_string`fixed impl back to std only

* remove duplicate runtime string impl

* use sp_std::alloc

* remove no_std compatible rpc

* remove no_std compatibility from serializer

* rename mpl_maybe_marker_serde to std_or_serde

* update .lock

* add sp-std to executor

* fix sp-std import

* fix sp_std::format import

* use crate import

* add serde feature

* Update primitives/core/src/lib.rs

---------

Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Bastian Köcher <git@kchr.de>
2023-05-17 11:31:12 +00:00
Robert Hambrock 40d6b65a0c bump API versions of {Beefy,Mmr}Api (#13509) 2023-03-03 10:23:15 +00:00
Vivek Pandya bc53b9a03a Remove years from copyright notes. (#13415)
* Change copyright year to 2023 from 2022

* Fix incorrect update of copyright year

* Remove years from copy right header

* Fix remaining files

* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
Alexandru Vasile 4b2cc67fc6 Metadata V15: Derive TypeInfo for describing runtime types (#13272)
* scale_info: Derive TypeInfo for types present in runtime API

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

* cargo: Update Cargo.lock

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>

---------

Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
2023-02-02 19:14:40 +02:00
Serban Iorga ff439ee335 pallet-mmr: move offchain logic to client-side gadget (#12753)
* Move MMR utils methods from pallet to primitives

Signed-off-by: Serban Iorga <serban@parity.io>

* Add method to MmrApi

* Move forks expanding logic from babe to primitives

* Implement MMR gadget

* Remove prunning logic from the MMR pallet

* Code review changes: 1st iteration

* Replace MaybeCanonEngine with CanonEngineBuilder

* fix mmr_leaves_count() for kitchen sink demo

* Update client/merkle-mountain-range/src/canon_engine.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Code review changes: 2nd iteration

* fix INDEXING_PREFIX

* impl review comments

* add documentation and minor rename

Signed-off-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
2022-11-29 14:39:52 +00:00
Robert Hambrock 53f9735058 Consolidate and deduplicate MMR API methods (#12530)
* histor. batch proof: make best block arg optional

* correct testing range

* make generate_batch_proof stub for historical

* merge generate_{historical_}batch_proof functions

* merge generate_{batch_}proof functions

* merge verify_{batch_}proof functions

* merge verify_{batch_}proof_stateless functions

* remove {Leaf}Proof

Not utilized by API anymore, so superfluous.
Removal consistent with prior changes to just use "batch" proof API.

* rename BatchProof->Proof

no need to qualify if only one universal proof type.

* cleanup

* expose verify_proof rpc api

* document verify_proof

* expose verify_proof_stateless rpc api

* add optional BlockHash to mmr_root rpc api

* fixup! expose verify_proof rpc api

* fix documentation phrasing

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* documentation grammar

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* define mmr error msgs together with error enum

Co-authored-by: Serban Iorga <serban@parity.io>

* fixup! define mmr error msgs together with error enum

* map decoding errors to CallError::InvalidParams

Co-authored-by: Serban Iorga <serban@parity.io>

* fixup! map decoding errors to CallError::InvalidParams

Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: parity-processbot <>
Co-authored-by: Serban Iorga <serban@parity.io>
2022-11-09 14:41:32 +00:00
Sergej Sakac 9dd89e7fa3 pallet-mmr: RPC API and Runtime API work with block numbers (#12345)
* pallet-mmr: RPC API works with block_numbers

* fixes

* update rpc

* fmt

* final touches in the rpc

* temporary fix

* fix

* fmt

* docs

* Update lib.rs

* use NumberFor

* validate input

* update runtime

* convert block_number to u64

* small edit

* update runtime api

* test fix

* runtime fix

* update test function

* fmt

* fix nits

* remove block_num_to_leaf_index from runtime api

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>

* fix tests

* get the code to compile after merge

* get the tests to compile

* fix in tests?

* fix test

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Update primitives/merkle-mountain-range/src/lib.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* fix errors & nits

* change block_num_to_leaf_index

* don't make any assumptions

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Update frame/merkle-mountain-range/src/tests.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* fix

* small fix

* use best_known_block_number

* best_known_block_number instead of leaves_count

* more readable?

* remove warning

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>

* simplify

* update docs

* nits

* fmt & fix

* merge fixes

* fix

* small fix

* docs & nit fixes

* Nit fixes

* remove leaf_indices_to_block_numbers()

* fmt

Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
2022-10-13 10:13:56 +00:00
Serban Iorga 6f9ae78d30 MMR: impl TypeInfo for some structures (#12423)
* BEEFY client: avoid unnecessary clone

* MMR: impl TypeInfo for some structures
2022-10-05 17:11:50 +03:00
Serban Iorga 54713ca17a pallet-mmr: generate historical proofs (#12324)
* BEEFY: generate historical proofs

Signed-off-by: Serban Iorga <serban@parity.io>

* Update frame/merkle-mountain-range/rpc/src/lib.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Update primitives/merkle-mountain-range/src/lib.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* Update frame/merkle-mountain-range/src/lib.rs

Co-authored-by: Adrian Catangiu <adrian@parity.io>

* cargo fmt

* fix off-by-one in leaves powerset generation

* test all possible mmr sizes for historical proofs

* remove now redundant simple_historical_proof

* cargo fmt

Signed-off-by: Serban Iorga <serban@parity.io>
Co-authored-by: Adrian Catangiu <adrian@parity.io>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
2022-09-30 12:46:48 +02:00
Xavier Lau 86198c5471 Use array-bytes for All Array/Bytes/Hex Operations (#12190)
* Use `array-bytes` for All Array/Bytes/Hex Operations

Signed-off-by: Xavier Lau <xavier@inv.cafe>

* Reorder

* Self Review

* Format

* Fix Tests

* Bump `array-bytes`

* Optimize large test res

Signed-off-by: Xavier Lau <xavier@inv.cafe>
Co-authored-by: parity-processbot <>
2022-09-21 06:12:20 +00:00
Adrian Catangiu 27b538222e pallet-mmr: handle forks without collisions in offchain storage (#11594)
* pallet-mmr: fix some typos

* pallet-mmr: make the MMR resilient to chain forks

* pallet-mmr: get hash for block that added node

* beefy-mmr: add debug logging

* add explanatory comment

* account for block offset of pallet activation

* add support for finding all nodes added by leaf

* minor improvements

* add helper to return all nodes added to mmr with a leaf append

* simplify leaf_node_index_to_leaf_index

summing the (shifted) differences in peak positions adds up to the (shifted) final position, so
don't need to fold over positions.

* dead fish: this also doesn't work

The idea was to keep a rolling window of `(parent_hash, pos)` leaf
entries in the offchain db, with the window matching the one
that provides `block_num -> block_hash` mappings in `frame_system`.

Once a leaf exits the window it would be "canonicalized" by switching
its offchain db key from `(parent_hash, pos)` to simple `pos`.

This doesn't work however because there's no way to get leaf contents
from offchain db while in runtime context.. so no way to get+clear+set
leaf to change its key in offchain db.

Ideas:
1. move the "canonicalization" logic to offchain worker
2. enhance IndexingApi with "offchain::move(old_key, new_key)"
   This is weird, but correct, deterministic and safe AFAICT, so
   it could be exposed to runtime.

* simplify rightmost_leaf_node_index_from_pos

* minor fix

* move leaf canonicalization to offchain worker

* move storage related code to storage.rs

* on offchain reads use canonic key for old leaves

* fix offchain worker write using canon key

* fix pallet-mmr tests

* add documentation and fix logging

* add offchain mmr canonicalization test

* test canon + generate + verify

* fix pallet-beefy-mmr tests

* implement review suggestions

* improve test

* pallet-mmr: add offchain pruning of forks

* pallet-mmr: improve offchain pruning

Instead of keeping pruning map as single blob in offchain db,
keep individual parent-hash lists with block-num identifier as part
of the offchain key.

Signed-off-by: acatangiu <adrian@parity.io>

* pallet-mmr: improve MMRStore<OffchainStorage>::get()

Do the math and retrieve node using correct (canon or non-canon)
offchain db key, instead of blindly looking in both canon and non-canon
offchain db locations for each node.

Still fallback on looking at both if for any reason it's not where
expected.

Signed-off-by: acatangiu <adrian@parity.io>

* pallet-mmr: storage: improve logs

* fix tests: correctly persist overlay

runtime indexing API works on overlay, whereas offchain context
bypasses overlay, so for loops > canon-window, canon would fail.

* pallet-mmr: fix numeric typo in test

* add comment around LeafData requirements

Signed-off-by: acatangiu <adrian@parity.io>

Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
2022-07-07 15:02:37 +00:00
Web3 Smith fd45676d11 Update MMR Runtime API with functionality to generate MMR proof for a series of leaf indices (#10635)
* updated mmr rpc api with functions for batch generation of proof

* update code comments

* fix build errors

* added tests to mmr-rpc

* add tests to pallet-mmr

* update comments

* minor comment fix

* remove unused variables

* fix rust doc errors

* refactor mmr runtime api

* fix tests

* minor fix

* minor fix

* fix node-runtime

* revert to initial api

* impl from proof fot batchproof

* minor fix

* minor fix

* use explicit functions to convert btw batch proof and single proof

* minor fix

* add new variant to mmr error

* fmt

* update conversion to single leaf proof

* fix style nit

Co-authored-by: Adrian Catangiu <adrian@parity.io>
2022-05-04 10:40:11 +00:00
Falco Hirschenberger b581604aa7 Apply some clippy lints (#11154)
* Apply some clippy hints

* Revert clippy ci changes

* Update client/cli/src/commands/generate.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/cli/src/commands/inspect_key.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/db/src/bench.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/service/src/client/block_rules.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/transactions.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/network/src/protocol.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Revert due to missing `or_default` function.

* Fix compilation and simplify code

* Undo change that corrupts benchmark.

* fix clippy

* Update client/service/test/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update client/state-db/src/noncanonical.rs

remove leftovers!

* Update client/tracing/src/logging/directives.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/fork-tree/src/lib.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* added needed ref

* Update frame/referenda/src/benchmarking.rs

* Simplify byte-vec creation

* let's just not overlap the ranges

* Correction

* cargo fmt

* Update utils/frame/benchmarking-cli/src/shared/stats.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Update utils/frame/benchmarking-cli/src/pallet/command.rs

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Giles Cope <gilescope@gmail.com>
2022-04-30 21:28:27 +00:00
Adrian Catangiu 80ce5c5768 Expose MMR root through runtime API - use it in BEEFY client (#11183)
* beefy-gadget: allow custom runtime api provider

* beefy-gadget: use mock runtime api in tests

* pallet-mmr: expose mmr root from state through runtime API

* beefy-gadget: get mmr root from runtime state

* pallet-beefy-mmr: remove MmrRoot from header digests

* frame/mmr: move mmr primitives out of frame

* frame/mmr: completely move primitives out of frame

* address review comments

* beefy-mmr: bring back mmr root from header digest

* clippy fixes for rustc 1.60

* address review comments
2022-04-13 10:13:06 +00:00