Introduces `CryptoBytes` type defined as:
```rust
pub struct CryptoBytes<const N: usize, Tag = ()>(pub [u8; N], PhantomData<fn() -> Tag>);
```
The type implements a bunch of methods and traits which are typically
expected from a byte array newtype
(NOTE: some of the methods and trait implementations IMO are a bit
redundant, but I decided to maintain them all to not change too much
stuff in this PR)
It also introduces two (generic) typical consumers of `CryptoBytes`:
`PublicBytes` and `SignatureBytes`.
```rust
pub struct PublicTag;
pub PublicBytes<const N: usize, CryptoTag> = CryptoBytes<N, (PublicTag, CryptoTag)>;
pub struct SignatureTag;
pub SignatureBytes<const N: usize, CryptoTag> = CryptoBytes<N, (SignatureTag, CryptoTag)>;
```
Both of them use a tag to differentiate the two types at a higher level.
Downstream specializations will further specialize using a dedicated
crypto tag. For example in ECDSA:
```rust
pub struct EcdsaTag;
pub type Public = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
pub type Signature = PublicBytes<PUBLIC_KEY_SERIALIZED_SIZE, EcdsaTag>;
```
Overall we have a cleaner and most importantly **consistent** code for
all the types involved
All these details are opaque to the end user which can use `Public` and
`Signature` for the cryptos as before
This is the significant step to make BEEFY client able to handle both
ECDSA and (ECDSA, BLS) type signature. The idea is having BEEFY Client
generic on crypto types makes migration to new types smoother.
This makes the BEEFY Keystore generic over AuthorityId and extends its
tests to cover the case when the AuthorityId is of type (ECDSA,
BLS12-377)
---------
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
Step towards https://github.com/paritytech/polkadot-sdk/issues/1975
As reported
https://github.com/paritytech/polkadot-sdk/issues/1975#issuecomment-1774534225
I'd like to encapsulate crypto related stuff in a dedicated folder.
Currently all cryptographic primitive wrappers are all sparsed in
`substrate/core` which contains "misc core" stuff.
To simplify the process, as the first step with this PR I propose to
move the cryptographic hashing there.
The `substrate/crypto` folder was already created to contains `ec-utils`
crate.
Notes:
- rename `sp-core-hashing` to `sp-crypto-hashing`
- rename `sp-core-hashing-proc-macro` to `sp-crypto-hashing-proc-macro`
- As the crates name is changed I took the freedom to restart fresh from
version 0.1.0 for both crates
---------
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
Serialized length is now statically computed depending on the domain
size.
Opens the primitive to more generic usages not related to Sassafras
expectations
Address one point of
https://github.com/paritytech/polkadot-sdk/issues/2364
Next step in process of making BEEFY being able to generate both ECDSA
and BLS signature after #1705. It allows BEEFY to use a pair of ECDSA
and BLS key as a AuthorityId.
---------
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
* Introduce bandersnatch vrf
* Some documentation
* Fix tests
* Fix docs refs
* Some more docs
* Comments about key derivation
* Make clippy happy
* Fix ring context enc/dec test
* Fix docs
* Switch to upstream ring-vrf
* Use sub-domains to construct VrfInput
* Bandersnatch VRF experimental feature
* Restore upstream dep
* Fix feature flags
* Apply typo fix
Co-authored-by: Anton <anton.kalyaev@gmail.com>
* Bump bandersnatch-vrfs
* Weiestrass form has been selected
* Rename bandersnatch testing app crypto id
* Support for seed recovery
* Clarified domain size <-> key size relationship
* cargo fmt
* Trigger CI
* Some required tweaks to crypto types
* Remove leftovers from Cargo.toml
* Remove some TODO notes
* Simplification of structs construction
* Trigger CI
* Apply review suggestion
Co-authored-by: Koute <koute@users.noreply.github.com>
* Docs typo
* Fix keystore tests
* Consistence
* Add ref to git rependency
* Static check of MAX_VRF_IOS value
* Clarify behavior for out of ring keys signatures
* Add test for ring-vrf to the keystore
* Fix docs
---------
Co-authored-by: Anton <anton.kalyaev@gmail.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
* Start
* More work!
* Moar
* More changes
* More fixes
* More worrk
* More fixes
* More fixes to make it compile
* Adds `NoOffchainStorage`
* Pass the extensions
* Small basti making small progress
* Fix merge errors and remove `ExecutionContext`
* Move registration of `ReadRuntimeVersionExt` to `ExecutionExtension`
Instead of registering `ReadRuntimeVersionExt` in `sp-state-machine` it is moved to
`ExecutionExtension` which provides the default extensions.
* Fix compilation
* Register the global extensions inside runtime api instance
* Fixes
* Fix `generate_initial_session_keys` by passing the keystore extension
* Fix the grandpa tests
* Fix more tests
* Fix more tests
* Don't set any heap pages if there isn't an override
* Fix small fallout
* FMT
* Fix tests
* More tests
* Offchain worker custom extensions
* More fixes
* Make offchain tx pool creation reusable
Introduces an `OffchainTransactionPoolFactory` for creating offchain transactions pools that can be
registered in the runtime externalities context. This factory will be required for a later pr to
make the creation of offchain transaction pools easier.
* Fixes
* Fixes
* Set offchain transaction pool in BABE before using it in the runtime
* Add the `offchain_tx_pool` to Grandpa as well
* Fix the nodes
* Print some error when using the old warnings
* Fix merge issues
* Fix compilation
* Rename `babe_link`
* Rename to `offchain_tx_pool_factory`
* Cleanup
* FMT
* Fix benchmark name
* Fix `try-runtime`
* Remove `--execution` CLI args
* Make clippy happy
* Forward bls functions
* Fix docs
* Update UI tests
* Update client/api/src/execution_extensions.rs
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
* Apply suggestions from code review
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update client/cli/src/params/import_params.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Update client/api/src/execution_extensions.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Pass the offchain storage to the MMR RPC
* Update client/api/src/execution_extensions.rs
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* Review comments
* Fixes
---------
Co-authored-by: Michal Kucharczyk <1728078+michalkucharczyk@users.noreply.github.com>
Co-authored-by: Koute <koute@users.noreply.github.com>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
* Cherry pick all crypto related changes from pull-request #13311
applied to master's head
* Import some stuff just if 'full_crypto' is on
* Remove copyright year
* Cleanup
* First generic BLS draft
* Finalize generic implementation
* Restore tests
* Fix rust docs
* Fix after master merge
* Fix after master merge
* Use double bls with G1 as signature group and verify individual signatures using DLEQ proof.
* Fix inclusions and types used within substrate
* Remove unused cruft
* Restore usage of upstream crates
* Fix test
* Reduce the diff by aligning Cargo.lock to master
* Application-crypto provides bls381
* Implement bls381 for local keystore
* Use new generic keystore features
* import DoublePublickey[Scheme] from the bls-like root to be less confusing.
* fix compilation
* Apply suggestions from code review
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
* Clean leftovers
* - update bls test vector after applying spec change recommendation.
- send message as ref.
* Different hard junction ids for different bls12 types
* update to new bls-like
* bls-like → w3f-bls
* Make clippy happy
* update test vector after replacing hash and crop with hash to field.
* cargo fmt
* account for #13972
* hide BLS behind "bls_non_production" feature flag
* Remove Cargo.lock entries duplicated in merge
* add bls377 to primitives/keystore and client/keystore
add bls377 to primitives/application-crypto/
add bls_non_production to primitives/keystore and client/keystore
bump up w3f-bls version
* rename feature `bls_non_production` to `bls-experimental`
---------
Co-authored-by: Davide Galassi <davxy@datawok.net>
Co-authored-by: André Silva <andrerfosilva@gmail.com>
Co-authored-by: Robert Hambrock <roberthambrock@gmail.com>
* Allow extra signing data
* Fix tests after renaming
* Rename VrfSecret/VrfVerifier to VrfSecret/VrfPublic
* Further encrapsulation of 'transcript' type to the sr25519 implementation
* Keystore sr25519 pre-output
* Leave additional custom input field hidden in the associated VrfInput type
* Fix test
* More ergonomic output_bytes
* Trigger pipeline
* Define a separated type for vrf signature data
* Fix docs
* Fix doc
* Remove annotation
* Directly use dleq_proove and dleq_verify in sr25519
* Trigger CI
* Remove cruft before merge
* First iteration to encapsulate schnorrkel and merlin usage
* Remove schnorkel direct dependency from BABE pallet
* Remove schnorrkel direct dependency from BABE client
* Trivial renaming for VrfTranscript data and value
* Better errors
* Expose a function to get a schnorrkel friendly transcript
* Keep the vrf signature stuff together (preventing some clones around)
* Fix tests
* Remove vrf agnostic transcript and define it as an associated type for VrfSigner and VrfVerifier
* Fix babe pallet mock
* Inner types are required to be public for polkadot
* Update client/consensus/babe/src/verification.rs
Co-authored-by: Koute <koute@users.noreply.github.com>
* Nit
* Remove Deref implementations
* make_bytes as a method
* Trigger CI
---------
Co-authored-by: Koute <koute@users.noreply.github.com>
* Introduce keystore specialized sign methods
* Get rid of 'AppKey::UntypedGeneric' associated type.
Untyped generics are accessible using associated types 'Generic' associated type.
I.e. <T as AppKey>::Public::Generic
* Get rid of 'CryptoTypePublicPair'
* Trivial fix
* Small refactory of local keystore implementations
* Remove 'crypto_id' method from 'Public'
* Trivial rename of 'AppKey' to 'AppCrypto'
* Remove unused import
* Improve docs
* Better signature related errors for authority-discovery
* Apply review suggestion
* Apply review suggestions
Co-authored-by: Koute <koute@users.noreply.github.com>
* Authority discoverty signing error revisited
* Signing error revisited for babe and aura as well
* Further cleanup
---------
Co-authored-by: Koute <koute@users.noreply.github.com>
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* Run cargo fmt on the whole code base
* Second run
* Add CI check
* Fix compilation
* More unnecessary braces
* Handle weights
* Use --all
* Use correct attributes...
* Fix UI tests
* AHHHHHHHHH
* 🤦
* Docs
* Fix compilation
* 🤷
* Please stop
* 🤦 x 2
* More
* make rustfmt.toml consistent with polkadot
Co-authored-by: André Silva <andrerfosilva@gmail.com>
* update to latest master
* updates on docs, license, meta
* hide ssrs behind feature flag
* implement remaining functions on the server
* sign server line length fix
* fix tests
* fixup in-memory-keystore
* adding failsafe
* skipping ecdsa test for now
* remote keystore param
* remote sign urls made available
* integrating keystore remotes features
* don't forget the dependency
* remove old cruft
* reset local keystore
* applying suggestions
* Switch to single remote, minor grumbles
* minor grumbles, docs
* Asyncify sign_with
* Asyncify generate/get keys
* Complete BareCryptoStore asyncification
* Cleanup
* Rebase
* Add Proxy
* Inject keystore proxy into extensions
* Implement some methods
* Await on send
* Cleanup
* Send result over the oneshot channel sender
* Process one future at a time
* Fix cargo stuff
* Asyncify sr25519_vrf_sign
* Cherry-pick and fix changes
* Introduce SyncCryptoStore
* SQUASH ME WITH THE first commit
* Implement into SyncCryptoStore
* Implement BareCryptoStore for KeystoreProxyAdapter
* authority-discovery
* AURA
* BABE
* finality-grandpa
* offchain-workers
* benchmarking-cli
* sp_io
* test-utils
* application-crypto
* Extensions and RPC
* Client Service
* bin
* Update cargo.lock
* Implement BareCryptoStore on proxy directly
* Simplify proxy setup
* Fix authority-discover
* Pass async keystore to authority-discovery
* Fix tests
* Use async keystore in authority-discovery
* Rename BareCryptoStore to CryptoStore
* WIP
* Remote mutable borrow in CryptoStore trait
* Implement Keystore with backends
* Remove Proxy implementation
* Fix service builder and keystore user-crates
* Fix tests
* Rework authority-discovery after refactoring
* futures::select!
* Fix multiple mut borrows in authority-discovery
* Merge fixes
* Require sync
* Restore Cargo.lock
* PR feedback - round 1
* Remove Keystore and use LocalKeystore directly
Also renamed KeystoreParams to KeystoreContainer
* Join
* Remove sync requirement
* Fix keystore tests
* Fix tests
* client/authority-discovery: Remove event stream dynamic dispatching
With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.
Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.
* Make it compile
* Fix submit_transaction
* Fix block_on issue
* Use await in async context
* Fix manual seal keystore
* Fix authoring_blocks test
* fix aura authoring_blocks
* Try to fix tests for auth-discovery
* client/authority-discovery: Fix lookup_throttling test
* client/authority-discovery: Fix triggers_dht_get_query test
* Fix epoch_authorship_works
* client/authority-discovery: Remove timing assumption in unit test
* client/authority-discovery: Revert changes to termination test
* PR feedback
* Remove deadcode and mark test code
* Fix test_sync
* Use the correct keyring type
* Return when from_service stream is closed
* Convert SyncCryptoStore to a trait
* Fix line width
* Fix line width - take 2
* Remove unused import
* Fix keystore instantiation
* PR feedback
* Remove KeystoreContainer
* Revert "Remove KeystoreContainer"
This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.
* Take a ref of keystore
* Move keystore to dev-dependencies
* Address some PR feedback
* Missed one
* Pass keystore reference - take 2
* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX
Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:
1. No additional mental overhead of a `SyncCryptoStorePtr`.
2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.
Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.
* Remove SyncCryptoStorePtr
* Remove KeystoreContainer & SyncCryptoStorePtr
* PR feedback
* *: Use CryptoStorePtr whereever possible
* *: Define SyncCryptoStore as a pure extension trait of CryptoStore
* Follow up to SyncCryptoStore extension trait
* Adjust docs for SyncCryptoStore as Ben suggested
* Cleanup unnecessary requirements
* sp-keystore
* Use async_std::task::block_on in keystore
* Fix block_on std requirement
* Update primitives/keystore/src/lib.rs
Co-authored-by: Max Inden <mail@max-inden.de>
* Fix wasm build
* Remove unused var
* Fix wasm compilation - take 2
* Revert async-std in keystore
* Fix indent
* Fix version and copyright
* Cleanup feature = "std"
* Auth Discovery: Ignore if from_service is cloed
* Max's suggestion
* Revert async-std usage for block_on
* Address PR feedback
* Fix example offchain worker build
* Address PR feedback
* Update Cargo.lock
* Move unused methods to test helper functions
* Restore accidentally deleted cargo.lock files
* Fix unused imports
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
Davide Galassi