* Add pallet template from Substrate Dev Hub
* Clean up un-needed stuff from template
* Sketch out dispatchable interface
* Introduce notion of finality chain
* Add dependencies which were removed during a rebase
* Sketch out idea for finality header-chain pallet
* Sketch out ChainVerifier trait
* Add storage parameter to verifier
* Write out some things I think I need for finality verification
* Add some pseudocode for marking finalized headers
* Remove parity_scale_codec duplicate
* Move verification logic into pallet
I've been struggling with getting the generic types between the storage and verifier
traits to play nice with each other. As a way to continue making progress I'm moving
everything to the pallet. This way I hope to make progress towards a functional
pallet.
* Start doing verification around authority set changes
* Remove commented BridgeStorage and ChainVerifier traits
* Create Substrate bridge primitives crate
* Add logic for updating scheduled authority sets
* Introduce notion of imported headers
* Implement basic header ancestry checker
* Add mock runtime for tests
* Add testing boilerplate
* Add some storage read/write sanity tests
* Add some basic header import tests
* Add tests for ancestry proofs
* Create helper for changing authority sets
* Fix authority set test
Fixes a problem with how the scheduled change was counted as well as
a SCALE encoding issue
* Correctly check for scheduled change digests
There's no guarantee that the consensus digest item will be the last
one in a header, which is how it was previously being checked.
Thanks to Andre for pointing me to the Grandpa code that does this.
* Mark imported headers as finalized when appropriate
When a header that finalizes a chain of headers is succesfully imported
we also want to mark its ancestors as finalized.
* Add helper for writing test headers
* Add test helper for scheduling authority set changes
* Bump Substrate pallet and primitives to rc6
* Remove Millau verifier implementation
* Add some doc comments
* Remove some needless returns
* Make Clippy happy
* Split block import from finalization
* Make tests compile again
* Add test for finalizing header after importing children
* Create a test stub for importing future justifications
* Start adding genesis config
* Reject justifications from future
We should only be accepting justifications for the header
which enacted the current authority set. Any ancestors of
that header which require a justification can be imported
but they must not be finalized.
* Add explanation to some `expect()` calls
* Start adding GenesisConfig
* Plug genesis config into runtime
* Remove tests module
* Check for overflow when updating authority sets
* Make verifier take ownership of headers during import
* Only store best finalized header hash
Removed the need to store the whole header, since we store
it was part of the ImportedHeaders structure anyways
* Add some helpers to ImportedHeader
* Update ancestry checker to work with ImportedHeaders
* Update ancestry tests to use ImportedHeaders
* Update import tests to use ImportedHeaders
* Clean up some of the test helpers
* Remove stray dbg!
* Add doc comments throughout
* Remove runtime related code
* Fix Clippy warnings
* Remove trait bound on ImportedHeader struct
* Simplify checks in GenesisConfig
* Rename `get_header_by_hash()`
* Alias `parity_scale_codec` to `codec`
* Reword Verifier documentation
* Missed codec rename in tests
* Split ImportError into FinalizationError
* Remove ChainVerifier trait
This trait was a remenant of the original design, and it is not required
at the moment. Something like it should be added back in the future to
ensure that other chains which conform to this interface can be used
by higher-level bridge applications.
* Fix the verifier tests so they compile
* Implement Deref for ImportedHeader
* Get rid of `new` methods for some Substrate primitives
* Ensure that a child header's number follows its parent's
* Prevent ancestry checker from aimlessly traversing to genesis
If an ancestor which was newer than the child header we were checking we
would walk all the way to genesis before realizing that we weren't related.
This commit fixes that.
* Remove redundant clones
* Ensure that old headers are not finalized
Prevents a panic where if the header being imported and `best_finalized`
were the same header the ancestry checker would return an empty list. We
had made an assumption that the list would always be populated, and if this
didn't hold we would end up panicking.
* Disallow imports at same height as `best_finalized`
* Fix Clippy warnings
* Make NextScheduledChange optional
* Rework how scheduled authority set changes are enacted
We now require a justification for headers which _enact_ changes
instead of those which _schedule_ changes. A few changes had to
be made to accomodate this, such as changing when we check for
scheduled change logs in incoming headers.
* Update documentation for Substrate Primitives
* Clarify why we skip header in requires_justification check
* Add description to assert! call
* Fix formatting within macros
* Remove unused dependencies from runtime
* Remove expect call in GenesisConfig
* Turn FinalityProof into a struct
* Add some inline TODOs for follow up PRs
* Remove test which enacted multiple changes
This should be added back at some later point in time, but right now
the code doesn't allow for this behaviour.
* Use `contains_key` when checking for header
This is better than using `get().is_some()` since we skip
decoding the storage value
* Use initial hash when updating best_finalized
* Add better checks around enacting scheduled changes
* Rename finality related functions
* Appease Clippy
* Rename Header to AuraHeader
This prevents some type conflicts with the PolkadotJS Apps types.
* Fix test and benchmark builds
* Update AuraHeader in types.json
* Add header-chain primitive crate
* Make MinimalHeaderChain functionaly the same as PeerBlockchain
* Use a better doc comment for MinimalHeaderChain
* Fix benchmark compilation
* Rust Fmt
* Remove Substrate based dependencies
* Rename MinimalHeaderChain to BaseHeaderChain
* Rename Ethereum PoA primitives crate
The "sp" prefix comes from Substrate primitives, since this crate originated
in that repo. However, it is not part of Substrate anymore and its name should
be updated to reflect that.
* Rename currency exchange primitives
* Rust Fmt
* Update import in benchmarking module
* Rust Fmt
* Split pub and no-pub
* Sort toml files.
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
* Bump Substrate to rc5
* Bump async-std to v1.6.2
There was a bug in v.1.6.0 which kept us locked to v1.5 releases.
I think that's fixed now so I'm bumping this.
* Update bridge node runtime
* Update node service
* Update CLI
* Add SystemWeightInfo type to test runtimes
* Add RPC extension builder to service
* Directly return rpc_extensions_builder
* Allow complex types in service
This comes from Substrate, so I'd rather just keep the code as is
* Update benchmarking code for new CLI
* Add Instance type parameter to pallet
* Sketch out what the runtime could look like
* Allow runtime to compile with multiple bridge pallets
* Cargo Fmt
* Allow an instance of a PoA chain to be used with currency-exchange
I specify that it's only _an instance_ instead of _instances_ since the currency-exchange
pallet does not support multiple instances itself. What this commit does is make it so
that the different instances of the PoA chains we currently have are compatible with the
currency-exchange pallet through the implementation of the PeerBlockchain trait.
* Add Instance type parameter to Currency Exchange pallet
* Wire up currency exchange intances in runtime
* Rust Fmt
* Show sccache
* Allow Eth pallet to use a default instance
* Use a default instance in Eth pallet tests
* Remove Rialto and Kovan feature flags
Through some discussions it has been decided that the `bridge-node` should, like
Substrate's `node-template`, be a showcase of the different pallets available in
a project. Because of this I've removed the feature flags for the Rialto and Kovan
networks in favour of having both of them included in the runtime.
* Update the chain_spec to use both Rialto and Kovan configs
* Update pallet level calls used by Substrate client
Allows the project to compile. However, it should be noted that in reality
we shouldn't be hardcoding the pallet we're calling.
* Allow currency-exchange pallet to use a default instance
* Support benchmarking an instance of the Eth pallet
* Update currency exchange benchmarks to work with instances
* Fix test helpers which now need a PoA instance
* Remove Actions for checking Rialto and Kovan features
* Add missing comments
* Update Runtime API string constants
* Add issue number for generic chain support in relay
* Add Runtime APIs for instances of the currency-exchange pallet
* Rust Fmt
Co-authored-by: Denis S. Soldatov aka General-Beck <general.beck@gmail.com>
* Bump Substrate to rc4
* Add BaseCallFilter type
* Add DenyUnsafe to SystemApi extension
* Use new ServiceBuilder build functions
* Add BaseCallFilter to test runtimes
* Remove old comments
* Add `rev` and `git` fields back
Turns out that if you don't have these future release candidates will
be used if available. For instance, once `rc5` is released a fresh pull
would use that instead of `rc4` which is what we want.
* Use tag release instead of specific commit
Will make scripted updates easier in the future
* Add short script to update between `rc` versions
* Update scripts/update_rc.sh
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Add skeleton for worst case import_unsigned_header
* Fix a typo
* Add benchmark test for best case unsigned header import
* Add finality verification to worst case bench
* Move `insert_header()` from mock to test_utils
Allows the benchmarking code to use this without having to pull it in from the mock.
* Add a rough bench to test a finalizing a "long" chain
* Try to use complexity parameter for finality bench
* Improve long finality bench
* Remove stray dot file
* Remove old "worst" case bench
* Scribble some ideas down for pruning bench
* Prune headers during benchmarking
* Clean up some comments
* Make finality bench work for entire range of complexity parameter
* Place initialization code into a function
* Add bench for block finalization with caching
* First attempt at bench with receipts
* Try and trigger validator set change
* Perform a validator set change during benchmarking
* Move `validators_change_receipt()` to shared location
Allows unit tests and benchmarks to access the same helper function
and const
* Extract a test receipt root into a constant
* Clean up description of pruning bench
* Fix cache and pruning tests
* Remove unecessary `build_custom_header` usage
* Get rid of warnings
* Remove code duplication comment
I don't think its entirely worth it to split out so few lines of code.
The benches aren't particularly hard to read anyways.
* Increase the range of the complexity parameter
* Use dynamic number of receipts while benchmarking
As part of this change we have removed the hardcoded TEST_RECEIPT_ROOT
and instead chose to calculate the receipt root on the fly. This will
make tests and benches less fragile.
* Prune a dynamic number of headers
* runtime benchmarks: start
* merge tests + benchmarks infrastructure
* fix compilation
* Fix compilation issues with runtime-benchmark feature flag
Mainly involved pulling in correct dependencies and adding some functions
which were called but didn't yet exist.
* Fix broken compilation for tests
* Move header signing methods into trait
* Move signing related test helpers to own module
* Remove comment about feature flag
* Add constants to tests
* Add top level comment for testing utilities
Co-authored-by: Hernando Castano <castano.ha@gmail.com>
* Update dependencies
Upgrades Substrate based dependencies from v2.0.0 -> v2.0.0-alpha.1
and uses the `jsonrpsee`'s new feature flags. The actual code hasn't
been updated though, so this won't compile.
* Use `RawClient`s from `jsonrpsee`
* Update to use jsonrpsee's new API
* Hook up Ethereum Bridge Runtime, Relay, and Node Runtime
* Bump `parity-crypto` from v0.4 to v0.6
Fixes error when trying to compile tests. This was caused by
`parity-crypto` v0.4's use of `parity-secp256k1` over `secp256k1'.
Using the Parity fork meant multiple version of the same underlying
C library were being pulled in. `parity-crypto` v0.6 moved away from
this, only relying on `secp256k1` thus fixing the issue.
* Copy node-template over from Substrate repo
Got the template at rev=6e6d06c33911
* Use dependencies from crates.io + stop renaming on import
* Remove template pallet
* Stop using crates.io dependencies
Instead they're going to be pinned at v2.0.0-alpha.2
at commit `2afecf81ee19b8a6edb364b419190ea47c4a4a31`
until something stable comes along.
* Remove LICENSE
* Change references of `node-template` to `bridge-node`
* Remove README
* Fix some missed node-template references
* Add WASM toolchain to CI
* Be more specific about nightly version to use
* Maybe don't tie to a specific nightly
* Use composite accounts
* Update to use lazy reaping
* Only use Development chain config
# Description
- What does this PR do?
1. Upgrades `trie-db`'s version to the latest release. This release
includes, among others, an implementation of `DoubleEndedIterator` for
the `TrieDB` struct, allowing to iterate both backwards and forwards
within the leaves of a trie.
2. Upgrades `trie-bench` to `0.39.0` for compatibility.
3. Upgrades `criterion` to `0.5.1` for compatibility.
- Why are these changes needed?
Besides keeping up with the upgrade of `trie-db`, this specifically adds
the functionality of iterating back on the leafs of a trie, with
`sp-trie`. In a project we're currently working on, this comes very
handy to verify a Merkle proof that is the response to a challenge. The
challenge is a random hash that (most likely) will not be an existing
leaf in the trie. So the challenged user, has to provide a Merkle proof
of the previous and next existing leafs in the trie, that surround the
random challenged hash.
Without having DoubleEnded iterators, we're forced to iterate until we
find the first existing leaf, like so:
```rust
// ************* VERIFIER (RUNTIME) *************
// Verify proof. This generates a partial trie based on the proof and
// checks that the root hash matches the `expected_root`.
let (memdb, root) = proof.to_memory_db(Some(&root)).unwrap();
let trie = TrieDBBuilder::<LayoutV1<RefHasher>>::new(&memdb, &root).build();
// Print all leaf node keys and values.
println!("\nPrinting leaf nodes of partial tree...");
for key in trie.key_iter().unwrap() {
if key.is_ok() {
println!("Leaf node key: {:?}", key.clone().unwrap());
let val = trie.get(&key.unwrap());
if val.is_ok() {
println!("Leaf node value: {:?}", val.unwrap());
} else {
println!("Leaf node value: None");
}
}
}
println!("RECONSTRUCTED TRIE {:#?}", trie);
// Create an iterator over the leaf nodes.
let mut iter = trie.iter().unwrap();
// First element with a value should be the previous existing leaf to the challenged hash.
let mut prev_key = None;
for element in &mut iter {
if element.is_ok() {
let (key, _) = element.unwrap();
prev_key = Some(key);
break;
}
}
assert!(prev_key.is_some());
// Since hashes are `Vec<u8>` ordered in big-endian, we can compare them directly.
assert!(prev_key.unwrap() <= challenge_hash.to_vec());
// The next element should exist (meaning there is no other existing leaf between the
// previous and next leaf) and it should be greater than the challenged hash.
let next_key = iter.next().unwrap().unwrap().0;
assert!(next_key >= challenge_hash.to_vec());
```
With DoubleEnded iterators, we can avoid that, like this:
```rust
// ************* VERIFIER (RUNTIME) *************
// Verify proof. This generates a partial trie based on the proof and
// checks that the root hash matches the `expected_root`.
let (memdb, root) = proof.to_memory_db(Some(&root)).unwrap();
let trie = TrieDBBuilder::<LayoutV1<RefHasher>>::new(&memdb, &root).build();
// Print all leaf node keys and values.
println!("\nPrinting leaf nodes of partial tree...");
for key in trie.key_iter().unwrap() {
if key.is_ok() {
println!("Leaf node key: {:?}", key.clone().unwrap());
let val = trie.get(&key.unwrap());
if val.is_ok() {
println!("Leaf node value: {:?}", val.unwrap());
} else {
println!("Leaf node value: None");
}
}
}
// println!("RECONSTRUCTED TRIE {:#?}", trie);
println!("\nChallenged key: {:?}", challenge_hash);
// Create an iterator over the leaf nodes.
let mut double_ended_iter = trie.into_double_ended_iter().unwrap();
// First element with a value should be the previous existing leaf to the challenged hash.
double_ended_iter.seek(&challenge_hash.to_vec()).unwrap();
let next_key = double_ended_iter.next_back().unwrap().unwrap().0;
let prev_key = double_ended_iter.next_back().unwrap().unwrap().0;
// Since hashes are `Vec<u8>` ordered in big-endian, we can compare them directly.
println!("Prev key: {:?}", prev_key);
assert!(prev_key <= challenge_hash.to_vec());
println!("Next key: {:?}", next_key);
assert!(next_key >= challenge_hash.to_vec());
```
- How were these changes implemented and what do they affect?
All that is needed for this functionality to be exposed is changing the
version number of `trie-db` in all the `Cargo.toml`s applicable, and
re-exporting some additional structs from `trie-db` in `sp-trie`.
---------
Co-authored-by: Bastian Köcher <git@kchr.de>
**Update:** Pushed additional changes based on the review comments.
**This pull request fixes various spelling mistakes in this
repository.**
Most of the changes are contained in the first **3** commits:
- `Fix spelling mistakes in comments and docs`
- `Fix spelling mistakes in test names`
- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`
Other source code spelling mistakes are separated into individual
commits for easier reviewing:
- `Fix the spelling of 'authority'`
- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`
- `Fix the spelling of 'prev_enqueud_messages'`
- `Fix the spelling of 'endpoint'`
- `Fix the spelling of 'children'`
- `Fix the spelling of 'PenpalSiblingSovereignAccount'`
- `Fix the spelling of 'PenpalSudoAccount'`
- `Fix the spelling of 'insufficient'`
- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`
- `Fix the spelling of 'subtracted'`
- `Fix the spelling of 'CandidatePendingAvailability'`
- `Fix the spelling of 'exclusive'`
- `Fix the spelling of 'until'`
- `Fix the spelling of 'discriminator'`
- `Fix the spelling of 'nonexistent'`
- `Fix the spelling of 'subsystem'`
- `Fix the spelling of 'indices'`
- `Fix the spelling of 'committed'`
- `Fix the spelling of 'topology'`
- `Fix the spelling of 'response'`
- `Fix the spelling of 'beneficiary'`
- `Fix the spelling of 'formatted'`
- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`
- `Fix the spelling of 'succeeded'`
- `Fix the spelling of 'reopened'`
- `Fix the spelling of 'proposer'`
- `Fix the spelling of 'InstantiationNonce'`
- `Fix the spelling of 'depositor'`
- `Fix the spelling of 'expiration'`
- `Fix the spelling of 'phantom'`
- `Fix the spelling of 'AggregatedKeyValue'`
- `Fix the spelling of 'randomness'`
- `Fix the spelling of 'defendant'`
- `Fix the spelling of 'AquaticMammal'`
- `Fix the spelling of 'transactions'`
- `Fix the spelling of 'PassingTracingSubscriber'`
- `Fix the spelling of 'TxSignaturePayload'`
- `Fix the spelling of 'versioning'`
- `Fix the spelling of 'descendant'`
- `Fix the spelling of 'overridden'`
- `Fix the spelling of 'network'`
Let me know if this structure is adequate.
**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.
~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~
~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~
~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~
**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.
~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~
Related to
https://github.com/paritytech/parity-bridges-common/issues/2538
This PR doesn't contain any functional changes.
The PR moves specific bridged chain definitions from
`bridges/primitives` to `bridges/chains` folder in order to facilitate
the migration of the `parity-bridges-repo` into `polkadot-sdk` as
discussed in https://hackmd.io/LprWjZ0bQXKpFeveYHIRXw?view
Apart from this it also includes some cosmetic changes to some
`Cargo.toml` files as a result of running `diener workspacify`.
Svyatoslav Nikolsky