* Allow creation of authority lists with any number of authorities
* Move keyring helpers into their own module
* Add helper for generating list of test accounts
* Fix import names in tests
* Rename Keyring trait to Signer
* Get list of accounts in a more functional way
* Clarify meaning of `test_keyring` return type
* Use concrete test account type instead of generics
* Make sure voter set contains all authorities which signed off on pre-commits
* Add way to create tunable GRANDPA justifications
* Use new function in tests
* Allow multiple authorities on a single fork
* Only store pre-commit targets instead of full ancestry chains
* Rename precommit_header to be more generic
* Push new digest item instead of overriding entire digest
* Ensure that we generate chains with non-zero length
* Extract justification creation parameters into struct
* Appease Clippy
* Use `sp-application-crypto` when generating test justifications
* Try to justifications tests compiling
* Try using `ed25519_dalek` for justification creation
* Small cleanup
* Correctly generate a Keypair
The public key didn't correspond to what would be derived
with the given private key.
* Remove enum index hack
* Clean up test account helper functions
* Fix tests in `pallet-finality-verifier`
* Get `pallet-substrate-bridge` tests compiling again
* Use the correct keyring in justification tests
* Clean up Keyring related code a bit
* Appease Clippy
* Remove unused import
* Use keyring variants directly
* Remove unecessary From implementation
* Move storage proof checker to runtime primtives
* Add method for parsing storage proofs
* Use finality-verifier pallet in runtime-common
* Get bridge pallet compiling again
* Use storage prover from bp-runtime in a few more places
* Don't leak `std` items from proof helper into `no-std` builds
* Fix benchmarking compilation
* Remove unused import in fuzzer
* Bump Substrate to version used by Polkadot (`5f056830`)
* Use `log` crate for runtime logging
See https://github.com/paritytech/substrate/pull/8128/ for more info.
* Stop using return value from `execute_block`
* Update test weight
* Use no-op ancestry checker
* Check that current header height is greater than last finalized
* Ensure that incoming headers are strictly greater than last finalized
* Ensure that header numbers always increase in tests
* Revert "Pin Rust Nightly to 2020-12-17 (#652)"
This reverts commit e54e6f7e3d34c28d698e637f9099162b3c1917e9.
* fix clippy
* clippy again
* more clippy in test code
* and new cargo fmt
* another try
* Make AncestryProof type more flexible
* Only import single finalized header instead of a chain
* Fix unchecked header import tests
* Add option for limiting ancestry proof size
* Update finality verifier Config in runtimes
* Update some documentation
* Fix Clippy warning
* Allow AncestryChecker to return proof size
Stops us from abusing the `Size` trait
* Remove Size impl for Vec<T>
* Remove size contraints for ancestry proofs
With different proof types its unclear how to "size" should be interpreted,
so we remove this requirement all together to avoid confusion.
* Add method for unchecked header imports
* Update docs for importing unchecked headers
* Import unchecked headers in HeaderChain implementation
* Fix Clippy warnings
* Move unchecked header import out of Verifier struct
* Clean up unchecked import tests
* Change HeaderChain API to accept iterator of headers
* Use chains of headers in tests
* Remove unused Result return type when appending finalized headers
* Add test which shows that genesis changes are not enacted
* Use initial header's hash for unchecked authority set changes
* Appease Clippy
* Check ancestry before making unchecked writes
* Fix typo
* Fix Clippy warning
* Add note about `ancestry_proof` structure
* Use best hash storage item directly
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
* Add skeleton for `pallet-finality-verifier`
* Sketch out implementation for importing finality proofs
* Get pallet compiling
* Introduce skeleton for mock runtime
* Start using real Grandpa types in finality pallet
* Redefine types in header chain primitives crate
* Implement HeaderChain for Substrate bridge pallet
* Plug Substrate Bridge Pallet into verifier mock
* Fix compilation of `header-chain` primitives
* Start writing to base pallet storage
* Add first "cross-pallet" test
* Move keyring primitives used in tests to shared crate
* Stop pulling `std` deps into `no_std` builds
* Revert "Stop pulling `std` deps into `no_std` builds"
This reverts commit f74dd660652f98b7336936d1534a4e63cc9169a5.
* Revert "Move keyring primitives used in tests to shared crate"
This reverts commit b774fa730b2cdc40545afff308a66b0840266001.
* Use new SS58Prefix type in mock
* Start using `bp-test-utils` in finality pallet
* Start using real justification code
* Get a test working with real justification verification
* Add basic tests for invalid proofs
* Get rid of AncestryProof config type
* Add error types to transaction outcome
* Bound number of headers allowed in a single ancestry proof
* Disallow invalid authority sets
* Remove unused items
* Add some documentation
* Get rid of Clippy warnings
* Rename BaseHeaderChain to TransactionVerifier
* Remove unused code
* Make dummy trait implementations more generic
* Fix more Clippy complaints
* Update tests to use fix for duplicate headers
* Fix benchmarking compilation
* Rename TransactionVerifier to InclusionProofVerifier
* Move justification module to header-chain primitives crate
* Get justification module compiling in new location
* Get justification module tests compiling
* Use justification code from `header-chain` crate
Mostly compiles, having issues with std/test feature flags across crates.
* Move some code around
* Move justification tests to integration testing crate
* Add `test-utils` crate
* Remove tests and test-helper module from justification code
* Use `test-utils` in Substrate bridge pallet tests
* Remove `sp-keyring` related code from `pallet-substrate-bridge`
* Remove `helpers` module from `pallet-substrate-bridge`
* Add some documentation
* Add more documentation
* Fix typo
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update `sp-io` dependency
* Rename Trait to Config
* RustFmt
* Bump `sp-io` again
* Use new frame_system weight types in Rialto and Millau runtimes
* Update test Runtimes to use new weight types
* Bump `sp-io` again
* Update to not-the latest first.
* Update benchmarks.
* Another Trai.
* Move new weight types into runtime primitive crates
This allows us to check limits for extrinsics from other parts
of the codebase without pulling in the entire chain runtime.
* Remove leftover comments
* Move new functions to a better location
* Small formatting fixes
* Add actual documentation to new weight config types
* Decrease maximum block weight of Millau chain
* Decreease maximum block length of Millau chain
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
* High level docs - start.
* Clean up README
* Start adding details to high level docs
* More docs on the header sync pallet
* Testing scenarios document.
* Add some scenarios.
* Add multi-sig scenario.
* Start writing about message dispatch pallet
* Move content from old README into PoA specific doc
* Apply suggestions from code review
Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com>
* GRANDPA for consistency.
* Describe scenario steps.
* WiP
* Add notes about block production and forks
* Update.
* Add sequence diagram for Millau to Rialto transfer
* Clean up header sync pallet overview
* Remove leftover example code
* Clean up testing scenarios and amend sequence diagram.
* Linking docs.
* Add some more docs.
* Do a bit of cleanup on the high-level docs
* Clean up the testing scenario
* Fix typos in flow charts
* Fix small typo
* Fix indentation of Rust block
* Another attempt at rendering block correctly
* TIL about lazy list numbering in Markdown
* Add list numbers across sections
* Start counting from correct number
* Update README to use correct path to local scripts
* Wrap ASCII art in code block
Co-authored-by: Tomasz Drwięga <tomasz@parity.io>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Co-authored-by: Andreas Doerr <adoerr@users.noreply.github.com>
* fixed clippy warnings
* Revert "Actually use pinned nightly version when building runtimes (#465)"
This reverts commit dedddb6b0f22260e00053c28873a0cb1fbea22e2.
* Revert "Pin Rust Nightly Version (#420)"
This reverts commit 8902ac2030cf7ef48ec512463424f134a3b38804.
* fix after revert
* another fix after revert
* more clippy fixes
* Allow bridge pallet to return no finalized headers
* Update Runtime APIs to optionally return best finalized header
* Update relay to handle optional best finalized headers
* Fix Clippy lints
* Return a dummy header instead of an Option
* Remove Option from runtime Apis
* Remove support for handling optional finalized headers in relay
* Copy-Pasta owner and freezing code from `message-lane`
* Halt pallet if bridge hasn't been initialized
* Make owner optional in `message-lane` pallet
* Add `is_halted` to `InitializationData`
* Fix initialization tests
* Only allow pallet to be initialized once
* Add some logging around halting and ownership changes
* Remove `target` in debugging calls
* Add dispatchable for intializing pallet
* Add Polkadot JS types for Substrate bridge pallet
* Ensure Root is the only one that can initialize the pallet
* Add some tests
* Pack initialization data into struct
* Only allow pallet to be initialized once
* Use new initialization config in nodes
* Rename ScheduledChange in Ethereum pallet
We're renaming it to prevent clashes with Substrate bridge pallet type
of the same name. This is relevant when importing types to Polkadot JS
Apps.
* Move all Polkadot JS types into one file
* Appease Clippy
* Support multiple "best headers" from different forks
* Update the name of a test
* Add note about multiple scheduled changes
* Disallow multiple scheduled authority set changes
* Return multiple best headers from Runtime APIs
* Remove invalid test write-up
* Add some sketch-ups of test scenarios
* Clean up test scenarios
* Add module for testing forks
* Write headers to storage
* Add way to check expected outcome for header imports
* Add support for importing finality proofs
* Support importing headers which schedule changes
* Write out test scenario using new framework
* Map authority set changes across forks
Gets all the tests in the `forks` module passing
* Remove basic tests
These were used when working on the initial test helper
* Prevent multiple pending set changes on the same fork
* Remove old test which allowed imports past unfinalized header
* Ignore failing test (for now)
* Rewrite `if` comparison using `match` and `cmp`
Fixes Clippy warning: `comparison_chain`
* Add helper for writing test headers with default characteristics
* Fix test that checked authority set updates
* Make note about importing headers on different unfinalized fork
* Perform some cleanup on the fork module
* Fix Clippy complaints
* Provide list of unfinalized headers to Runtime APIs
* Add proofs to expect() calls
* Make tests the focus of the forks module
* Allow specific errors to be checked in fork tests
* Remove unused method
* Replace unreachable() statement with expect()
* Rename storage `unfinalized_headers() `to make its purpose more clear
* Update Runtime API name in relayer to match pallet
* Commit `unfinalized_headers` changes I forgot to add
* Rename ChainTipHeight to BestHeight
* Make schedule_next_set_change require a mutable reference
* Remove check for key when enacting authority set
We only expect to take the happy-path in the pallet anyways, so this check
to save ourselves the time spent decoding the entry isn't really used.
* Clear justification set when writing headers to storage
* Clarify why we only allow one set change per fork
* Change best_headers() to return HeaderIDs
Prevents us from returning full headers (which are more expensive to
get from storage) since we only care about header IDs (number, hash)
anyways.
* Fix Clippy complaint
* Make note about equivocations
* Use HeaderIds when returning incomplete headers
This change stops returning full headers which are more expensive
to get from storage than header Ids (number, hash) are. Clients likely
don't need the full header anyways which is why this change is fine.
* Introduce HeaderId type to reduce type complexity
* Add signal hash to storage during genesis config
* Return error instead of expect()-ing
* Fix Clippy lint about `ok_or` fn call
* Rename `forks` module to indicate that it's for testing
* Use `const` for `expect()` proofs
* Remove check that key exists before `kill`-ing value
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
* Add Rialto as a target for the Millau node
* Rename Rialto module to Rialto-PoA
This will reduce confusion as the Millau runtime has a
Rialto module as well which refers to the Substrate chain.
* Add Millau as a target for the Rialto node
* Be more explicit about Rialto PoA related code
* Missed some name changes in the Ethereum PoA relay
* Re-export Substrate pallet structs used by node
* Remove `first_scheduled_change` of Millau in Rialto node
* Make Millau's genesis config for Rialto bridge more accurate
* Set initial header for Millau config
* Update initial Millau authorities
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
* RustFmt Millau authorities
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
* Implement public helpers for querying header info
* Update `best_header` when importing headers
* Add BestHeader to GenesisConfig
* Define extra types for Millau primitives
* Start implementing runtime APIs in Millau runtime
* Add helper for getting headers which require a justification
* Add runtime API for getting headers requiring a justification
* Reword `expect()` proof for valid authority sets
* Fix typo
* Clean up Hasher comment
* Add the Call Dispatch Pallet back to the Millau runtime
* Use types from Rialto in bridge pallet config
* Use the Rialto runtime APIS in the Millau runtime
* Include Millau bridge instance in Rialto runtime
* Add missing doc comment
* Use one storage function for setting and clearing `RequiresJustification`
* Remove TODO comments
* Remove the Substrate primitives crate
The types here were only used in one place, the pallet itself. If other
components start using these types we can considering moving them back
into a standalone crate.
* Start trying to integrate justification module
* Make Substrate blocks configurable in Pallet
* WIP: Try and generalize justification test helpers
* Fix tests which use "real" justifications
* Put common test helpers alongside mock code
* Use common helper for creating headers
* Remove usage of UintAuthorityId
This change favours the use of the Ed25519Keyring authorities
in order to keep things consistent with the tests.
* Add documentation around config trait types
* Make test header, hash, and number types consistent
* Update modules/substrate/src/verifier.rs
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
* Update modules/substrate/src/lib.rs
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Update modules/substrate/Cargo.toml
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
* Derive `RuntimeDebug` instead of `Debug`
* Add `Paramter` as a trait constraint on config types
Since we use these types as part of the dispatchable functions
we should explicitly require this.
* Enforce that hasher output matches expected hash type
* Accept headers over indexes when making test justifications
* Check that authority sets are valid
* Make Clippy happy
* Apply correct Clippy fix
* Move justification code into primitives module
* Use new module in verifier code
* Add primitives module for Substrate test helpers
* WIP
* Move justification generation into test_helpers
* Revert commits which move `justification` into primitives
This reverts commit 03a381f0bc4a8dbe4785c30d42ab252a06ba876c.
Co-authored-by: Svyatoslav Nikolsky <svyatonik@gmail.com>
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
* Add pallet template from Substrate Dev Hub
* Clean up un-needed stuff from template
* Sketch out dispatchable interface
* Introduce notion of finality chain
* Add dependencies which were removed during a rebase
* Sketch out idea for finality header-chain pallet
* Sketch out ChainVerifier trait
* Add storage parameter to verifier
* Write out some things I think I need for finality verification
* Add some pseudocode for marking finalized headers
* Remove parity_scale_codec duplicate
* Move verification logic into pallet
I've been struggling with getting the generic types between the storage and verifier
traits to play nice with each other. As a way to continue making progress I'm moving
everything to the pallet. This way I hope to make progress towards a functional
pallet.
* Start doing verification around authority set changes
* Remove commented BridgeStorage and ChainVerifier traits
* Create Substrate bridge primitives crate
* Add logic for updating scheduled authority sets
* Introduce notion of imported headers
* Implement basic header ancestry checker
* Add mock runtime for tests
* Add testing boilerplate
* Add some storage read/write sanity tests
* Add some basic header import tests
* Add tests for ancestry proofs
* Create helper for changing authority sets
* Fix authority set test
Fixes a problem with how the scheduled change was counted as well as
a SCALE encoding issue
* Correctly check for scheduled change digests
There's no guarantee that the consensus digest item will be the last
one in a header, which is how it was previously being checked.
Thanks to Andre for pointing me to the Grandpa code that does this.
* Mark imported headers as finalized when appropriate
When a header that finalizes a chain of headers is succesfully imported
we also want to mark its ancestors as finalized.
* Add helper for writing test headers
* Add test helper for scheduling authority set changes
* Bump Substrate pallet and primitives to rc6
* Remove Millau verifier implementation
* Add some doc comments
* Remove some needless returns
* Make Clippy happy
* Split block import from finalization
* Make tests compile again
* Add test for finalizing header after importing children
* Create a test stub for importing future justifications
* Start adding genesis config
* Reject justifications from future
We should only be accepting justifications for the header
which enacted the current authority set. Any ancestors of
that header which require a justification can be imported
but they must not be finalized.
* Add explanation to some `expect()` calls
* Start adding GenesisConfig
* Plug genesis config into runtime
* Remove tests module
* Check for overflow when updating authority sets
* Make verifier take ownership of headers during import
* Only store best finalized header hash
Removed the need to store the whole header, since we store
it was part of the ImportedHeaders structure anyways
* Add some helpers to ImportedHeader
* Update ancestry checker to work with ImportedHeaders
* Update ancestry tests to use ImportedHeaders
* Update import tests to use ImportedHeaders
* Clean up some of the test helpers
* Remove stray dbg!
* Add doc comments throughout
* Remove runtime related code
* Fix Clippy warnings
* Remove trait bound on ImportedHeader struct
* Simplify checks in GenesisConfig
* Rename `get_header_by_hash()`
* Alias `parity_scale_codec` to `codec`
* Reword Verifier documentation
* Missed codec rename in tests
* Split ImportError into FinalizationError
* Remove ChainVerifier trait
This trait was a remenant of the original design, and it is not required
at the moment. Something like it should be added back in the future to
ensure that other chains which conform to this interface can be used
by higher-level bridge applications.
* Fix the verifier tests so they compile
* Implement Deref for ImportedHeader
* Get rid of `new` methods for some Substrate primitives
* Ensure that a child header's number follows its parent's
* Prevent ancestry checker from aimlessly traversing to genesis
If an ancestor which was newer than the child header we were checking we
would walk all the way to genesis before realizing that we weren't related.
This commit fixes that.
* Remove redundant clones
* Ensure that old headers are not finalized
Prevents a panic where if the header being imported and `best_finalized`
were the same header the ancestry checker would return an empty list. We
had made an assumption that the list would always be populated, and if this
didn't hold we would end up panicking.
* Disallow imports at same height as `best_finalized`
* Fix Clippy warnings
* Make NextScheduledChange optional
* Rework how scheduled authority set changes are enacted
We now require a justification for headers which _enact_ changes
instead of those which _schedule_ changes. A few changes had to
be made to accomodate this, such as changing when we check for
scheduled change logs in incoming headers.
* Update documentation for Substrate Primitives
* Clarify why we skip header in requires_justification check
* Add description to assert! call
* Fix formatting within macros
* Remove unused dependencies from runtime
* Remove expect call in GenesisConfig
* Turn FinalityProof into a struct
* Add some inline TODOs for follow up PRs
* Remove test which enacted multiple changes
This should be added back at some later point in time, but right now
the code doesn't allow for this behaviour.
* Use `contains_key` when checking for header
This is better than using `get().is_some()` since we skip
decoding the storage value
* Use initial hash when updating best_finalized
* Add better checks around enacting scheduled changes
* Rename finality related functions
* Appease Clippy
* Bump Substrate to rc5
* Bump async-std to v1.6.2
There was a bug in v.1.6.0 which kept us locked to v1.5 releases.
I think that's fixed now so I'm bumping this.
* Update bridge node runtime
* Update node service
* Update CLI
* Add SystemWeightInfo type to test runtimes
* Add RPC extension builder to service
* Directly return rpc_extensions_builder
* Allow complex types in service
This comes from Substrate, so I'd rather just keep the code as is
* Update benchmarking code for new CLI
* Bump Substrate to rc4
* Add BaseCallFilter type
* Add DenyUnsafe to SystemApi extension
* Use new ServiceBuilder build functions
* Add BaseCallFilter to test runtimes
* Remove old comments
* Add `rev` and `git` fields back
Turns out that if you don't have these future release candidates will
be used if available. For instance, once `rc5` is released a fresh pull
would use that instead of `rc4` which is what we want.
* Use tag release instead of specific commit
Will make scripted updates easier in the future
* Add short script to update between `rc` versions
* Update scripts/update_rc.sh
Co-authored-by: Tomasz Drwięga <tomusdrw@users.noreply.github.com>
Hernando Castano