* Add clippy config and remove .cargo from gitignore
* first fixes
* Clippyfied
* Add clippy CI job
* comment out rusty-cachier
* minor
* fix ci
* remove DAG from check-dependent-project
* add DAG to clippy
Co-authored-by: alvicsam <alvicsam@gmail.com>
* Add `DisputeState` to `DisputeCoordinatorMessage::RecentDisputes`
The new signature of the message is:
```
RecentDisputes(oneshot::Sender<Vec<(SessionIndex, CandidateHash, DisputeStatus)>>),
```
As part of the change also add `DispiteStatus` to
`polkadot_node_primitives`.
* Move dummy_signature() in primitives/test-helpers
* Enable staging runtime api on Rococo
* Implementation
* Move disputes to separate module
* Vote prioritisation
* Duplicates handling
* Double vote handling
* Unit tests
* Logs and metrics
* Code review feedback
* Fix ACTIVE/INACTIVE separation and update partition names
* Add `fn dispute_is_inactive` to node primitives and refactor `fn get_active_with_status()` logic
* Keep the 'old' logic if the staging api is not enabled
* Fix some comments in tests
* Add warning message if there are any inactive_unknown_onchain disputes
* Add file headers and remove `use super::*;` usage outside tests
* Adding doc comments
* Fix test methods names
* Fix staging api usage
* Fix `get_disputes` runtime function implementation
* Fix compilation error
* Fix arithmetic operations in tests
* Use smaller test data
* Rename `RuntimeApiRequest::StagingDisputes` to `RuntimeApiRequest::Disputes`
* Remove `staging-client` feature flag
* fmt
* Remove `vstaging` feature flag
* Some comments regarding the staging api
* Rename dispute selection modules in provisioner
with_staging_api -> prioritized_selection
without_staging_api -> random_selection
* Comments for staging api
* Comments
* Additional logging
* Code review feedback
process_selected_disputes -> into_multi_dispute_statement_set
typo
In trait VoteType: vote_value -> is_valid
* Code review feedback
* Fix metrics
* get_disputes -> disputes
* Get time only once during partitioning
* Fix partitioning
* Comments
* Reduce the number of hardcoded api versions
* Code review feedback
* Unused import
* Comments
* More precise log messages
* Code review feedback
* Code review feedback
* Code review feedback - remove `trait VoteType`
* Code review feedback
* Trace log for DisputeCoordinatorMessage::QueryCandidateVotes counter in vote_selection
* split metrics from collation generation
* move metrics to separate file out of backing
* split bitfield signing metrics
* split candidate validation metrics
* split chain api metrics
* split metrics from runtime API
* util is not used in backed metrics mod
* fmt
* missing types
* sure
* Move `trait ParachainHost` to a separate version independent module
`trait ParachainHost` is no longer part of a specific primitives
version. Instead there is a single trait for stable and staging api
versions. The trait contains stable AND staging methods. The latter are
explicitly marked as unstable.
* Fix `use` primitives
`polkadot_primitives::v2` becomes `polkadot_primitives::runtime_api`
* Staging API declaration and stubs
Introduces the concept for 'staging functions' in runtime API. These
functions are still in testing and they are meant to be used only
within test networks (Westend).
They coexist with the stable calls for technical reasons - maintaining
different runtime APIs for different networks is hard to implement.
Check the doc comments in source files for more details how the staging
API should be used.
* Add new staging method - get_session_disputes()
Add `staging_get_session_disputes` to `ParachainHost` as the first
method of the staging API.
* Hide vstaging runtime api implementations behind feature flag
* Fix test runtime
* fn staging_get_session_disputes() is renamed to fn staging_get_disputes()
* remove v0 primitives from polkadot-primitives
* first pass: remove v0
* fix fallout in erasure-coding
* remove v1 primitives, consolidate to v2
* the great import update
* update runtime_api_impl_v1 to v2 as well
* guide: add `Version` request for runtime API
* add version query to runtime API
* reintroduce OldV1SessionInfo in a limited way
* approval-voting: add more logs
* approval-voting: query finalized block on startup and increase look back
* runtime-api: do not cache None SessionInfo
The runtime version check `runtime_version <= version` was wrong, it
needs to be `>=`. Besides that the `WIDELY_DEPLOYED_API_VERSION` is
removed. The runtime version is already cached, aka we don't always call
into the runtime when requesting the runtime version. So, there is no
need to "optimize" this.
This commit hooks up the API provided by #4457 to the runtime API
subsystem. In a following PR this API will be consumed by the PVF
pre-checking subsystem.
Co-authored-by: Chris Sosnin <chris125_@live.com>
Co-authored-by: Chris Sosnin <chris125_@live.com>
As part of #3211 we will need to add a couple of runtime APIs. That
change is coming in a following PR.
The runtime API for pre-checking will be introduced with a version bump
for `ParachainHost`. This commit prepares the ground for that change,
by introducing a error condition that signals that the given API is not
supported.
* Companion PR for removing Prometheus metrics prefix
* Was missing some metrics
* Fix missing renames
* Fix test
* Fixes
* Update test
* Update Substrate
* Second time
* remove prefix from intergration test for zombienet
* update zombienet image
* Update Substrate
Co-authored-by: Bastian Köcher <info@kchr.de>
Co-authored-by: Javier Viola <pepoviola@gmail.com>
* remove Default from CandidateHash
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* chore: fmt
* remove backed candidate default
* Partial migration away from CandidateReceipt::default
* Remove more CandidateReceipt defaults
* fmt
* Mostly remove CommittedCandidateReceipt default usage
* Remove CommittedCandidateReceipt
* Remove more Defaults from polakdot primitives v1 + fmt
* Remove more Default from polkadot primites v1
* WIP trying to get overseer example + tests to compile
* feat: add primitives test helpers
* reduce deps of helper
* update primitive helpers
* make candidate validation compile
* fixup cargo lock
* make av-store compile
* fixup disputes coordinator tests
* test: fixup backing
* test: fixup approval voting
* fixup bitfield signing
* test: fixup runtime-api
* test: fixup availability dist
* foxi[ pverseer test]
* remove some Defaults, remove bounds from `dummy`
All `fn dummy` in primitives need to be removed anyways.
This aids in the transition.
* it's a test helper, so always use std
* test: fixup parachains runtime tests
Excluding benches.
* fix keyring
* fix paras runtime properly, no more default
* Remove fn dummy() usage from approval voting
* Move TestCandidateBuilder out of av store to test helpers
* Make candidate validation tests pass
* Make most dispute coirdinator tests pass
* Make provisioner tests work
* Make availability recovery tests work with test helpers
* Update polkadot-collator-protocol tests
* Update statement distribution tests
* Update polkadot overseer examples and tests
* Derive default for validation code so we don't break unrelated things
* Make para runtime test pass (no bench)
* Some more work
* chore: cargo fmt
* cargo fix
* avoid some Default::default
* fixup dispute coordinator test
* remove unused crate deps
* remove Default::default wherever possible, replace by dummy_* for the most part
* chore: cargo fmt
* Remove some warnings
* Remove CommittedCandidateReceipt dummy
* Remove CandidateReceipt dummy
* Remove CandidateDescriptor dummy
* Remove commented out code
* Fix para runtime tests
* chore: nightly
* Some updates to the builder
* Dynamically adjust mock head data size
* Make dispute cooridinator tests work
* Fix test candidate_backing_reorders_votes work
* +nightly-2021-10-29 fmt
* Spelling and remove a default use in builder
* Various clean up
* More small updates
* fmt
* More small updates
* Doc comments for test helpers
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=kusama-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/kusama/src/weights/runtime_parachains_paras_inherent.rs
* cargo run --quiet --release --features=runtime-benchmarks -- benchmark --chain=polkadot-dev --steps=50 --repeat=20 --pallet=runtime_parachains::paras_inherent --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --output=./runtime/polkadot/src/weights/runtime_parachains_paras_inherent.rs
* Update lib.rs
* review comments
* fix warnings
* fix test by using correct candidate receipt relay parent
Co-authored-by: Andronik Ordian <write@reusable.software>
Co-authored-by: emostov <32168567+emostov@users.noreply.github.com>
Co-authored-by: Parity Bot <admin@parity.io>
Co-authored-by: Gavin Wood <gavin@parity.io>
* Introduce new Runtime API endpoint
`persisted_validation_data_with_code_hash` that will be used
by the candidate validation subsystem in order to decrease amount
of runtime API requests.
* Node-side part of new runtime API request
* Define code hash getter via macro
* Rename new endpoint to `assumed_validation_data`
* Docs for runtime API impl of new endpoint
* AssumedValidationData specialized request function
* fmt
* dummy: impl another runtime API
* query the on chain disputes, and inform self
* make use of the refactor
* minro
* SPLIT ME
* write dispute values
* wip
* impl for all runtimes
* chore: fmt
* [] -> get
* fixup mock runtime
* fixup
* fixup discovery for overseer init
* chore: fmt
* spellcheck
* rename imported_on_chain_disputes -> on_chain_votes
* reduction
* make it mockable
* rename and refactor
* don't query on chain info if it's not needed
* yikes
* fmt
* fix test
* minimal fix for existing tests
* attempt to fetch the session info from the rolling window before falling back
* moved
* comments
* comments
* test for backing votes
* rename
* Update runtime/polkadot/src/lib.rs
* chore: spellcheck + dict
* chore: fmt
* fixup cache size
* add warning
* logging, rationale, less defense
* introduce new unchecked, that still checks in debug builds
* fix
* draft alt approach
* fix unused imports
* include the session
* Update node/core/dispute-coordinator/src/real/mod.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* provide where possible
* expand comment
* fixin
* fixup
* ValidityVote <-> ValidityAttestation <-> CompactStatement has a 1:1 representation
* mark TODO
* Update primitives/src/v1/mod.rs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* address review comments
* update docs
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* CI: add spellcheck
* revert me
* CI: explicit command for spellchecker
* spellcheck: edit misspells
* CI: run spellcheck on diff
* spellcheck: edits
* spellcheck: edit misspells
* spellcheck: add rules
* spellcheck: mv configs
* spellcheck: more edits
* spellcheck: chore
* spellcheck: one more thing
* spellcheck: and another one
* spellcheck: seems like it doesn't get to an end
* spellcheck: new words after rebase
* spellcheck: new words appearing out of nowhere
* chore
* review edits
* more review edits
* more edits
* wonky behavior
* wonky behavior 2
* wonky behavior 3
* change git behavior
* spellcheck: another bunch of new edits
* spellcheck: new words are koming out of nowhere
* CI: finding the master
* CI: fetching master implicitly
* CI: undebug
* new errors
* a bunch of new edits
* and some more
* Update node/core/approval-voting/src/approval_db/v1/mod.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Update xcm/xcm-executor/src/assets.rs
Co-authored-by: Andronik Ordian <write@reusable.software>
* Apply suggestions from code review
Co-authored-by: Andronik Ordian <write@reusable.software>
* Suggestions from the code review
* CI: scan only changed files
Co-authored-by: Andronik Ordian <write@reusable.software>
* code stored in para + modify CandidateDescriptor.
* WIP: digest + some more impl
* validation_code_hash in payload + check in inclusion
* check in client + refator
* tests
* fix encoding indices
* remove old todos
* fix test
* fix test
* add test
* fetch validation code inside collation-generation from the relay-chain
* HashMismatch -> PoVHashMismatch + miscompilation
* refactor, store hash when needed
* storage rename: more specific but slightly too verbose
* do not hash on candidate validation, fetch hash instead
* better test
* fix test
* guide updates
* don't panic in runtime
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
* Add a config field to babe epochs
* Fix test
* Add BABE_GENESIS_EPOCH_CONFIG consts
* Use PrimaryAndSecondaryVRFSlots and remove newlines
* Make epoch_configs Some
* Fix tests
* Fix test service tests
* Add a BabeEpochConfigMigrations OnRuntimeUpgrade
* Apply suggestions
* Use PrimaryAndSecondaryPlainSlots in kusama
* Remove migration from test runtime and rococo
* Add HasPalletPrefix
* Rename to BabePalletPrefix and change BabeApi -> Babe
* "Update Substrate"
* Update substrate
* Resolve parantheses errors
Co-authored-by: parity-processbot <>
* skeleton
* skeleton aux-schema module
* start approval types
* start aux schema with aux store
* doc
* finish basic types
* start approval types
* doc
* finish basic types
* write out schema types
* add debug and codec impls to approval types
* add debug and codec impls to approval types
also add some key computation
* add debug and codec impls to approval types
* getters for block and candidate entries
* grumbles
* remove unused AssignmentId
* load_decode utility
* implement DB clearing
* function for adding new block entry to aux store
* start `canonicalize` implementation
* more skeleton
* finish implementing canonicalize
* tag TODO
* implement a test AuxStore
* add allow(unused)
* basic loading and deleting test
* block_entry test function
* add a test for `add_block_entry`
* ensure range is exclusive at end
* test clear()
* test that add_block sets children
* add a test for canonicalize
* extract Pre-digest from header
* utilities for extracting RelayVRFStory from the header-chain
* add approval voting message types
* approval distribution message type
* subsystem skeleton
* state struct
* add futures-timer
* prepare service for babe slot duration
* more skeleton
* better integrate AuxStore
* RelayVRF -> RelayVRFStory
* canonicalize
* implement some tick functionality
* guide: tweaks
* check_approval
* more tweaks and helpers
* guide: add core index to candidate event
* primitives: add core index to candidate event
* runtime: add core index to candidate events
* head handling (session window)
* implement `determine_new_blocks`
* add TODO
* change error type on functions
* compute RelayVRFModulo assignments
* compute RelayVRFDelay assignments
* fix delay tranche calc
* assignment checking
* pluralize
* some dummy code for fetching assignments
* guide: add babe epoch runtime API
* implement a current_epoch() runtime API
* compute assignments
* candidate events get backing group
* import blocks and assignments into DB
* push block approval meta
* add message types, no overseer integration yet
* notify approval distribution of new blocks
* refactor import into separate functions
* impl tranches_to_approve
* guide: improve function signatures
* guide: remove Tick from ApprovalEntry
* trigger and broadcast assignment
* most of approval launching
* remove byteorder crate
* load blocks back to finality, except on startup
* check unchecked assignments
* add claimed core to approval voting message
* fix checks
* assign only to backing group
* remove import_checked_assignment from guide
* newline
* import assignments
* abstract out a bit
* check and import approvals
* check full approvals from assignment import too
* comment
* create a Transaction utility
* must_use
* use transaction in `check_full_approvals`
* wire up wakeups
* add Ord to CandidateHash
* wakeup refactoring
* return candidate info from add_block_entry
* schedule wakeups
* background task: do candidate validation
* forward candidate validation requests
* issue approval votes when requested
* clean up a couple TODOs
* fix up session caching
* clean up last unimplemented!() items
* fix remaining warnings
* remove TODO
* implement handle_approved_ancestor
* update Cargo.lock
* fix runtime API tests
* guide: cleanup assignment checking
* use claimed candidate index instead of core
* extract time to a trait
* tests module
* write a mock clock for testing
* allow swapping out the clock
* make abstract over assignment criteria
* add some skeleton tests and simplify params
* fix backing group check
* do backing group check inside check_assignment_cert
* write some empty test functions to implement
* add a test for non-backing
* test that produced checks pass
* some empty test ideas
* runtime/inclusion: remove outdated TODO
* fix compilation
* av-store: fix tests
* dummy cert
* criteria tests
* move `TestStore` to main tests file
* fix unused warning
* test harness beginnings
* resolve slots renaming fallout
* more compilation fixes
* wip: extract pure data into a separate module
* wip: extract pure data into a separate module
* move types completely to v1
* add persisted_entries
* add conversion trait impls
* clean up some warnings
* extract import logic to own module
* schedule wakeups
* experiment with Actions
* uncomment approval-checking
* separate module for approval checking utilities
* port more code to use actions
* get approval pipeline using actions
* all logic is uncommented
* main loop processes actions
* all loop logic uncommented
* separate function for handling actions
* remove last unimplemented item
* clean up warnings
* State gives read-only access to underlying DB
* tests for approval checking
* tests for approval criteria
* skeleton test module for import
* list of import tests to do
* some test glue code
* test reject bad assignment
* test slot too far in future
* test reject assignment with unknown candidate
* remove loads_blocks tests
* determine_new_blocks back to finalized & harness
* more coverage for determining new blocks
* make `imported_block_info` have less reliance on State
* candidate_info tests
* tests for session caching
* remove println
* extricate DB and main TestStores
* rewrite approval checking logic to counteract early delays
* move state out of function
* update approval-checking tests
* tweak wakeups & scheduling logic
* rename check_full_approvals
* test that assignment import updates candidate
* some approval import tests
* some tests for check_and_apply_approval
* add 'full' qualifier to avoid confusion
* extract should-trigger logic to separate function
* some tests for all triggering
* tests for when we trigger assignments
* test wakeups
* add block utilities for testing
* some more tests for approval updates
* approved_ancestor tests
* new action type for launch approval
* process-wakeup tests
* clean up some warnings
* fix in_future test
* approval checking tests
* tighten up too-far-in-future
* special-case genesis when caching sessions
* fix bitfield len
Co-authored-by: Andronik Ordian <write@reusable.software>
* collation-generation: use persisted validation data
* node: remote FullValidationData API
* runtime: remove FullValidationData API
* backing tests: use persisted validation data
* FullCandidateReceipt: use persisted validation data
This is not a big change since this type is not used anywhere
* Remove ValidationData and TransientValidationData
Also update the guide
alexgparity