* upgrade primitives to allow changing validation function
* set up storage schema for old parachains code
* fix compilation errors
* fix test compilation
* add some tests for past code meta
* most of the runtime logic for code upgrades
* implement old-code pruning
* add a couple tests
* clean up remaining TODOs
* add a whole bunch of tests for runtime functionality
* remove unused function
* fix runtime compilation
* extract some primitives to parachain crate
* add validation-code upgrades to validation params and result
* extend validation params with code upgrade fields
* provide maximums to validation params
* port test-parachains
* add a code-upgrader test-parachain and tests
* fix collator tests
* move test-parachains to own folder to work around compilation errors
* fix test compilation
* update the Cargo.lock
* fix parachains tests
* remove dbg! invocation
* use new pool in code-upgrader
* bump lockfile
* link TODO to issue
* add dummy parachains.toml
* flesh out parachains.toml
* finish phase-1 rendering
* render to svg instead
* put graphviz svg through sanitizer so github can render
* return to PNG
* add pov-block gossip message to network
* tests for pov-block gossip
* integrate pov-block gossip into main protocol
* message validation fetches pov blocks
* remove stray dbg! invocation
* test that pov-block is fetched from relay-parent topic
* move protocol.rs into subfolder
* add trait for mocking network behavior
* add a mock version of network ops
* remove some redundant parameters from service messages
* ensure fetching erasure chunks automatically cancels
* introduce dummy ProvideRuntimeApi
* abstract over gossip somewhat
* add mock gossip handler
* skeleton test
* remove dependence of shared table on router
* remove worker dependence on its own sender
* test shutdown
* add tests
* test that gossip streams are cleaned up correctly
* refactor worker out into its own struct and reduce bound on executor
* remove reliance of tests on global thread pool
* expunge legacy code from polkadot-network
* mostly rip out old legacy protocol from service
* ensure validation work is spawned by incoming messages
* decouple availabliity store from network logic; clean up data flow
* av_store: test helpers and use futures-abort
* update polkadot-validation to pass n_validators when submitting chunks
* fallible erasure-chunk fetching
* implement `ErasureNetworking` for new network prot
* API for registering availability store in network
* fully integrate new network service into service
* fix validation tests
* scaffolding for porting collator over to new network
* track connected validators' peer IDs and distribute collators' collations
* helper in network for fetching all checked statements
* fix adder-collator
* actually register notifications protocol
* Update service/src/lib.rs
* merge with master
* encode the candidate statement as only the hash
* refactor CandidateReceipt and CollationInfo
* introduce an abridged candidate receipt type
* erasure coding stores candidate receipt
* store omitted data instead and introduce AvailableData type
* refactor availability-store schema
* tweak schema and APIs a bit more
* get availability-store tests passing
* accept AbridgedCandidateReceipt in `set_heads`
* change statement type in primitives to be hash-only
* fix parachains runtime tests
* fix bad merge
* rewrite validation pipeline
* remove evaluation module
* use abridged candidate hash as canonical
* statement table uses abridged candidate receipts
* kill availability_store::Data struct
* port shared table to new validation pipelines
* extract full validation pipeline to helper
* remove old validation pipeline from collation module
* polkadot-validation compiles
* polkadot-validation tests compile
* make local collation available in validation service
* port legacy network code
* polkadot-network fully ported
* network: ensure fresh statement is propagated
* remove pov_block_hash from LocalValidationData
* remove candidate_hash field from AttestedCandidate and update runtime
* port runtimes to new ParachainHost definition
* port over polkadot-collator
* fix test compilation
* better fix
* remove unrelated validation work dispatch fix
* address grumbles
* fix equality check
* add a maximum code size and head data size
* get existing tests passing
* add tests for slots logic
* test registrar behavior
* introduce maximums and bump versions
* address review grumbles
* work around publicizing derive
* remove unneeded and wrong doc
* extract all network code to legacy submodule
* update references to legacy proto
* skeleton of futures-based protocol
* refactor skeleton to use background task
* rename communication_for to build_table_router
* implement internal message types for validation network
* basic ParachainNetwork and TableRouter implementations
* add some module docs
* remove exit-future from validation
* hack: adapt legacy protocol to lack of exit-future
* generalize RegisteredMessageValidator somewhat
* instantiate and teardown table routers
* clean up RouterInner drop logic
* implement most of the statement import loop
* implement statement loop in async/await
* remove unneeded TODO
* most of the collation skeleton
* send session keys and validator roles
* also send role after status
* use config in startup
* point TODO to issue
* fix test compilation
* runtime: candidate receipt must pass parent head
* construct parachain candidates using correct parent_head
* validate that the parent header is correct in candidate receipt
* fix test fallout
* bump runtime versions
* add some more docs about statement import
* instantiate environment async
* move attestation service into subfolder
* refactor validation service architecture somewhat
* remove dependence on validation service in proposer
* fix a bunch of warnings
* improve docs
* introduce a builder for the validation service
* extract block production to its own file
* integrate new API into service
* address review grumbles
* point to in-progress Substrate branch
* instantiate environment async
* Fix futures
* Bump runtime
* Fix collation tests
* point to polkadot-master again
* point to polkadot-master again
* update deps
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
* core logic for ICMP gossip
* refactor gossip to make more extension friendly
* move files aroun
* extract attestation-gossip logic to its own module
* message validation and broadcast logic
* fix upstream crates' compilation
* add a test
* another test for overlapping
* Some grammar and phrasing tweaks
Co-Authored-By: Luke Schoen <ltfschoen@users.noreply.github.com>
* add since parameter to ingress runtime API
* broadcast out known unrouted message queues
* fix compilation of service and collator
* remove useless index_mapping
* some tests for icmp propagation
* fix decoding bug and test icmp queue validation
* simplify engine-id definition
Co-Authored-By: Sergei Pepyakin <sergei@parity.io>
* address some grumbles
* some cleanup of old circulation code
* give network a handle to extrinsic store on startup
* an honest collator ensures data available as well
* address some grumbles
* add docs; rename the attestation session to "leaf work"
* module docs
* move gossip back to gossip.rs
* clean up and document attestation-gossip a bit
* some more docs on the availability store
* store all outgoing message queues in the availability store
* filter `Extrinsic` out of validation crate
* expunge Extrinsic from network
* expunge Extrinsic from erasure-coding
* expunge Extrinsic from collator
* expunge from adder-collator
* rename ExtrinsicStore to AvailabilityStore everywhere
* annotate and clean up message-routing tests
* record attestations in storage ringbuffer
* remove some reliance on Aura
* fix up test configuration
* extract attestations stuff out to its own module
* add dummy inherent
* use double_map
* fix a couple more compilation errors
* strip out all ICMP code and begin gossip refactor
* validate incoming statements
* message_allowed logic
* compiles
* do reporting and neighbor packet validation
* tests compile
* propagate gossip messages
* test message_allowed
* some more tests
* address grumbles
* refactor out a consensus data fetcher from table router
* move statement checking logic into router
* refuse to start authority if collator
* support building the table router asynchronously
* instantiate_consensus does not overwrite old
* update key in new consensus if there was none before
* collator collects ingress from network
* test produced egress roots
* fix adder-collator compilation
* address first grumbles
* integrate new gossip with collator network launch
* address review
* Integrates new gossip system into Polkadot (#166)
* new gossip validation in network
* integrate new gossip into service
* Fix build
* Fix claims module
* fix warning
* update to latest master again
* update runtime
* compute ingress and routing in polkadot runtime
* extract parent candidates from block when beginning consensus
* fetch incoming messages when validating
* fix consensus tests
* parachain wasm execution uses messages
* update parachain tests to check if messages are executed
* abstract out network service to make room for network tests
* skeleton for incoming data fetch
* collate ingress from consensus-gossip
* keep track of validated candidates in the shared-table
* add some shared_table tests for new behavior
* broadcast egress messages on gossip
* test compute_ingress
* move network tests to module folder
* dummy network for consensus-network tests
* make consensus network generic over executor
* test egress broadcast and ingress fetch
* fix test compilation
* address some grumbles
* address grumbles and fix parachain shuffle
* remove broadcast parameter from consensus network trait
* erasure-coding block data
* adjust error handling
* merkleize chunks and yield branches for each
* construction and proving of merkle branches
* port over to new GF(2^16) impl
* some tests for wrapped_shard
* handle extra byte from GF(2^16) better
* point to github dependency
* add issue link
* point to master for reed-solomon-erasure
* add missing license header
* read head-data directly out of WASM memory
* implement ext_post_message for parachain WASM
* further refactoring of the parachain module
* add externalities error type
* accumulate posted messages when validating parachain candidate
* define Extrinsic type in primitives
* availability-store: store extrinsic data
* compute extrinsic and check against candidate
* add some egress queue tests
* grumbles & substrate update
* ensure everything builds
* update to latest substrate
* update WASM and runtime
* Rename Id to ParaId in decl_{module,storage} (exported metadata type) (#58)
* Rename Id to ParaId in decla_module (exported type)
* AccountParaId -> AccountId
* provide through inherent-data when authoring
* remove unneeded codec round-trip in proposer
* refactor polkadot-consensus service architecture
* integrate block authorship into polkadot service
* remove unused extern substrate-network crate in service
* write wrapper for unifying errors in consensus proposer
* extend wrapper further
* switch temporarily to macro-changing branch
* runtime compiles
* implement `inherent_extrinsics` for runtime
* block authorship works
* add GRANDPA to polkadot runtime
* get everything compiling
* use substrate master branch again
* remove some unneeded params
* update WASM
* parse only extrinsics when pruning availability store
* update recent deps
* runtime almost compiles
* need to expose trait type in build : I had to put phantomdata manually.
* finish updating authorship to latest GRANDPA and Aura
* fix tests
* update wasm
* multiple consensus sessions in network
* tests compile, add a test for RecentSessionKeys
* track recently received session keys from validators
* add a test for desired key-sending behavior
* statement table yields fully-attested candidates
* attestation types in polkadot-primitives
* propose block with fully-attested candidates in consensus
* some signature-checking logic in the runtime
* fix runtime compilation
* ensure attestations are full and without duplicate when checking
* fix consensus-service compilation
* add some tests
* use bitvec from crates.io now that it's published
* sign statements based on primitive statement's encoding
* remove some serialize bounds
* Fix error message with duplicate availability attestations
Co-Authored-By: rphmeier <rphmeier@gmail.com>
fix a deadlock when spawning agreement as non-authority
fix test compilation for BFT
more accurate consensus superseding logic
mild revision to `can_build_on` logic
block evaluation without redundant initialisation
refactor BFT delay: update rhododendron and poll after delaying. (#589)
dropping BFT future before poll doesn't lead to service deadlock
* notify when an authority appears to have missed their block
* Runtime API
* offline tracker
* Move to consensus
* generating reports of offline indices
* stubbed-out evaluation logic
* Slashing data pathwat
* usize -> u32
* Slash bad validators.
* update to rhododendron 0.3
* fix compilation of polkadot-consensus
* Support offline noting in checked_block
* include offline reports in block authorship voting
* do not vote validators offline after some time
* add test for offline-tracker
* fix test build
* bump spec version
* update wasm
* Only allow validators that are possible to slash
* Fix grumble
* More idiomatic
* New Wasm.
* update rhododendron
* improve logging and reduce round time exponent
* format offline validators in ss58
* update basic_add wasm
* wasm feature and collator feature
* move test parachains around a little
* fix wasm build for basic_add
* move basic_add to adder, introduce README
* minimal basic_add collator
* ensure collator messages are sent in the right order
* more logging
* route consensus statements to all peers
* minor bugfixes for parachains
* genesis builder accounts for parachain heads
* fix parachains tests
* targets for txpool
* tweak runtime + collator
* fix version in adder-collator
* consistency for overflowing
* adjust comment
* fix stable test run
* remove dummy registration test
* final grumbles
* skeleton of collators object
* awaiting and handling collations. rename `collators` to CollationPool
* add some tests
* add tests
* implement Collators trait for ConsensusNetwork
* plug collators into main polkadot-network
* ignore collator role message
* add a couple more tests
* garbage collection for collations
* extract session-key tracking from consensus
* add local_collations.rs
* finish polish of local_collations
* integrate local_collations into network layer
* introduce API for adding local collations
* mostly finish collator implementation pending service fix
* Specialized network()
* push collations to the network
* grumbles
* substrate-service has custom configuration
* initialize network in collator mode as necessary
* skeleton of collators object
* awaiting and handling collations. rename `collators` to CollationPool
* add some tests
* add tests
* implement Collators trait for ConsensusNetwork
* plug collators into main polkadot-network
* ignore collator role message
* add a couple more tests
* garbage collection for collations
* ensure disconnected backup collator is removed from pool
* address other grumbles
* arbitrary application logic in CLI
* collation work
* split up exit and work futures in application
* collation node workflow
* typo
* indentation fix
* doc grumbles
* rename Application to Worker
* refactor Worker::exit to exit_only
* dynamic inclusion threshold calculator
* collators interface
* collation helpers
* initial proposal-creation future
* create proposer when asked to propose
* remove local_availability duty
* statement table tracks includable parachain count
* beginnings of timing future
* finish proposal logic
* remove stray println
* extract shared table to separate module
* change ordering
* includability tracking
* fix doc
* initial changes to parachains module
* initialise dummy block before API calls
* give polkadot control over round proposer based on random seed
* propose only after enough candidates
* flesh out parachains module a bit more
* set_heads
* actually introduce set_heads to runtime
* update block_builder to accept parachains
* split block validity errors from real errors in evaluation
* update WASM runtimes
* polkadot-api methods for parachains additions
* delay evaluation until candidates are ready
* comments
* fix dynamic inclusion with zero initial
* test for includability tracker
* wasm validation of parachain candidates
* move primitives to primitives crate
* remove runtime-std dependency from codec
* adjust doc
* polkadot-parachain-primitives
* kill legacy polkadot-validator crate
* basic-add test chain
* test for basic_add parachain
* move to test-chains dir
* use wasm-build
* new wasm directory layout
* reorganize a bit more
* Fix for rh-minimal-parachain (#141)
* Remove extern "C"
We already encountered such behavior (bug?) in pwasm-std, I believe.
* Fix `panic_fmt` signature by adding `_col`
Wrong `panic_fmt` signature can inhibit some optimizations in LTO mode.
* Add linker flags and use wasm-gc in build script
Pass --import-memory to LLD to emit wasm binary with imported memory.
Also use wasm-gc instead of wasm-build.
* Fix effective_max.
I'm not sure why it was the way it was actually.
* Recompile wasm.
* Fix indent
* more basic_add tests
* validate parachain WASM
* produce statements on receiving statements
* tests for reactive statement production
* fix build
* add OOM lang item to runtime-io
* use dynamic_inclusion when evaluating as well
* fix update_includable_count
* remove dead code
* grumbles
* actually defer round_proposer logic
* update wasm
* address a few more grumbles
* schedule collation work as soon as BFT is started
* impl future in collator
* fix comment
* governance proposals for adding and removing parachains
* bump protocol version
* tear out polkadot-specific pieces of substrate-network
* extract out polkadot-specific stuff from substrate-network
* begin polkadot network subsystem
* grumbles
* update WASM checkins
* parse status from polkadot peer
* allow invoke of network specialization
* begin statement router implementation
* remove dependency on tokio-timer
* fix sanity check and have proposer factory create communication streams
* pull out statement routing from consensus library
* fix comments
* adjust typedefs
* extract consensus_gossip out of main network protocol handler
* port substrate-bft to new tokio
* port polkadot-consensus to new tokio
* fix typo
* start message processing task
* initial consensus network implementation
* remove known tracking from statement-table crate
* extract router into separate module
* defer statements until later
* double signature is invalid
* propagating statements
* grumbles
* request block data
* fix compilation
* embed new consensus network into service
* port demo CLI to tokio
* all test crates compile
* some tests for fetching block data
* whitespace
* adjusting some tokio stuff
* update exit-future
* remove overly noisy warning
* clean up collation work a bit
* address review grumbles
* fix lock order in protocol handler
* rebuild wasm artifacts
* tag AuthorityId::from_slice for std only
* address formatting grumbles
* rename event_loop to executor
* some more docs for polkadot-network crate
* dynamic inclusion threshold calculator
* collators interface
* collation helpers
* initial proposal-creation future
* create proposer when asked to propose
* remove local_availability duty
* statement table tracks includable parachain count
* beginnings of timing future
* finish proposal logic
* remove stray println
* extract shared table to separate module
* change ordering
* includability tracking
* fix doc
* initial changes to parachains module
* initialise dummy block before API calls
* give polkadot control over round proposer based on random seed
* propose only after enough candidates
* flesh out parachains module a bit more
* set_heads
* actually introduce set_heads to runtime
* update block_builder to accept parachains
* split block validity errors from real errors in evaluation
* update WASM runtimes
* polkadot-api methods for parachains additions
* delay evaluation until candidates are ready
* comments
* fix dynamic inclusion with zero initial
* test for includability tracker
* wasm validation of parachain candidates
* move primitives to primitives crate
* remove runtime-std dependency from codec
* adjust doc
* polkadot-parachain-primitives
* kill legacy polkadot-validator crate
* basic-add test chain
* test for basic_add parachain
* move to test-chains dir
* use wasm-build
* new wasm directory layout
* reorganize a bit more
* Fix for rh-minimal-parachain (#141)
* Remove extern "C"
We already encountered such behavior (bug?) in pwasm-std, I believe.
* Fix `panic_fmt` signature by adding `_col`
Wrong `panic_fmt` signature can inhibit some optimizations in LTO mode.
* Add linker flags and use wasm-gc in build script
Pass --import-memory to LLD to emit wasm binary with imported memory.
Also use wasm-gc instead of wasm-build.
* Fix effective_max.
I'm not sure why it was the way it was actually.
* Recompile wasm.
* Fix indent
* more basic_add tests
* validate parachain WASM
* produce statements on receiving statements
* tests for reactive statement production
* fix build
* add OOM lang item to runtime-io
* use dynamic_inclusion when evaluating as well
* fix update_includable_count
* remove dead code
* grumbles
* actually defer round_proposer logic
* update wasm
* address a few more grumbles
* grumbles
* update WASM checkins
* remove dependency on tokio-timer
* move responsibility of storage_root calculation to state backend
* have `storage_root` produce a memoizable transaction
* store trie nodes in kvdb
* fix up test fallout
* remove stray newline
* Fix comment
* test for setting and checking state data
* fiddle with dependencies
* all parity deps on same commit hash
* fix network protocol registration
* call `on_new_transactions` when we import
* fix trace
* pass correct bytes to network
* clean up
* cull before repropagating; repropagate on timer
* add a little tracing
* barrier on starting network
* handle exit better
* give consensus service its own internal exit signal
* update comment
* remove stop_notifications and fix build
* double-commit and prepare misbehavior
* get misbehavior on completion
* collect misbehavior on drop, not only on success
* kill unused transaction_index field
* add primitive misbehavior report type
* add misbehavior report transaction
* store prior session
* fix set_items
* basic checks for misbehavior reports
* crate for substrate bft misbehavior checking
* integrate misbehavior check crate
* fix comment
* new wasm binaries
* fix hash in test
* import misbehavior transactions into queue
* fix test build
* sign on digest and full proposal when proposing
* detect proposal misbehavior
* fix fallout
* restore balance/bondage types
* reshuffle consensus libraries
* polkadot-useful type definitions for statement table
* begin BftService
* primary selection logic
* bft service implementation without I/O
* extract out `BlockImport` trait
* allow bft primitives to compile on wasm
* Block builder (substrate)
* take polkadot-consensus down to the core.
* test for preemption
* fix test build
* Fix wasm build
* Bulid on any block
* Test for block builder.
* Block import tests for client.
* Tidy ups
* clean up block builder instantiation
* justification verification logic
* JustifiedHeader and import
* Propert block generation for tests
* network and tablerouter trait
* use statement import to drive creation of further statements
* Fixed rpc tests
* custom error type for consensus
* create proposer
* asynchronous proposal evaluation
* inherent transactions in polkadot runtime
* fix tests to match real polkadot block constraints
* implicitly generate inherent functions
* add inherent transaction functionality to block body
* block builder logic for polkadot
* some tests for the polkadot API
* avoid redundancy in native code compatibility check
* helper for extracting nonce
* transaction pool implementation
* transaction pool
* integrate transaction pool with proposer
* indentation
* kill storage keys module
* accept new transactions to replace old
* reshuffle consensus libraries
* polkadot-useful type definitions for statement table
* begin BftService
* primary selection logic
* bft service implementation without I/O
* extract out `BlockImport` trait
* allow bft primitives to compile on wasm
* Block builder (substrate)
* take polkadot-consensus down to the core.
* test for preemption
* fix test build
* Fix wasm build
* Bulid on any block
* Test for block builder.
* Block import tests for client.
* Tidy ups
* clean up block builder instantiation
* justification verification logic
* JustifiedHeader and import
* Propert block generation for tests
* network and tablerouter trait
* use statement import to drive creation of further statements
* Fixed rpc tests
* custom error type for consensus
* create proposer
* asynchronous proposal evaluation
* inherent transactions in polkadot runtime
* fix tests to match real polkadot block constraints
* implicitly generate inherent functions
* add inherent transaction functionality to block body
* block builder logic for polkadot
* some tests for the polkadot API
Robert Habermeier