Commit Graph

56 Commits

Author SHA1 Message Date
Bastian Köcher 934f27d92b Downward & Upward messages (#1266)
* Downward messages, the front-end.

* Move types around to make them accessible from Parachains

* Fix compilation

* Fix branch

* Make it compile for Cumulus

* Update the branch names

* Add default generic parameter

* Implement `Partialeq`

* Move upward messages into the `ValidationResult`

* Support disabling of the runtime api

* Update branch

* Adds support for handling downward messages

* Implement sending XCMP messages as up/downward messages

* service: update to latest ServiceBuilder changes

* Make it compile

* Initial commit

Forked at: ef2aa428d7
Parent branch: origin/master

* Update substrate branch to cecton-update-polkadot-substrate

* Update substrate & polkadot to cumulus-branch

* Reset branch

* Update primitives/src/parachain.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Update runtime/common/src/parachains.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Update runtime/common/src/parachains.rs

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>

* Minor fixes

* Fix wasm build

Co-authored-by: Gav Wood <gavin@parity.io>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2020-07-01 10:20:38 -04:00
Shawn Tabrizi 1e86cc30a5 Companion PR for #6386 (Read/Write Tracking) (#1284)
* add new apis to externalities

* Update mod.rs

* update for new host function

* Rename unused

* cargo update -p sp-io

* Add whitelists

* Apply changes to polkadot runtime as well

Co-authored-by: kianenigma <kian.peymani@gmail.com>
2020-06-25 10:04:34 +02:00
Alexander Theißen 8fb1e2547c Companion for Substrate #6269 (nested storage tx) (#1210)
* Companion for Substrate #6269 (nested storage tx)

* Adjust to PR changes

* Bump to newest substrate

* Bump spec
2020-06-23 12:50:29 +02:00
Pierre Krieger 626a4aeccf Some fixes to compile for Android (#1063)
* Some fixes to compile for Android

* Revert change to cli
2020-05-04 16:04:33 +02:00
Nikolay Volf 59f5743852 Companion PR for 5795 (#1041)
* add storage_append

* Bump versions

Co-authored-by: Gav Wood <gavin@parity.io>
2020-04-27 18:23:09 +02:00
Bernhard Schuster 8151b175bc Companion PR for #5200 (#1025) 2020-04-24 16:48:01 +02:00
Bastian Köcher a8be8e01fb Companion pr for Substrate #5741 (#1026)
* Companion pr for Substrate #5741

* update to substrate master

Co-authored-by: André Silva <andre.beat@gmail.com>
2020-04-23 18:07:11 +01:00
Bastian Köcher ed9c8ce662 Fix extensions handling for Parachain validation (#1015)
The `TaskExecutor` extension is now required by Substrate to batch verify signatures.

Co-authored-by: NikVolf <nikvolf@gmail.com>
2020-04-21 19:59:59 +03:00
cheme 1a4db93c50 Companion to child trie api change #4857 (#950) 2020-04-21 10:39:35 +02:00
Nikolay Volf b50d2f2d1f Companion PR for 5023 (#939)
* add extension registrations

* update to latest

* merge&update
2020-04-16 22:58:16 +02:00
Robert Habermeier a5034dbe98 Use strong types in runtime for parachain heads and validation code (#964)
* use stronger types for HeadData and ValidationCode in runtime

* fix weird debug compile error

* fix runtime build

* update invocations invalidation.rs

* fix tests
2020-04-13 10:24:25 +02:00
Robert Habermeier 10cec3b591 Upgradeable validation functions (#918)
* upgrade primitives to allow changing validation function

* set up storage schema for old parachains code

* fix compilation errors

* fix test compilation

* add some tests for past code meta

* most of the runtime logic for code upgrades

* implement old-code pruning

* add a couple tests

* clean up remaining TODOs

* add a whole bunch of tests for runtime functionality

* remove unused function

* fix runtime compilation

* extract some primitives to parachain crate

* add validation-code upgrades to validation params and result

* extend validation params with code upgrade fields

* provide maximums to validation params

* port test-parachains

* add a code-upgrader test-parachain and tests

* fix collator tests

* move test-parachains to own folder to work around compilation errors

* fix test compilation

* update the Cargo.lock

* fix parachains tests

* remove dbg! invocation

* use new pool in code-upgrader

* bump lockfile

* link TODO to issue
2020-04-06 10:43:19 -04:00
Robert Habermeier 15a83079ba refactor out validation hosts to pool struct (#972)
* refactor out validation hosts to pool struct

* make web-wasm compatible

* typo

* remove now-unused static hosts
2020-04-04 15:15:54 -04:00
Gavin Wood d8e1180030 Companion for #5412 (#942)
* Fixes for democracy using Scheduler

* Revert branch change
2020-04-01 17:01:14 +02:00
Gavin Wood 1ddfb5c4e1 Bump to latest Substrate (#898)
* Flag to force kusama runtime

* Chainspecs for kusama

* Polkadot config for westend

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* network/src/legacy/gossip: Wrap GossipEngine in Arc Mutex & lock it on use

`GossipEngine` in itself has no need to be Send and Sync, given that it
does not rely on separately spawned background tasks anymore.
`RegisteredMessageValidator` needs to be `Send` and `Sync` due to the
inherited trait bounds from implementing `GossipService`. In addition
`RegisteredMessageValidator` derives `Clone`. Thereby `GossipEngine`
needs to be wrapped in an `Arc` and `Mutex` to keep the status quo.

* Needed fixes.

* Fixes

* Fixed build

* Fixed build w benchmarking CLI

* Fixed building tests

* Added --dev shortcut

Co-authored-by: arkpar <arkady.paronyan@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Max Inden <mail@max-inden.de>
2020-03-13 14:43:31 +01:00
Shawn Tabrizi 70a5bda2ce Benchmark Polkadot Claims Pallet (#876)
* fix

* Starting to add benchmarks

* make compile

* add benchmarks

* Make work with Substrate master

* Bench validate unsigned

* back to polkadot master

* starting to add cli with feature flag

* more stuff

* Add to kusama

* Update Cargo.lock

* fix dev dep

* bump wasm builder

* Remove encode from keccak benchmark

* bump spec

* Add weight documentation

* Update Cargo.lock

* Update check_runtime.sh

* Update publish_draft_release.sh

* Update Cargo.lock

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
2020-03-13 10:42:31 +01:00
Gavin Wood e13fdc884c Merged companions and update Subtrate (#882)
* expunge legacy code from polkadot-network

* mostly rip out old legacy protocol from service

* ensure validation work is spawned by incoming messages

* decouple availabliity store from network logic; clean up data flow

* av_store: test helpers and use futures-abort

* update polkadot-validation to pass n_validators when submitting chunks

* fallible erasure-chunk fetching

* implement `ErasureNetworking` for new network prot

* API for registering availability store in network

* fully integrate new network service into service

* fix validation tests

* scaffolding for porting collator over to new network

* track connected validators' peer IDs and distribute collators' collations

* helper in network for fetching all checked statements

* fix adder-collator

* actually register notifications protocol

* Update service/src/lib.rs

* Make needed changes to service

* Merge two companion PRs.

- #880
- #881

* Some effort towards compilation

* Fix

* remove `NetworkSpecialization` references from network

* fix compilation errors in service and collator

* ensure protocol name is valid

* Fixes

* Fix

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
2020-03-05 23:20:42 +01:00
Gavin Wood b2df51d296 Update substrate (#878)
* Switch branch

* Small changes

* Update substrate branch

* Switch

* Revert "Switch branch"

This reverts commit b9d48b2ce8f5cbfa379dd385e817e80870391d9d.

* fix

* add `wipe` and `commit`

* Remove deprecated_host_interface

* Switch branch

* HasherFor -> HashFor

* More HasherFor changes

* Final touches

* Revert "Switch branch"

This reverts commit d0da27313839559de01f59690f3826fe587becb8.

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-03-05 13:27:28 +01:00
Robert Habermeier b7d30aa379 A more comprehensive model for PoV-Blocks and Candidate receipts (#843)
* encode the candidate statement as only the hash

* refactor CandidateReceipt and CollationInfo

* introduce an abridged candidate receipt type

* erasure coding stores candidate receipt

* store omitted data instead and introduce AvailableData type

* refactor availability-store schema

* tweak schema and APIs a bit more

* get availability-store tests passing

* accept AbridgedCandidateReceipt in `set_heads`

* change statement type in primitives to be hash-only

* fix parachains runtime tests

* fix bad merge

* rewrite validation pipeline

* remove evaluation module

* use abridged candidate hash as canonical

* statement table uses abridged candidate receipts

* kill availability_store::Data struct

* port shared table to new validation pipelines

* extract full validation pipeline to helper

* remove old validation pipeline from collation module

* polkadot-validation compiles

* polkadot-validation tests compile

* make local collation available in validation service

* port legacy network code

* polkadot-network fully ported

* network: ensure fresh statement is propagated

* remove pov_block_hash from LocalValidationData

* remove candidate_hash field from AttestedCandidate and update runtime

* port runtimes to new ParachainHost definition

* port over polkadot-collator

* fix test compilation

* better fix

* remove unrelated validation work dispatch fix

* address grumbles

* fix equality check
2020-02-25 15:16:58 -08:00
Ashley bbb2fbc556 Remove TargetedMessage (#848)
* Remove TargetedMessage

* Nitpicks
2020-02-19 08:33:38 -08:00
Ashley 5f9e602af7 Strip out old XCMP primitives (#823)
* WIP

* WIp

* Mostly get tests to compile

* Fix adder collator

* Remove more stuff

* Revert some changes to av store

* Fix av store tests

* Nitpicks

* Restore some things

* Small changes

* Remvoe unused error variants
2020-02-13 17:12:05 +01:00
Bastian Köcher 2eb7621759 Adds some debug logging (#837) 2020-02-11 22:01:45 +01:00
André Silva 2b081f9edd v0.17.18: Update latest substrate (#776)
* update latest substrate polkadot-master

* fix test compilation

* bump version to 0.7.18

* bump impl_version

* update substrate

* Revert "Instantiate environment with asynchronous API (#768)"

This reverts commit a5d9645bf4.

* update substrate

* remove unused parameter type

* bump trie-db version for tests

* fix collator test

* update substrate

* remove unnecessary service changes
2020-01-17 18:37:55 +00:00
Arkadiy Paronyan 95afc7c2fd Remote execution with additional logging (#767) 2020-01-15 16:18:59 +01:00
Bastian Köcher 9ec65c94d7 Some fixes required for Cumulus (#766)
* Add some more bounds

* More bounds

* More fixes

* More fixes
2020-01-15 14:17:55 +01:00
Gavin Wood ddce5fe496 Bump Substrate & versions (#747)
* Bump versions

* Update for tipping treasury

* Bump substrate

* Fixes

* Put send_consensus back in.

* Fix test

* Fixes

* Fixes

* Fix warning
2020-01-10 00:23:43 +01:00
Bastian Köcher 06386558ae Switch parachain interface to new runtime_interface macro (#665)
* Make use of `runtime_interface` for parachain externalities

This also changes the encoding of the `ValidationResult` return value to
match the default encoding used in Substrate.

* Fix compilation for web

* Update `Cargo.lock`

* Include feedback

* Move proc macro

* Update parachain/src/lib.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

Co-authored-by: Robert Habermeier <rphmeier@gmail.com>
2020-01-08 20:44:50 +01:00
Shawn Tabrizi c975d4ee84 Update copyright year (#718) 2020-01-05 17:57:13 +01:00
Benjamin Kampmann 7832ad93cd fix for latest substrate crate renaming (#641)
* rename crates as appropriate

* Rename to use master - easy to revert, but shows it's working

* Bump runtime version

* Bump version

* Bump Substrate again
2019-12-02 16:56:50 +01:00
André Silva dd2b43cd10 Update to latest substrate master (#491)
* update to latest substrate master

* Fix compilation
2019-10-23 17:17:49 +02:00
Bastian Köcher 2842be7b81 Make ParaId constructible from a const context (#483) 2019-10-19 13:09:21 +02:00
yjh 78fc9edd5f fix var name for post_upward_message (#474)
Signed-off-by: yjhmelody <465402634@qq.com>
2019-10-11 13:25:36 +02:00
Gavin Wood b1558157cb Introduce Parathreads (runtime) (#341)
* Rest of parathread draft implementation, parachain permissioning.

* Update Substrate

* Update Substrate again

* Integrate weight/fee stuff.

* Council

* Build fixes

* More fixes

* Minor additions

* fix some small errors

* Revert "fix some small errors"

This reverts commit 4fb52c82adfdaf3af98edfe36b280133bcd4f9d3.

* Merge fix.

* do_swap -> on_swap

* Update depdendency to polkadot-master

* Fix more merge problems

* Some patching of errors

* Fix storage closure

* Actually fix storage. It builds!

* Tests run... but not successfully.

* Add `run_to_block` to get parachains active to start

* More `run_to_block`

* Fix build

* Queue up changes to threads

* Move registration test

* Fix regsiter/deregister test

* Retry queue.

* Minor refactor

* Refactor to avoid heavy storage items

* Make tests pass

* remove para on deregister, add events

* Remove println

* Fix register/deregister parathread test

* fix merge

* Parathread can be activated test

* Test auction

* Add `Debtors` storage item

I considered putting the debtor information in `ParaInfo`, but it did not make sense to me since this information only applies to parathreads, not `paras` in general.

* remove comment code

* Some new tests

* Fixes for removing threads when scheduled. Tests.

* Test progression of threads.

* Test that reschedule queuing works properly.

* Make test slightly more interesting

* whitespace

* Swap works properly.

* Update locks

* Build

* Rename can_swap

* Add test for funds to be correctly returned after a swap

Swap does not seem to have logic which correctly swaps the debtor account to the new parathread.

* Make tests consistant

* Add check that `PendingSwap` is cleaned up

* Update runtime/src/parachains.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Update runtime/src/registrar.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Some fixes/suggestions from review

* Docs

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>
Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update network/src/gossip.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Rename OnSwap

* Add missing `]`

* Rejig ordering semantics, making everything a bit slower but correct.

* Some Fixes to Parathread Compile (#470)

* Some Fixes

* Fix queue_upward_messages

* Change back to const

* Build fixes

* Fix tests
2019-10-11 08:27:30 +02:00
yjh ecfea4536f wasm_executor: fix wasm signature checker (#471)
Signed-off-by: yjhmelody <465402634@qq.com>
2019-10-10 13:49:25 +02:00
Bastian Köcher dd0009a006 Fix compilation in wasm (#465)
Also fix the weird file structure by making `wasm_executor.rs` ->
`wasm_executor/mod.rs`.
2019-10-08 17:40:48 +02:00
郭光华 512ca35c62 Typo (#417) 2019-09-03 10:29:09 +02:00
Arkadiy Paronyan 908473c26b Allow parallel parachain validation (#370)
* Allow parallel parachain validation

* Fixed test interference

* Switch to pooled implementation

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Update validation_host.rs

* Minor cleanups

* Fixed build
2019-08-24 11:52:00 +02:00
Kian Paimani 10fc88f6b1 Update to latest Substrate master. (#353)
* Integrate srml/im-online

* Fix all build errors with old aura.

* Fix most of the build errors.

* Builds and tests seem to pass (I will not trust this commit yet)

* Apply suggestions from code review

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Kill some warnings.

* fix panics on 0 validators

* Fix dev chain.

* Fix author stuff

* fix im online integration.

* Some tweaks

* Introduce app-crypto

* Initial build work

* codec update / tweaks

* patch polkadot-erasure-coding input

* More fixes for new crypto

* More fixes

* Update parachains module

* evamp parachain crypto

* More crypto work.

* Chain spec and service.

* ChainSpec stuff

* Last bits for a clean build

* Tweak coment

* adapt polkadot-validation to the new keystore

* polkadot-network compiles, but tests don't

* Integrate the new parachain validation stuff

* delete message_routing file

* make polkadot-network tests compile and pass

* runtime tests compile and pass

* update substrate ref

* service compiles

* all tests pass

* Add TODO, change branch back to polkadot-master

* Lock file

* TODOs done

* Issue number

* Remove old tODO

* Remove commented code
2019-08-12 15:48:29 +02:00
Pierre Krieger 56845f74b5 Make polkadot-parachain compile for WASM (#350) 2019-08-06 01:07:24 +02:00
Arkadiy Paronyan f1fdb0cb83 Parachain validation moved to external process (#325)
* Improved execution & tests

* Style

* Made CLI arg const

* Moved Upwards message

* CLI subcommand for validation worker

* Build halting parachain

* Build halting parachain

* Made stuff private

* Reorganized parachain tests

* Comment

* Whitespace

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Fixed call data size check and introduced an enum

* Apply suggestions from code review

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>
2019-07-23 11:09:30 +02:00
Bastian Köcher 9004fb3f97 Make polkadot-parachain call validate_block instead of validate (#297)
* Make `polkadot-parachain` call `validate_block` instead of `validate`

Also switch to rust 2018 in the crate

* Use `rstd`

* Make `load_params` a pointer
2019-06-25 21:08:56 +02:00
Andrew Jones bdc1502411 Remove dependency on error_chain (#277)
* Convert validation error

* Convert wasm_executor error

* Convert block evaluation error

* Convert collation errors and the compilation

* Remove error-chain dep from service

* Remove unused Result type

* Remove unused error variants

* Remove redundant intos

* Add missing comments

* Update validation/src/collation.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Fix new error variant
2019-06-05 10:25:08 +02:00
Gavin Wood 4b7dfc4c25 Allow parachains to send messages (#274)
* Slots module

* Integrate slots

* More drafting

* Minor updates

* Update parachains to use trati

* More build fixes

* Full code now compiles

* Add renew bid function

* Implement calculate_winner

* Warning remove

* Update gitignore

* Test framework

* Tests

* Further testing

* More tests, new parameterisation.

* Fix and new test

* Thread-safe tests

* Test off-boarding and a fix.

* Test onboarding

* Allow late onboarding.

* Another test and fix

* Avoid println in nostd

* Compact representation of paraids

* Introduce documentation.

* Introduce events.

* Additional test and fix

* Additional test

* Tidy up line lengths.

* Remove printlns

* Use later substrate utils.

* Allow parachains to send messages.

* Fix build/test

* Make slots work with latest substrate

* Update runtime/src/slot_range.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Polish logic

* Rewind to earlier substrate master

* Remove dead code.

* Fix build

* Update substrate ref to master

* Update to new inherent digests API

* address grumbles

* fix

* Fix a warning.

* Reworded a comment.

* Check that receipt matches expectations

* Add test for final checks

* Split out queuing logic.

* Test final piece of queuing logic

* Fix up docs.

* More docs fixes
2019-06-03 16:48:33 +02:00
Gavin Wood 32f9519743 Parachain auctions (#239)
* Slots module

* Integrate slots

* More drafting

* Minor updates

* Update parachains to use trati

* More build fixes

* Full code now compiles

* Add renew bid function

* Implement calculate_winner

* Warning remove

* Update gitignore

* Test framework

* Tests

* Further testing

* More tests, new parameterisation.

* Fix and new test

* Thread-safe tests

* Test off-boarding and a fix.

* Test onboarding

* Allow late onboarding.

* Another test and fix

* Avoid println in nostd

* Compact representation of paraids

* Introduce documentation.

* Introduce events.

* Additional test and fix

* Additional test

* Tidy up line lengths.

* Remove printlns

* Use later substrate utils.

* Fix build/test

* Make slots work with latest substrate

* Update runtime/src/slot_range.rs

Co-Authored-By: Robert Habermeier <rphmeier@gmail.com>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Update runtime/src/slots.rs

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* Polish logic

* Rewind to earlier substrate master

* Remove dead code.
2019-05-25 13:34:23 +02:00
Robert Habermeier 1437c8e224 Introduce a Proof-of-Validation block type and use that in place of BlockData (#227)
* validators expect collators to give them parachain messages

* mostly port network to use pov_block

* network tests pass

* verify ingress when fetching pov block

* fix runtime compilation

* all tests build

* fix some grumbles

* Update validation/src/collation.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update primitives/src/parachain.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>

* Update network/src/lib.rs

Co-Authored-By: rphmeier <rphmeier@gmail.com>
2019-04-24 13:24:03 +02:00
joe petrowski 2bbfa0ae98 Create issues for all TODOs/FIXMEs (#224)
* network and validation

* remainder of todos

* ref new issue

* change issue number
2019-04-08 17:46:20 +02:00
Robert Habermeier 4ef53912e6 define Id type in polkadot-parachain and depend on that in primitives. (#144)
* define Id type in polkadot-parachain and depend on that in primitives.

* fix tests
2019-02-20 12:33:52 +01:00
Robert Habermeier f8af277006 Interchain message-passing (#117)
* compute ingress and routing in polkadot runtime

* extract parent candidates from block when beginning consensus

* fetch incoming messages when validating

* fix consensus tests

* parachain wasm execution uses messages

* update parachain tests to check if messages are executed

* abstract out network service to make room for network tests

* skeleton for incoming data fetch

* collate ingress from consensus-gossip

* keep track of validated candidates in the shared-table

* add some shared_table tests for new behavior

* broadcast egress messages on gossip

* test compute_ingress

* move network tests to module folder

* dummy network for consensus-network tests

* make consensus network generic over executor

* test egress broadcast and ingress fetch

* fix test compilation

* address some grumbles

* address grumbles and fix parachain shuffle

* remove broadcast parameter from consensus network trait
2019-02-19 13:59:29 -03:00
Sergei Pepyakin 31a3a2e4df Add comment about guarantees (#105) 2019-01-23 14:21:54 +01:00
Robert Habermeier fe6351ca65 Parachain execution yields messages to send (#96)
* read head-data directly out of WASM memory

* implement ext_post_message for parachain WASM

* further refactoring of the parachain module

* add externalities error type

* accumulate posted messages when validating parachain candidate

* define Extrinsic type in primitives

* availability-store: store extrinsic data

* compute extrinsic and check against candidate

* add some egress queue tests

* grumbles & substrate update

* ensure everything builds
2019-01-22 12:32:32 +01:00