1790d0ae36
Bump parity-scale-codec to 3.3.0 ( #6666 )
2023-02-03 14:03:38 +00:00
1a1bfd2af9
XCM v3 ( #4097 )
...
* cargo fmt
* Create benchmarks for XCM instructions introduced in v3 (#4564 )
* Create benchmarks for BurnAsset and ExpectAsset
* Add benchmarks for ExpectOrigin and ExpectError
* Add benchmarks for QueryPallet and ExpectPallet
* Add benchmarks for ReportTransactStatus and ClearTransactStatus
* cargo fmt
* Use AllPalletsWithSystem in mocks
* Update XCM generic benchmarks for westend
* Remove default impls for some XCM weight functions
* Fix compilation error
* Add weight_args helper attribute
* Remove manually written XcmWeightInfo
* Parse trailing comma
* Revert "Add weight_args helper attribute"
This reverts commit 3b7c47a6182e1b9227036c38b406d494c3fcf6fd.
* Fixes
* Fixes
* XCM v3: Introduce querier field into `QueryReponse` (#4732 )
* Introduce querier field into QueryReponse
* Convert &Option<MultiLocation> to Option<&MultiLocation>
&Option<T> is almost always never quite useful, most of the time it
still gets converted to an Option<&T> via `as_ref`, so we should simply
make functions that accept Option<&T> instead.
* Fix tests
* cargo fmt
* Fix benchmarks
* Appease spellchecker
* Fix test
* Fix tests
* Fix test
* Fix mock
* Fixes
* Fix tests
* Add test for response queriers
* Update xcm/pallet-xcm/src/lib.rs
* Test for non-existence of querier
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* Fixes
* Fixes
* Add `starts_with` function to `MultiLocation` and `Junctions` (#4835 )
* add matches_prefix function to MultiLocation and Junctions
* rename matches_prefix to starts_with
* remove unnecessary main in doc comment
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com >
* Make use of starts_with in match_and_split
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com >
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* XCM v3: Bridge infrastructure (#4681 )
* XCM bridge infrastructure
* Missing bit of cherry-pick
* Revamped XCM proc macros; new NetworkIds
* Fixes
* Formatting
* ExportMessage instruction and config type
* Add MessageExporter definitions
* Formatting
* Missing files
* Fixes
* Initial bridging config API
* Allow for two-stage XCM execution
* Update xcm/src/v3/mod.rs
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* XCM crate building again
* Initial bridging primitive
* Docs
* Docs
* More work
* More work
* Merge branch 'gav-xcm-v3' into gav-xcm-v3-bridging
* Make build
* WithComputedOrigin and SovereignPaidRemoteExporter
* Remove TODOs
* Slim bridge API and tests.
* Fixes
* More work
* First bridge test passing
* Formatting
* Another test
* Next round of bridging tests
* Repot tests
* Cleanups
* Paid bridging
* Formatting
* Tests
* Spelling
* Formatting
* Fees and refactoring
* Fixes
* Formatting
* Refactor SendXcm to become two-phase
* Fix tests
* Refactoring of SendXcm and ExportXcm complete
* Formatting
* Rename CannotReachDestination -> NotApplicable
* Remove XCM v0
* Minor grumbles
* Formatting
* Formatting
* Fixes
* Fixes
* Cleanup XCM config
* Fee handling
* Fixes
* Formatting
* Fixes
* Bump
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* Bump Substrate
* XCM v3: `ExchangeAsset` and Remote-locking (#4945 )
* Asset Exchange and Locks
* Make sure XCM typers impl MaxEncodedLen
* Basic implementation for locks
* Bump Substrate
* Missing files
* Use new API
* Introduce instruction
* Big refactor
* Docs
* Remove deprecated struct
* Remove deprecated struct
* Repot XCM builder tests
* ExchangeAsset test
* Exchange tests
* Locking tests
* Locking tests
* Fixes and tests
* Fixes
* Formatting
* Spelling
* Add simulator test for remote locking
* Fix tests
* Bump
* XCM v3: Support for non-fungibles (#4950 )
* NFT support and a test
* New files.
* Integration tests for sending NFTs
* Formatting
* Broken Cargo features
* Use 2021 edition
* Fixes
* Formatting
* Formatting
* Update xcm/xcm-builder/src/asset_conversion.rs
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* Update xcm/xcm-builder/src/nonfungibles_adapter.rs
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* Update xcm/xcm-executor/src/lib.rs
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* Fixes
* Fixes
* Fixes
* Formatting
* Fixes
Co-authored-by: Bastian Köcher <info@kchr.de >
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* XCM v3: Context & ID hash (#4756 )
* send_xcm returns message hash
* cargo fmt
* Create topic register and instructions
* Fix weights
* Use tabs
* Sketch out XcmContext
* Fix doc test
* Add the XCM context as a parameter to executor trait fns
* Fixes
* Add XcmContext parameter
* Revert adding context as an arg to SendXcm trait methods
* Revert adding context argument to ConvertOrigin trait methods
* cargo fmt
* Do not change the API of XcmExecutor::execute
* Fixes
* Fixes
* Fixes
* Fixes
* Remove convenience method
* Fixes
* Fixes
* cargo fmt
* Fixes
* Add benchmarks for XCM topic instructions
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs
* Remove context argument on FilterAssetLocation
* Fixes
* Remove unused import
* Fixes
* Fixes
* Fixes
* Accept XCM hash parameter in ExecuteXcm trait methods
* cargo fmt
* Properly enable sp-io/std
* Fixes
* default-features = false
* Fixes
* Fixes
* Fixes
* Make XcmContext optional in withdraw_asset
* Fixes
* Fixes
* Fixes
* Modify tests to check for the correct XCM hash
* Small refactor
* cargo fmt
* Check for expected hash in xcm-builder unit tests
* Add doc comment for the optionality of the XCM context in withdraw_asset
* Update xcm/src/v3/traits.rs
* Update xcm/src/v3/traits.rs
* Store XcmContext and avoid rebuilding
* Use ref for XcmContext
* Formatting
* Fix incorrect hash CC @KiChjang
* Refactor and make clear fake hashes
* Fixes
* Fixes
* Fixes
* Fix broken hashing
* Docs
* Fixes
* Fixes
* Fixes
* Formatting
* Fixes
* Fixes
* Fixes
* Remove unknowable hash
* Formatting
* Use message hash for greater identifiability
* Formatting
* Fixes
* Formatting
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
Co-authored-by: Parity Bot <admin@parity.io >
* Fixes
* Fixes
* Fixes
* Fixes
* Formatting
* Fixes
* Formatting
* Fixes
* Fixes
* Formatting
* Formatting
* Remove horrible names
* Bump
* Remove InvertLocation trait (#5092 )
* Remove InvertLocation trait
* Remove unneeded functions
* Formatting
* Fixes
* Remove XCMv1 (#5094 )
* Remove XCMv1
* Remove XCMv1
* Formatting
* Fixes
* Fixes
* Formatting
* derive serialize/deserialize for xcm primitives (#5036 )
* derive serialize/deserialize for xcm primitives
* derive serialize/deserialize for xcm primitives
* update v3
* update v2
Co-authored-by: Gav Wood <gavin@parity.io >
* Update lock
* Fixes
* Add benchmarks for the ExchangeAsset instruction
* `AliasOrigin` instruction stub (#5122 )
* AliasOrigin instruction stub
* Fixes
* Fixes
* Update substrate
* Fixes
* Ensure same array length before using copy_from_slice
* Fixes
* Add benchmarks for the UniversalOrigin instruction
* Remove unused import
* Remove unused import
* Add benchmarks for SetFeesMode instruction
* Add benchmarks for asset (un)locking instructions
* Leave AliasOrigin unbenchmarked
* Fixes after merge
* cargo fmt
* Fixes
* Fixes
* Set TrustedReserves to None on both Kusama and Westend
* Remove extraneous reserve_asset_deposited benchmark
* Fix universal_origin benchmark
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark pallet --chain=westend-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/westend/src/weights/xcm/pallet_xcm_benchmarks_generic.rs
* Don't rely on skipped benchmark functions
* Fixes
* cargo run --quiet --profile=production --features=runtime-benchmarks -- benchmark pallet --chain=kusama-dev --steps=50 --repeat=20 --pallet=pallet_xcm_benchmarks::generic --extrinsic=* --execution=wasm --wasm-execution=compiled --heap-pages=4096 --header=./file_header.txt --template=./xcm/pallet-xcm-benchmarks/template.hbs --output=./runtime/kusama/src/weights/xcm/pallet_xcm_benchmarks_generic.rs
* Fix unused variables
* Fixes
* Spelling
* Fixes
* Fix codec index of VersionedXcm
* Allows to customize how calls are dispatched from XCM (#5657 )
* CallDispatcher trait
* fmt
* unused import
* fix test-runtime
* remove JustDispatch type
* fix typo in test-runtime
* missing CallDispatcher
* more missing CallDispatcher
* Update comment `NoteAssetLocked` -> `NoteUnlockable`
* Fixes
* Fixes
* Adjust MultiAssets weights based on new wild card variants
* Fixes
* Fixes
* Fixes
* Fixes
* Fixes
* Some late fixes for XCMv3 (#5237 )
* Maximise chances that trapped assets can be reclaimed
* Do origin check as part of ExportMessage for security
* Formatting
* Fixes
* Cleanup export XCM APIs
* Formatting
* Update xcm/src/v3/junctions.rs
* UnpaidExecution instruction and associated barrier.
* Tighten barriers (ClearOrigin/QueryResponse)
* Allow only 1 ClearOrigin instruction in AllowTopLevelPaidExecutionFrom
* Bi-directional teleport accounting
* Revert other fix
* Build fixes]
* Tests build
* Benchmark fixes
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
* Update Substrate
* Re-export `pub` stuff from universal_exports.rs + removed unecessary clone (#6145 )
* Re-export `pub` stuff from universal_exports.rs
* Removed unnecessary clone
* Use 2D weights in XCM v3 (#6134 )
* Depend upon sp-core instead of sp-runtime
* Make sp-io a dev-dependency
* Use 2D weights in XCM v3
* cargo fmt
* Add XCM pallet migration to runtimes
* Use from_parts
* cargo fmt
* Fixes
* cargo fmt
* Remove XCMWeight import
* Fixes
* Fixes
* Fixes
* Fixes
* Use translate in migration
* Increase max upward message size in tests
* Fix doc test
* Remove most uses of from_ref_time
* cargo fmt
* Fixes
* Fixes
* Add extrinsic benchmarking to XCM pallet
* cargo fmt
* Fixes
* Use old syntax
* cargo fmt
* Fixes
* Remove hardcoded weights
* Add XCM pallet to benchmarks
* Use successful origin
* Fix weird type parameter compilation issue
* Fixes
* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
* Use benchmarked XCM pallet weights
* Fixes
* Fixes
* Use override instead of skip
* Fixes
* Fixes
* Fixes
* Fixes
* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
* Fixes
* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
Co-authored-by: command-bot <>
* Replace Weight::MAX with 100b weight units
* Add test to ensure all_gte in barriers is correct
* Update xcm/src/v3/junction.rs
Co-authored-by: asynchronous rob <rphmeier@gmail.com >
* Add more weight tests
* cargo fmt
* Create thread_local in XCM executor to limit recursion depth (#6304 )
* Create thread_local in XCM executor to limit recursion depth
* Add unit test for recursion limit
* Fix statefulness in tests
* Remove panic
* Use defer and environmental macro
* Fix the implementation
* Use nicer interface
* Change ThisNetwork to AnyNetwork
* Move recursion check up to top level
* cargo fmt
* Update comment
Co-authored-by: Bastian Köcher <info@kchr.de >
* Add upper limit on the number of overweight messages in the queue (#6298 )
* Add upper limit on the number of ovwerweight messages in the queue
* Add newline
* Introduce whitelist for Transact and limit UMP processing to 10 messages per block (#6280 )
* Add SafeCallFilter to XcmConfig
* Limit UMP to receive 10 messages every block
* Place 10 message limit on processing instead of receiving
* Always increment the message_processed count whenever a message is processed
* Add as_derivative to the Transact whitelist
* cargo fmt
* Fixes
* Update xcm/xcm-builder/src/universal_exports.rs
Co-authored-by: Branislav Kontur <bkontur@gmail.com >
* Fixes
* Fixes
* Remove topic register and instead use the topic field in XcmContext
* Derive some common traits for DispatchBlobError
* Fixes
* cargo fmt
* Fixes
* Fixes
* Fix comments
* Fixes
* Introduce WithOriginFilter and apply it as the CallDispatcher for runtimes
* Fixes
* Appease clippy and fixes
* Fixes
* Fix more clippy issues
* Fixes
* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
* Add benchmark function for ExportMessage
* Fix comment
* Add upper limit to DownwardMessageQueues size
* Add max size check for queue in can_queue_downward_message
* Fixes
* Make Transact runtime call configurable
* Return Weight::MAX when there is no successful send XCM origin
* Update substrate
* Fixes
* Fixes
* Remove ExportMessage benchmark
* Remove assertion on Transact instruction benchmark
* Make reachable destination configurable in XCM pallet benchmarks
* Fixes
* Fixes
* Remove cfg attribute in fuzzer
* Fixes
* Remove cfg attribute for XCM pallet in test runtime
* Fixes
* Use ReachableDest where possible
* Fixes
* Add benchmark for UnpaidExecution
* Update substrate
* Ensure benchmark functions pass filters
* Add runtime-benchmarks feature to fuzzer
* Ensure FixedRateOfFungible accounts for proof size weights
* cargo fmt
* Whitelist remark_with_event when runtime-benchmarks feature is enabled
* Use remark_with_event for Transact benchmarks
* Fix Cargo.lock
* Allow up to 3 DescendOrigin instructions before UnpaidExecution
* cargo fmt
* Edit code comment
* Check check_origin for unpaid execution privilege
* Fixes
* Small nits for xcm-v3 (#6408 )
* Add possibility to skip benchmark for export_message
* ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic
* Revert
* ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic
* Add HaulBlobError to `fn haul_blob`
* ".git/.scripts/bench-bot.sh" xcm westend-dev pallet_xcm_benchmarks::generic
Co-authored-by: command-bot <>
* Revert changes to UnpaidExecution
* Change AllowUnpaidExecutionFrom to be explicit
* Fix log text
* cargo fmt
* Add benchmarks for XCM pallet version migration (#6448 )
* Add benchmarks for XCM pallet version migration
* cargo fmt
* Fixes
* Fixes
* Fixes
* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
* Fix benchmarks
* Fix benchmarks
* ".git/.scripts/bench-bot.sh" runtime westend-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime kusama-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime rococo-dev pallet_xcm
* ".git/.scripts/bench-bot.sh" runtime polkadot-dev pallet_xcm
Co-authored-by: command-bot <>
* Merge remote-tracking branch 'origin/master' into gav-xcm-v3
* Fixes
* Fix comments (#6470 )
* Specify Ethereum networks by their chain id (#6286 )
Co-authored-by: Squirrel <gilescope@gmail.com >
* Use for Kusama
* Use WithComputedOrigin for Polkadot, Rococo and Westend
* Update lock
* Fix warning
* Update xcm/pallet-xcm/src/tests.rs
Co-authored-by: Squirrel <gilescope@gmail.com >
* Update runtime/parachains/src/ump/migration.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Update xcm/pallet-xcm/src/migration.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fixes
* cargo fmt
* Typo
* Update xcm/src/v3/mod.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Docs
* Docs
* Docs
* Docs
* Docs
* Update xcm/src/v3/multiasset.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add tests for MultiAssets::from_sorted_and_deduplicated
* Fail gracefully when same instance NFTs are detected during push
* Update Substrate to fix benchmarks
* Apply suggestions from code review
* Update runtime/kusama/src/xcm_config.rs
* Rename arguments
* Attempt to fix benchmark
* ".git/.scripts/commands/bench/bench.sh" runtime polkadot-dev runtime_parachains::ump
* Use actual weights for UMP pallet in Polkadot
* ".git/.scripts/commands/bench/bench.sh" runtime kusama-dev runtime_parachains::ump
* ".git/.scripts/commands/bench/bench.sh" runtime westend-dev runtime_parachains::ump
* ".git/.scripts/commands/bench/bench.sh" runtime rococo-dev runtime_parachains::ump
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
Co-authored-by: Alexander Popiak <alexander.popiak@parity.io >
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com >
Co-authored-by: Bastian Köcher <info@kchr.de >
Co-authored-by: Parity Bot <admin@parity.io >
Co-authored-by: stanly-johnson <stanlyjohnson@outlook.com >
Co-authored-by: nanocryk <6422796+nanocryk@users.noreply.github.com >
Co-authored-by: Branislav Kontur <bkontur@gmail.com >
Co-authored-by: asynchronous rob <rphmeier@gmail.com >
Co-authored-by: command-bot <>
Co-authored-by: Vincent Geddes <vincent.geddes@hey.com >
Co-authored-by: Squirrel <gilescope@gmail.com >
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com >
2023-01-17 07:04:34 +00:00
2efa3bab98
pre-checking: Reject failed PVFs ( #6492 )
...
* pre-checking: Reject failed PVFs
* paras: immediately reject any PVF that cannot reach a supermajority
* Make the `quorum` reject condition a bit more clear semantically
* Add comment
* Update implementer's guide
* Update a link
Not related to the rest of the PR, but I randomly noticed and fixed this.
* Update runtime/parachains/src/paras/tests.rs
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com >
* Remove unneeded loop
* Log PVF retries using `info!`
* Change retry logs to `warn!` and add preparation failure log
* Log PVF execution failure
* Clarify why we reject failed PVFs
* Fix PVF reject runtime benchmarks
Co-authored-by: s0me0ne-unkn0wn <48632512+s0me0ne-unkn0wn@users.noreply.github.com >
2023-01-12 09:24:42 +00:00
ae74d33a93
pvf: Fix missing execution request when retrying preparation ( #6537 )
...
* pvf: Add checks for result sender when retrying preparation in tests
* pvf: Fix missing execution request when retrying preparation
* Update comment
2023-01-11 20:01:43 +00:00
3cf18f11ac
Minor fixes ( #6533 )
...
* Minor fixes
* Fix compile errors
2023-01-10 17:59:12 +00:00
44fd95661c
Replace async-std with tokio in PVF subsystem ( #6419 )
...
* Replace async-std with tokio in PVF subsystem
* Rework workers to use `select!` instead of a mutex
The improvement in code readability is more important than the thread overhead.
* Remove unnecessary `fuse`
* Add explanation for `expect()`
* Update node/core/pvf/src/worker_common.rs
Co-authored-by: Bastian Köcher <info@kchr.de >
* Update node/core/pvf/src/worker_common.rs
Co-authored-by: Bastian Köcher <info@kchr.de >
* Address some review comments
* Shutdown tokio runtime
* Run cargo fmt
* Add a small note about retries
* Fix up merge
* Rework `cpu_time_monitor_loop` to return when other thread finishes
* Add error string to PrepareError::IoErr variant
* Log when artifacts fail to prepare
* Fix `cpu_time_monitor_loop`; fix test
* Fix text
* Fix a couple of potential minor data races.
First data race was due to logging in the CPU monitor thread even if the
job (other thread) finished. It can technically finish before or after the log.
Maybe best would be to move this log to the `select!`s, where we are guaranteed
to have chosen the timed-out branch, although there would be a bit of
duplication.
Also, it was possible for this thread to complete before we executed
`finished_tx.send` in the other thread, which would trigger an error as the
receiver has already been dropped. And right now, such a spurious error from
`send` would be returned even if the job otherwise succeeded.
* Update Cargo.lock
Co-authored-by: Bastian Köcher <info@kchr.de >
2023-01-10 10:51:13 +01:00
f0106b30fa
Log PVF retries ( #6504 )
2023-01-04 22:18:41 +00:00
e0a0475a05
PVF preparation: do not conflate errors ( #6384 )
...
* PVF preparation: do not conflate errors
+ Adds some more granularity to the prepare errors.
+ Better distinguish whether errors occur on the host side or the worker.
+ Do not kill the worker if the error happened on the host side.
+ Do not retry preparation if the error was `Panic`.
+ Removes unnecessary indirection with `Selected` type.
* Add missing docs, resolve TODOs
* Address review comments and remove TODOs
* Fix error in CI
* Undo unnecessary change
* Update couple of comments
* Don't return error for stream shutdown
* Update node/core/pvf/src/worker_common.rs
2022-12-20 08:32:12 -05:00
ab090ab7d5
Let the PVF host kill the worker on timeout ( #6381 )
...
* Let the PVF host kill the worker on timeout
* Fix comment
* Fix inaccurate comments; add missing return statement
* Fix a comment
* Fix comment
2022-12-06 13:03:18 -05:00
be403051dc
Set polkadot version in one place ( #6095 )
...
* rust 1.64 enables workspace properties
* add edition, repository and authors.
* of course, update the version in one place.
Co-authored-by: Andronik <write@reusable.software >
2022-12-05 11:36:16 +00:00
28a4e90912
Use CPU clock timeout for PVF jobs ( #6282 )
...
* Put in skeleton logic for CPU-time-preparation
Still needed:
- Flesh out logic
- Refactor some spots
- Tests
* Continue filling in logic for prepare worker CPU time changes
* Fix compiler errors
* Update lenience factor
* Fix some clippy lints for PVF module
* Fix compilation errors
* Address some review comments
* Add logging
* Add another log
* Address some review comments; change Mutex to AtomicBool
* Refactor handling response bytes
* Add CPU clock timeout logic for execute jobs
* Properly handle AtomicBool flag
* Use `Ordering::Relaxed`
* Refactor thread coordination logic
* Fix bug
* Add some timing information to execute tests
* Add section about the mitigation to the IG
* minor: Change more `Ordering`s to `Relaxed`
* candidate-validation: Fix build errors
2022-11-30 13:17:31 +01:00
9ea14e66c8
Clippyfy ( #6341 )
...
* Add clippy config and remove .cargo from gitignore
* first fixes
* Clippyfied
* Add clippy CI job
* comment out rusty-cachier
* minor
* fix ci
* remove DAG from check-dependent-project
* add DAG to clippy
Co-authored-by: alvicsam <alvicsam@gmail.com >
2022-11-30 08:34:06 +00:00
b76086c617
sync versions with current release (0.9.33) ( #6363 )
...
* westend: update transaction version
* polkadot: update transaction version
* kusama: update transaction version
* Bump spec_version to 9330
* bump versions to 0.9.33
2022-11-30 10:32:20 +02:00
186a572907
Provide some more granular metrics for polkadot_pvf_execution_time ( #6346 )
2022-11-28 10:26:17 -05:00
1dec2433ae
Add PVF module documentation ( #6293 )
...
* Add PVF module documentation
TODO (once the PRs land):
- [ ] Document executor parametrization.
- [ ] Document CPU time measurement of timeouts.
* Update node/core/pvf/src/lib.rs
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com >
* Clarify meaning of PVF acronym
* Move PVF doc to implementer's guide
* Clean up implementer's guide a bit
* Add page for PVF types
* pvf: Better separation between crate docs and implementer's guide
* ci: Add "prevalidating" to the dictionary
* ig: Remove types/chain.md
The types contained therein did not exist and the file was not referenced
anywhere.
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com >
2022-11-23 08:20:25 -05:00
6d7f33e612
Retry failed PVF execution (AmbiguousWorkerDeath) ( #6235 )
...
* Fix a couple of typos
* Retry failed PVF execution
PVF execution that fails due to AmbiguousWorkerDeath should be retried once.
This should reduce the occurrence of failures due to transient conditions.
Closes #6195
* Address a couple of nits
* Write tests; refactor (add `validate_candidate_with_retry`)
* Update node/core/candidate-validation/src/lib.rs
Co-authored-by: Andronik <write@reusable.software >
Co-authored-by: eskimor <eskimor@users.noreply.github.com >
Co-authored-by: Andronik <write@reusable.software >
2022-11-08 15:36:36 -05:00
63c6f184cb
Retry failed PVF prepare jobs ( #6213 )
2022-11-08 13:30:14 +00:00
1f8219767e
PVF timeouts follow-up ( #6151 )
...
* Rename timeout consts and timeout parameter; bump leniency
* Update implementor's guide with info about PVFs
* Make glossary a bit easier to read
* Add a note to LENIENT_PREPARATION_TIMEOUT
* Remove PVF-specific section from glossary
* Fix some typos
2022-11-01 10:59:53 -04:00
d4e3501ec2
Make some fixes to logging in PVF subsystem ( #6180 )
...
* Log exit status code for workers
* Make log for execute job conclusion match prepare job conclusion
Trace log for conclusion of prepare job:
```rs
gum::debug!(
target: LOG_TARGET,
validation_code_hash = ?artifact_id.code_hash,
?worker,
?rip,
"prepare worker concluded",
);
```
Co-authored-by: parity-processbot <>
2022-10-26 17:49:51 +02:00
f8cc39a761
sync versions with current release (0.9.31) ( #6176 )
...
* Bump spec_version to 9310
* bump transaction_version (0.9.31) (#6171 )
* Bump transaction_version for polkadot
* Bump transaction_version for kusama
* Bump transaction_version for rococo
* Bump transaction_version for westend
* Bump transaction_version for polkadot
* Bump transaction_version for kusama
* Bump transaction_version for rococo
* Bump transaction_version for westend
* Bump crate versions (0.9.31)
2022-10-22 00:08:53 -04:00
17730b85be
Separate preparation timeouts for PVF prechecking and execution ( #6139 )
...
* Add some documentation
* Add `compilation_timeout` parameter for PVF preparation job
* Update buckets in prometheus metrics
* Update prepare/queue tests
* Update pvf-prechecking overview in implementer docs
* Fix some CI checks
2022-10-13 11:00:57 +00:00
afc7dd3735
node/core/pvf: strip some deps ( #6016 )
2022-09-15 12:25:03 +00:00
1acc3fa150
Sync versions with current release (v0.9.29) ( #5982 )
...
* Bump transaction_version for westend
* Bump transaction_version for rococo
* Bump transaction_version for kusama
* Bump transaction_version for polkadot
* Bump spec_version to 9290
* Bump crate versions
2022-09-09 11:24:06 +00:00
fc97080cb2
version bumps (0.9.28) ( #5933 )
...
* Bump crate versions
* Bump spec_version to 9280 for kusama
* Bump spec_version to 9280 for polkadot
* Bump spec_version to 9280 for rococo
* Bump spec_version to 9280 for westend
* update Cargo.lock
Co-authored-by: parity-processbot <>
2022-08-29 18:01:58 +02:00
85dabbd850
Backport crate version bumps to 0.9.27 ( #5826 )
...
* Bump crate versions
* Update cargo.lock
2022-07-28 10:03:27 +00:00
9fc4e16eff
Version bump to v0.9.26/9260 ( #5755 )
...
* Bump spec_version to 9260
* Version bump to v0.9.26
2022-07-06 10:31:45 +00:00
09b10fa25e
Bump parity-scale-codec from 3.1.2 to 3.1.5 ( #5720 )
...
Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec ) from 3.1.2 to 3.1.5.
- [Release notes](https://github.com/paritytech/parity-scale-codec/releases )
- [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md )
- [Commits](https://github.com/paritytech/parity-scale-codec/compare/parity-scale-codec-3.1.2...parity-scale-codec-3.1.5 )
---
updated-dependencies:
- dependency-name: parity-scale-codec
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-06-27 10:27:49 +00:00
94a85eeac7
pvf: ensure enough stack space ( #5712 )
...
* pvf: ensure enough stack space
* fix typos
Co-authored-by: Andronik <write@reusable.software >
* Use rayon to cache the thread
Co-authored-by: Andronik <write@reusable.software >
2022-06-24 13:16:36 +02:00
e03c24ec47
pvf: unignore terminates_on_timeout test ( #5722 )
2022-06-23 12:45:19 +00:00
97f9b840e4
bump versions to 0.9.25 ( #5684 )
...
* kusama: bump spec_version to 9250
* polkadot: bump spec_version to 9250
* rococo: bump spec_version to 9250
* westend: bump spec_version to 9250
* bump version to 0.9.25
* bump transaction version (polkadot & kusama) (#5690 )
* kusama: bump transaction_version to 12
* polkadot: bump transaction_version to 13
2022-06-21 13:02:04 +00:00
026848c932
Companion to Substrate #11490 ( #5603 )
...
* Fix warnings
* Bump
2022-05-30 11:21:03 +01:00
7eba8f2162
bump to 0.9.23 ( #5593 )
...
* bump version to 0.9.23
* bump spec_version to 9230
2022-05-27 10:18:23 +02:00
d9eff4ecd4
Switch to pooling copy-on-write instantiation strategy for WASM (companion for Substrate#11232) ( #5337 )
...
* Switch to pooling copy-on-write instantiation strategy for WASM
* Fix compilation of `polkadot-test-service`
* Update comments
* Move `max_memory_size` to `Semantics`
* Rename `WasmInstantiationStrategy` to `WasmtimeInstantiationStrategy`
* Update a safety comment
* update lockfile for {"substrate"}
Co-authored-by: parity-processbot <>
2022-05-19 13:06:34 +02:00
3b685fbf91
version bumps (0.9.22) ( #5495 )
...
* bump versions to 0.9.22
* bump spec_version to 9220
2022-05-16 13:24:24 +02:00
b89dc00ec0
Fixes the dead lock when any of the channels get at capacity. ( #5297 )
...
The PVF host is designed to avoid spawning tasks to minimize knowledge
of outer code. Using `async_std::task::spawn` (or Tokio's counterpart)
deemed unacceptable, `SpawnNamed` undesirable. Instead there is only one
task returned that is spawned by the candidate-validation subsystem.
The tasks from the sub-components are polled by that root task.
However, the way the tasks are bundled was incorrect. There was a giant
select that was polling those tasks. Particularly, that implies that as soon as
one of the arms of that select goes into await those sub-tasks stop
getting polled. This is a recipe for a deadlock which indeed happened
here.
Specifically, the deadlock happened during sending messages to the
execute queue by calling
[`send_execute`](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L601 ).
When the channel to the queue reaches the capacity, the control flow is
suspended until the queue handles those messages. Since this code is
essentially reached from [one of the select
arms](https://github.com/paritytech/polkadot/blob/a68d9be35656dcd96e378fd9dd3d613af754d48a/node/core/pvf/src/host.rs#L371 ),
the queue won't be given the control and thus no further progress can be
made.
This problem is solved by bundling the tasks one level higher instead,
by `selecting` over those long-running tasks.
We also stop treating returning from those long-running tasks as error
conditions, since that can happen during legit shutdown.
2022-04-09 10:57:34 +00:00
7ed275bff8
bump version to 0.9.19 ( #5290 )
2022-04-08 11:48:43 +00:00
b424e5e741
Companion for Substrate#11107 ( #5197 )
...
* Rename to BagError
* Additional parameter for 'revert' command
* Set aux revert param to None
* Align to changes in how the WASM executor is configured in `substrate`
* update lockfile for {"substrate"}
* update lockfile for {"substrate"}
* Update substrate
* Update substrate
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
Co-authored-by: Davide Galassi <davxy@datawok.net >
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com >
Co-authored-by: parity-processbot <>
2022-03-24 15:56:57 -04:00
1656c4cd02
Companion to Implementing MaxEncodedLen for generate_solution_type #11032 ( #5145 )
...
* Fixing `generate_solution_type`
This is needed after changes to substrate
* `VoterSnapshotPerBlock` -> `MaxElectingVoters`
* rename `SizeBound` to `MaxVoters`
* cargo update -p sp-io
* ignore failing test
* spell-check
* fix pub OffchainRepeat
Co-authored-by: kianenigma <kian@parity.io >
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com >
2022-03-23 13:18:57 +00:00
fbfcf3f066
Bump async-std from 1.10.0 to 1.11.0 ( #5186 )
...
Bumps [async-std](https://github.com/async-rs/async-std ) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/async-rs/async-std/releases )
- [Changelog](https://github.com/async-rs/async-std/blob/master/CHANGELOG.md )
- [Commits](https://github.com/async-rs/async-std/compare/v1.10.0...v1.11.0 )
---
updated-dependencies:
- dependency-name: async-std
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-23 11:16:30 +01:00
61a6004cf1
Bump parity-scale-codec from 3.1.0 to 3.1.2 ( #5153 )
...
Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec ) from 3.1.0 to 3.1.2.
- [Release notes](https://github.com/paritytech/parity-scale-codec/releases )
- [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md )
- [Commits](https://github.com/paritytech/parity-scale-codec/commits )
---
updated-dependencies:
- dependency-name: parity-scale-codec
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-18 10:32:49 +00:00
d631f1dea8
observability: tracing gum, automatically cross ref traceID ( #5079 )
...
* add some gum
* bump expander
* gum
* fix all remaining issues
* last fixup
* Update node/gum/proc-macro/src/lib.rs
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com >
* change
* netowrk
* fixins
* chore
* allow optional fmt str + args, prep for expr as kv field
* tracing -> gum rename fallout
* restrict further
* allow multiple levels of field accesses
* another round of docs and a slip of the pen
* update ADR
* fixup lock fiel
* use target: instead of target=
* minors
* fix
* chore
* Update node/gum/README.md
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com >
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com >
Co-authored-by: Andrei Sandu <54316454+sandreim@users.noreply.github.com >
2022-03-15 11:05:16 +00:00
3335ddc0da
Bump parity-scale-codec from 3.0.0 to 3.1.0 ( #5096 )
...
Bumps [parity-scale-codec](https://github.com/paritytech/parity-scale-codec ) from 3.0.0 to 3.1.0.
- [Release notes](https://github.com/paritytech/parity-scale-codec/releases )
- [Changelog](https://github.com/paritytech/parity-scale-codec/blob/master/CHANGELOG.md )
- [Commits](https://github.com/paritytech/parity-scale-codec/compare/v3.0...parity-scale-codec-v3.1.0 )
---
updated-dependencies:
- dependency-name: parity-scale-codec
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-14 19:34:46 +01:00
6fc2da146f
Bump all spec version to 9180 and versions to 0.9.18 ( #5083 )
...
* bump all spec version to 9180 and versions to 0.9.18
* all to 0.9.18
* lock update
2022-03-11 10:19:47 +01:00
a863bfe69d
Bump tracing from 0.1.31 to 0.1.32 ( #5075 )
...
Bumps [tracing](https://github.com/tokio-rs/tracing ) from 0.1.31 to 0.1.32.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.31...tracing-0.1.32 )
---
updated-dependencies:
- dependency-name: tracing
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-03-10 09:45:57 +00:00
e8cb6cdaac
Companion to "Updating scale to v3" ( #4958 )
...
* Updating dependencies
* Adapting code to scale v3
* Upgrade bitvec to 1.0.0
* Fix bitvec arithmetics
* Update Cargo.lock
* Update sp-io
* Fixing the build
* Yanked scale-info 2.0.0
Co-authored-by: Bastian Köcher <info@kchr.de >
2022-02-25 13:07:06 +01:00
d9d53c17a1
Switch to edition 2021 ( #4976 )
...
* Switch to edition 2021
* Reset bridges
* Fix compilation
* Fix test-runtime
* Fix malus
2022-02-24 10:20:58 +00:00
6fdd026010
Bump tracing from 0.1.30 to 0.1.31 ( #4941 )
...
Bumps [tracing](https://github.com/tokio-rs/tracing ) from 0.1.30 to 0.1.31.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.30...tracing-0.1.31 )
---
updated-dependencies:
- dependency-name: tracing
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-18 09:54:13 +00:00
98a1343cd9
Bump rand from 0.8.4 to 0.8.5 ( #4917 )
...
Bumps [rand](https://github.com/rust-random/rand ) from 0.8.4 to 0.8.5.
- [Release notes](https://github.com/rust-random/rand/releases )
- [Changelog](https://github.com/rust-random/rand/blob/master/CHANGELOG.md )
- [Commits](https://github.com/rust-random/rand/compare/0.8.4...0.8.5 )
---
updated-dependencies:
- dependency-name: rand
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-14 11:37:02 +01:00
f00f24e656
Bump tracing from 0.1.29 to 0.1.30 ( #4897 )
...
Bumps [tracing](https://github.com/tokio-rs/tracing ) from 0.1.29 to 0.1.30.
- [Release notes](https://github.com/tokio-rs/tracing/releases )
- [Commits](https://github.com/tokio-rs/tracing/compare/tracing-0.1.29...tracing-0.1.30 )
---
updated-dependencies:
- dependency-name: tracing
dependency-type: direct:production
update-type: version-update:semver-patch
...
Signed-off-by: dependabot[bot] <support@github.com >
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-02-11 23:17:19 +01:00
167df32561
Bump version ( #4904 )
...
* Bump spec_version
* Bump version to 0.9.17
2022-02-11 20:43:39 +01:00
Keith Yeung