7e7fe99047
[FRAME] Make MQ pallet re-entrancy safe ( #2356 )
...
Closes https://github.com/paritytech/polkadot-sdk/issues/2319
Changes:
- Ensure that only `enqueue_message(s)` is callable from within the
message processor. This prevents messed up storage that can currently
happen when the pallet is called into recursively.
- Use `H256` instead of `[u8; 32]` for clearer API.
## Details
The re-entracy check is done with the `environmental` crate by adding a
`with_service_mutex(f)` function that runs the closure exclusively. This
works since the MQ pallet is not instantiable.
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Francisco Aguirre <franciscoaguirreperez@gmail.com >
2023-12-07 17:48:13 +01:00
2a8ca4ceb2
Add try_state check to Pallet MessageQueue ( #13502 )
...
* feat: Add try_state for message_queue
* Update frame/message-queue/src/lib.rs
change if let to while let and modify bump_service_head function
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* feat: update try_state, add checks for storage
* fix: add try_state builder for remaining tests
* Update frame/message-queue/src/mock.rs
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* chore: assert statement to ensure
* Fix tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Use ensure instead of assert
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix function signature and feature gate
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Cleanup code
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Juan <juangirini@gmail.com >
2023-08-14 17:19:07 +00:00
892831e55c
WeightMeter: more consistent naming (#14586 )
...
* Rename WeightMeter functions
* Fixes
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fixup and doc + tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* One more test
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fixup pallets
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Use correct function 🤦
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Apply suggestions from code review
Co-authored-by: Juan <juangirini@gmail.com >
* Update primitives/weights/src/weight_meter.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update primitives/weights/src/weight_meter.rs
Co-authored-by: Bastian Köcher <git@kchr.de >
* Update primitives/weights/src/weight_meter.rs
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Juan <juangirini@gmail.com >
Co-authored-by: Bastian Köcher <git@kchr.de >
2023-07-18 14:38:44 +00:00
4249643df2
pallet-message-queue: add queue pausing (#14318 )
...
* pallet-message-queue: add queue pausing
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix build
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Remove check
Otherwise it would not start servicing queues that started paused
and became unpaused afterwards.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
2023-06-28 08:58:31 +00:00
7f0597a1e6
FRAME: Allow message ID to be mutated in ProcessMessage ( #14183 )
2023-05-21 20:20:14 +01:00
e16f15119f
MessageQueue: unknit permanently overweight books (#13528 )
...
* Unknit permanently overweight books
A book with only permanently overweight messages should be unkit
from the ready ring. This does currently not happen since perm.
overweight messages are not counted as "processed" and therefore
not increase the "total_processed" counter.
This is only a problem when the next and only message that is
processed is overweight. Eventually this should resolve itself
when another non-overweight message is enqueued and processed.
But for correctness it should be unknitted.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* fmt
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* One more tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet-message-queue
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: command-bot <>
2023-03-07 11:31:57 +00:00
16773d3696
Yieldable queues for pallet MessageQueue ( #13424 )
...
* Add Yield message processing error
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add NoopServiceQueues
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Implement temporary error aka Yield
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Make NoopMessageProcessor generic
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Mock pausable message processor
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Test paused queues
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Integration test paused queues
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Use WeightMeter instead of weight return
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* fix
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Make compile
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* ".git/.scripts/commands/bench/bench.sh" pallet dev pallet_message_queue
* Fix test
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
---------
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: command-bot <>
2023-02-25 16:13:20 +00:00
bc53b9a03a
Remove years from copyright notes. ( #13415 )
...
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
2023-02-21 18:46:41 +00:00
6e0453a298
pallet-message-queue: Fix license (#12895 )
...
* Fix license
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add mock doc
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
2022-12-12 10:48:43 +00:00
6f3d1a8143
General Message Queue Pallet ( #12485 )
...
* The message queue
* Make fully generic
* Refactor
* Docs
* Refactor
* Use iter not slice
* Per-origin queues
* Multi-queue processing
* Introduce MaxReady
* Remove MaxReady in favour of ready ring
* Cleanups
* ReadyRing and tests
* Stale page reaping
* from_components -> from_parts
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Move WeightCounter to sp_weights
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add MockedWeightInfo
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Deploy to kitchensink
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Use WeightCounter
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Small fixes and logging
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add service_page
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Typo
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Move service_page below service_queue
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add service_message
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Use correct weight function
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Overweight execution
* Refactor
* Missing file
* Fix WeightCounter usage in scheduler
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix peek_index
Take into account that decoding from a mutable slice modifies it.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add tests and bench service_page_item
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add debug_info
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add no-progress check to service_queues
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add more benches
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Bound from_message and try_append_message
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add PageReaped event
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Rename BookStateOf and BookStateFor
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Update tests and remove logging
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Remove redundant per-message origins; add footprint() and sweep_queue()
* Move testing stuff to mock.rs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add integration test
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix no-progress check
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix debug_info
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fixup merge and tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix footprint tracking
* Introduce
* Formatting
* OverweightEnqueued event, auto-servicing config item
* Update tests and benchmarks
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Clippy
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Provide change handler
* Add missing BookStateFor::insert and call QueueChangeHandler
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Docs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Update benchmarks and weights
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* More tests...
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Use weight metering functions
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* weightInfo::process_message_payload is gone
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add defensive_saturating_accrue
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Rename WeightCounter to WeightMeter
Ctr+Shift+H should do the trick.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Test on_initialize
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add module docs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Remove origin from MaxMessageLen
The message origin is not encoded into the heap and does
therefore not influence the max message length anymore.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add BoundedVec::as_slice
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Test Page::{from_message, try_append_message}
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fixup docs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Docs
* Do nothing in sweep_queue if the queue does not exist
... otherwise it inserts default values into the storage.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Test ring (un)knitting
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Upgrade stress-test
Change the test to not assume that all queued messages will be
processed in the next block but split it over multiple.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* More tests...
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Beauty fixes
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* clippy
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Rename BoundedVec::as_slice to as_bounded_slice
Conflicts with deref().as_slice() otherwise.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix imports
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Remove ReadyRing struct
Was used for testing only. Instead use 'fn assert_ring' which also
check the service head and backlinks.
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Beauty fixes
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix stale page watermark
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Cleanup
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix test feature and clippy
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* QueueChanged handler is called correctly
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Update benches
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Abstract testing functions
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* More tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Cleanup
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Clippy
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* fmt
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Simplify tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Make stuff compile
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Extend overweight execution benchmark
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Remove TODOs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Test service queue with faulty MessageProcessor
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* fmt
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Update pallet ui tests to 1.65
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* More docs
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Review doc fixes
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com >
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Add weight_limit to extrinsic weight of execute_overweight
* Correctly return unused weight
* Return actual weight consumed in do_execute_overweight
* Review fixes
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Set version 7.0.0-dev
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Make it compile
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Switch message_size to u64
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Switch message_count to u64
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Fix benchmarks
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Make CI green
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Docs
* Update tests
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* ".git/.scripts/bench-bot.sh" pallet dev pallet_message_queue
* Dont mention README.md in the Cargo.toml
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
* Remove reference to readme
Signed-off-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: Oliver Tale-Yazdi <oliver.tale-yazdi@parity.io >
Co-authored-by: parity-processbot <>
Co-authored-by: Robert Klotzner <eskimor@users.noreply.github.com >
Co-authored-by: Keith Yeung <kungfukeith11@gmail.com >
2022-12-09 10:38:24 +00:00
Oliver Tale-Yazdi