**Update:** Pushed additional changes based on the review comments.
**This pull request fixes various spelling mistakes in this
repository.**
Most of the changes are contained in the first **3** commits:
- `Fix spelling mistakes in comments and docs`
- `Fix spelling mistakes in test names`
- `Fix spelling mistakes in error messages, panic messages, logs and
tracing`
Other source code spelling mistakes are separated into individual
commits for easier reviewing:
- `Fix the spelling of 'authority'`
- `Fix the spelling of 'REASONABLE_HEADERS_IN_JUSTIFICATION_ANCESTRY'`
- `Fix the spelling of 'prev_enqueud_messages'`
- `Fix the spelling of 'endpoint'`
- `Fix the spelling of 'children'`
- `Fix the spelling of 'PenpalSiblingSovereignAccount'`
- `Fix the spelling of 'PenpalSudoAccount'`
- `Fix the spelling of 'insufficient'`
- `Fix the spelling of 'PalletXcmExtrinsicsBenchmark'`
- `Fix the spelling of 'subtracted'`
- `Fix the spelling of 'CandidatePendingAvailability'`
- `Fix the spelling of 'exclusive'`
- `Fix the spelling of 'until'`
- `Fix the spelling of 'discriminator'`
- `Fix the spelling of 'nonexistent'`
- `Fix the spelling of 'subsystem'`
- `Fix the spelling of 'indices'`
- `Fix the spelling of 'committed'`
- `Fix the spelling of 'topology'`
- `Fix the spelling of 'response'`
- `Fix the spelling of 'beneficiary'`
- `Fix the spelling of 'formatted'`
- `Fix the spelling of 'UNKNOWN_PROOF_REQUEST'`
- `Fix the spelling of 'succeeded'`
- `Fix the spelling of 'reopened'`
- `Fix the spelling of 'proposer'`
- `Fix the spelling of 'InstantiationNonce'`
- `Fix the spelling of 'depositor'`
- `Fix the spelling of 'expiration'`
- `Fix the spelling of 'phantom'`
- `Fix the spelling of 'AggregatedKeyValue'`
- `Fix the spelling of 'randomness'`
- `Fix the spelling of 'defendant'`
- `Fix the spelling of 'AquaticMammal'`
- `Fix the spelling of 'transactions'`
- `Fix the spelling of 'PassingTracingSubscriber'`
- `Fix the spelling of 'TxSignaturePayload'`
- `Fix the spelling of 'versioning'`
- `Fix the spelling of 'descendant'`
- `Fix the spelling of 'overridden'`
- `Fix the spelling of 'network'`
Let me know if this structure is adequate.
**Note:** The usage of the words `Merkle`, `Merkelize`, `Merklization`,
`Merkelization`, `Merkleization`, is somewhat inconsistent but I left it
as it is.
~~**Note:** In some places the term `Receival` is used to refer to
message reception, IMO `Reception` is the correct word here, but I left
it as it is.~~
~~**Note:** In some places the term `Overlayed` is used instead of the
more acceptable version `Overlaid` but I also left it as it is.~~
~~**Note:** In some places the term `Applyable` is used instead of the
correct version `Applicable` but I also left it as it is.~~
**Note:** Some usage of British vs American english e.g. `judgement` vs
`judgment`, `initialise` vs `initialize`, `optimise` vs `optimize` etc.
are both present in different places, but I suppose that's
understandable given the number of contributors.
~~**Note:** There is a spelling mistake in `.github/CODEOWNERS` but it
triggers errors in CI when I make changes to it, so I left it as it
is.~~
This PR adds tests for the `transaction_broadcast` method.
The testing needs to coordinate the following components:
- The `TestApi` marks transactions as invalid and implements
`ChainApi::validate_transaction`
- this is what dictates if a transaction is valid or not and is called
from within the `BasicPool`
- The `BasicPool` which maintains the transactions and implements
`submit_and_watch` needed by the tx broadcast to submit the transaction
- The status of the transaction pool is exposed by mocking the BasicPool
- The `ChainHeadMockClient` which mocks the
`BlockchainEvents::import_notification_stream` needed by the tx
broadcast to know to which blocks the transaction is submitted
The following changes have been added to the substrate testing to
accommodate this:
- `TestApi` gets ` remove_invalid`, counterpart to `add_invalid` to
ensure an invalid transaction can become valid again; as well as a
priority setter for extrinsics
- `BasicPool` test constructor is extended with options for the
`PoolRotator`
- this mechanism is needed because transactions are banned for 30mins
(default) after they are declared invalid
- testing bypasses this by providing a `Duration::ZERO`
### Testing Scenarios
- Capture the status of the transaction as it is normally broadcasted
- `transaction_stop` is valid while the transaction is in progress
- A future transaction is handled when the dependencies are completed
- Try to resubmit the transaction at a later block (currently invalid)
- An invalid transaction status is propagated; the transaction is marked
as temporarily banned; then the ban expires and transaction is
resubmitted
This builds on top of:
https://github.com/paritytech/polkadot-sdk/pull/3079
Part of: https://github.com/paritytech/polkadot-sdk/issues/3084
cc @paritytech/subxt-team
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>
This PR backports the changes from the rpc-v2 spec:
https://github.com/paritytech/json-rpc-interface-spec/pull/134
The `Broadcasted` event has been removed:
- it is hard to enforce a `Dropped { broadcasted: bool }` event in cases
of a load-balancer being placed in front of an RPC server
- when the server exists, it is impossible to guarantee this field if
the server did not previously send a `Broadcasted` event
- the number of peers reported by this event does not guarantee that
peers are unique
- the same peer can disconnect and reconnect, increasing this metric
number
- the number of peers that receive this transaction offers no guarantee
about the transaction being included in the chain at a later time
cc @paritytech/subxt-team
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: James Wilson <james@jsdw.me>
Close#2992
Breaking changes:
- rpc server grafana metric `substrate_rpc_requests_started` is removed
(not possible to implement anymore)
- rpc server grafana metric `substrate_rpc_requests_finished` is removed
(not possible to implement anymore)
- rpc server ws ping/pong not ACK:ed within 30 seconds more than three
times then the connection will be closed
Added
- rpc server grafana metric `substrate_rpc_sessions_time` is added to
get the duration for each websocket session
This PR implements the
[transaction_unstable_broadcast](https://github.com/paritytech/json-rpc-interface-spec/blob/main/src/api/transaction_unstable_broadcast.md)
and
[transaction_unstable_stop](https://github.com/paritytech/json-rpc-interface-spec/blob/main/src/api/transaction_unstable_stop.md).
The
[transaction_unstable_broadcast](https://github.com/paritytech/json-rpc-interface-spec/blob/main/src/api/transaction_unstable_broadcast.md)
submits the provided transaction at the best block of the chain.
If the transaction is dropped or declared invalid, the API tries to
resubmit the transaction at the next available best block.
### Broadcasting
The broadcasting operation continues until either:
- the user called `transaction_unstable_stop` with the operation ID that
identifies the broadcasting operation
- the transaction state is one of the following:
- Finalized: the transaction is part of the chain
- FinalizedTimeout: we have waited for 256 finalized blocks and timedout
- Usurped the transaction has been replaced in the tx pool
The broadcasting retires to submit the transaction when the transaction
state is:
- Invalid: the transaction might become valid at a later time
- Dropped: the transaction pool's capacity is full at the moment, but
might clear when other transactions are finalized/dropped
### Stopping
The `transaction_unstable_broadcast` spawns an abortable future and
tracks the abort handler.
When the
[transaction_unstable_stop](https://github.com/paritytech/json-rpc-interface-spec/blob/main/src/api/transaction_unstable_stop.md)
is called with a valid operation ID; the abort handler of the
corresponding `transaction_unstable_broadcast` future is called. This
behavior ensures the broadcast future is finishes on the next polling.
When the `transaction_unstable_stop` is called with an invalid operation
ID, an invalid jsonrpc specific error object is returned.
### Testing
This PR adds the testing harness of the transaction API and validates
two basic scenarios:
- transaction enters and exits the transaction pool
- transaction stop returns appropriate values when called with valid and
invalid operation IDs
Closes: https://github.com/paritytech/polkadot-sdk/issues/3039
Note that the API should be enabled after:
https://github.com/paritytech/polkadot-sdk/issues/3084.
cc @paritytech/subxt-team
---------
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Sebastian Kunert <skunert49@gmail.com>
This is a rather big change in jsonrpsee, the major things in this bump
are:
- Server backpressure (the subscription impls are modified to deal with
that)
- Allow custom error types / return types (remove jsonrpsee::core::Error
and jsonrpee::core::CallError)
- Bug fixes (graceful shutdown in particular not used by substrate
anyway)
- Less dependencies for the clients in particular
- Return type requires Clone in method call responses
- Moved to tokio channels
- Async subscription API (not used in this PR)
Major changes in this PR:
- The subscriptions are now bounded and if subscription can't keep up
with the server it is dropped
- CLI: add parameter to configure the jsonrpc server bounded message
buffer (default is 64)
- Add our own subscription helper to deal with the unbounded streams in
substrate
The most important things in this PR to review is the added helpers
functions in `substrate/client/rpc/src/utils.rs` and the rest is pretty
much chore.
Regarding the "bounded buffer limit" it may cause the server to handle
the JSON-RPC calls
slower than before.
The message size limit is bounded by "--rpc-response-size" thus "by
default 10MB * 64 = 640MB"
but the subscription message size is not covered by this limit and could
be capped as well.
Hopefully the last release prior to 1.0, sorry in advance for a big PR
Previous attempt: https://github.com/paritytech/substrate/pull/13992
Resolves https://github.com/paritytech/polkadot-sdk/issues/748, resolves
https://github.com/paritytech/polkadot-sdk/issues/627
This PR adjusts the serialized format of the the returned RuntimeVersion
in the rpc-spec-v2 methods. This is done to match the format defined
here:
https://paritytech.github.io/json-rpc-interface-spec/api/chainHead_unstable_follow.html#about-the-runtime
- ##### `apis` field as object
`apis` field of `RuntimeVersion` is now returned as an object, e.g.
```
"apis": {
"0xdf6acb689907609b": 3,
"0x37e397fc7c91f5e4": 1,
}
```
instead of
```
"apis": [
["0xdf6acb689907609b", 3],
["0x37e397fc7c91f5e4", 1],
]
```
- ##### removed `stateVersion` and `authoringVersion`
`stateVersion` and `authoringVersion` are no longer returned in the
`RuntimeVersion` JSON Object.
- ##### block index in chain head events as integer
### Related Issues
Closes: #1507Closes: #1146
### Testing Done
Adjusted existing tests to make sure data is returned in the correct
format.
* Change copyright year to 2023 from 2022
* Fix incorrect update of copyright year
* Remove years from copy right header
* Fix remaining files
* Fix typo in a header and remove update-copyright.sh
* rpc/tx: Add transaction structures for serialization
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Add public facing `TransactionEvent`
To circumvent the fact that serde does not allow mixing
`#[serde(tag = "event")]` with
`#[serde(tag = "event", content = "block")]`
the public facing subscription structure is serialized
and deserialized to an intermmediate representation.
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Add trait for the `transaction` API
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Convert RPC errors to transaction events
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Implement `transaction` RPC methods
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* tx-pool: Propagate tx index to events
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* tx-pool: Adjust testing to reflect tx index in events
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Convert tx-pool events for the new RPC spec
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Convert tx-pool `FinalityTimeout` event to `Dropped`
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* service: Enable the `transaction` API
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Add tests for tx event encoding and decoding
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* tx: Add indentation for subscriptions
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Fix documentation
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Serialize usize to hex
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* tx-pool: Rename closure parameters
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* service: Separate RPC spec versions
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Use `H256` for testing block's hash
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Serialize numbers as string
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* tx-pool: Backward compatibility with RPC v1
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update client/rpc-spec-v2/src/transaction/transaction.rs
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
* rpc/tx: Remove comment about serde clone
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* rpc/tx: Use RPC custom error code for invalid tx format
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
* Update client/rpc-spec-v2/src/transaction/event.rs
Co-authored-by: James Wilson <james@jsdw.me>
* rpc/tx: Adjust internal structures for serialization/deserialization
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Signed-off-by: Alexandru Vasile <alexandru.vasile@parity.io>
Co-authored-by: Niklas Adolfsson <niklasadolfsson1@gmail.com>
Co-authored-by: James Wilson <james@jsdw.me>
Alexandru Vasile