Offchain Phragmén BREAKING. (#4517)

* Initial skeleton for offchain phragmen

* Basic compact encoding decoding for results

* add compact files

* Bring back Self::ensure_storage_upgraded();

* Make staking use compact stuff.

* First seemingly working version of reduce, full of todos

* Everything phragmen related works again.

* Signing made easier, still issues.

* Signing from offchain compile fine 😎

* make compact work with staked asssignment

* Evaluation basics are in place.

* Move reduce into crate. Document stuff

* move reduce into no_std

* Add files

* Remove other std deps. Runtime compiles

* Seemingly it is al stable; cycle implemented but not integrated.

* Add fuzzing code.

* Cleanup reduce a bit more.

* a metric ton of tests for staking; wip 🔨

* Implement a lot more of the tests.

* wip getting the unsigned stuff to work

* A bit gleanup for unsigned debug

* Clean and finalize compact code.

* Document reduce.

* Still problems with signing

* We officaly duct taped the transaction submission stuff. 🤓

* Deadlock with keys again

* Runtime builds

* Unsigned test works 🙌

* Some cleanups

* Make all the tests compile and stuff

* Minor cleanup

* fix more merge stuff

* Most tests work again.

* a very nasty bug in reduce

* Fix all integrations

* Fix more todos

* Revamp everything and everything

* Remove bogus test

* Some review grumbles.

* Some fixes

* Fix doc test

* loop for submission

* Fix cli, keyring etc.

* some cleanup

* Fix staking tests again

* fix per-things; bring patches from benchmarking

* better score prediction

* Add fuzzer, more patches.

* Some fixes

* More docs

* Remove unused generics

* Remove max-nominator footgun

* Better fuzzer

* Disable it 

* Bump.

* Another round of self-review

* Refactor a lot

* More major fixes in perThing

* Add new fuzz file

* Update lock

* fix fuzzing code.

* Fix nominator retain test

* Add slashing check

* Update frame/staking/src/tests.rs

Co-Authored-By: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>

* Some formatting nits

* Review comments.

* Fix cargo file

* Almost all tests work again

* Update frame/staking/src/tests.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Fix review comments

* More review stuff

* Some nits

* Fix new staking / session / babe relation

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Update primitives/phragmen/compact/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Some doc updates to slashing

* Fix derive

* Remove imports

* Remove unimplemented tests

* nits

* Remove dbg

* Better fuzzing params

* Remove unused pref map

* Deferred Slashing/Offence for offchain Phragmen  (#5151)

* Some boilerplate

* Add test

* One more test

* Review comments

* Fix build

* review comments

* fix more

* fix build

* Some cleanups and self-reviews

* More minor self reviews

* Final nits

* Some merge fixes.

* opt comment

* Fix build

* Fix build again.

* Update frame/staking/fuzz/fuzz_targets/submit_solution.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update frame/staking/src/slashing.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update frame/staking/src/offchain_election.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Fix review comments

* fix test

* === 🔑 Revamp without staking key.

* final round of changes.

* Fix cargo-deny

* Update frame/staking/src/lib.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
This commit is contained in:
Kian Paimani
2020-03-26 15:37:40 +01:00
committed by GitHub
parent 2a67e6c437
commit 970c5f94f2
64 changed files with 11953 additions and 892 deletions
+401
View File
@@ -0,0 +1,401 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
[[package]]
name = "arbitrary"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "64cf76cb6e2222ed0ea86b2b0ee2f71c96ec6edd5af42e84d59160e91b836ec4"
[[package]]
name = "arrayvec"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cff77d8686867eceff3105329d4698d96c2391c176d5d03adc90c7389162b5b8"
[[package]]
name = "autocfg"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8aac770f1885fd7e387acedd76065302551364496e46b3dd00860b2f8359b9d"
[[package]]
name = "bitvec"
version = "0.15.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a993f74b4c99c1908d156b8d2e0fb6277736b0ecbd833982fd1241d39b2766a6"
[[package]]
name = "byte-slice-cast"
version = "0.3.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b0a5e3906bcbf133e33c1d4d95afc664ad37fbdb9f6568d8043e7ea8c27d93d3"
[[package]]
name = "byteorder"
version = "1.3.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "08c48aae112d48ed9f069b33538ea9e3e90aa263cfa3d1c24309612b1f7472de"
[[package]]
name = "c2-chacha"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "214238caa1bf3a496ec3392968969cab8549f96ff30652c9e56885329315f6bb"
dependencies = [
"ppv-lite86",
]
[[package]]
name = "cfg-if"
version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4785bdd1c96b2a846b2bd7cc02e86b6b3dbf14e7e53446c4f54c92a361040822"
[[package]]
name = "crunchy"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7a81dae078cea95a014a339291cec439d2f232ebe854a9d672b796c6afafa9b7"
[[package]]
name = "fixed-hash"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3367952ceb191f4ab95dd5685dc163ac539e36202f9fcfd0cb22f9f9c542fefc"
dependencies = [
"byteorder",
"rand",
"rustc-hex",
"static_assertions",
]
[[package]]
name = "getrandom"
version = "0.1.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7abc8dd8451921606d809ba32e95b6111925cd2906060d2dcc29c070220503eb"
dependencies = [
"cfg-if",
"libc",
"wasi",
]
[[package]]
name = "honggfuzz"
version = "0.5.45"
dependencies = [
"arbitrary",
"lazy_static",
"memmap",
]
[[package]]
name = "impl-codec"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1be51a921b067b0eaca2fad532d9400041561aa922221cc65f95a85641c6bf53"
dependencies = [
"parity-scale-codec",
]
[[package]]
name = "integer-sqrt"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f65877bf7d44897a473350b1046277941cee20b263397e90869c50b6e766088b"
[[package]]
name = "lazy_static"
version = "1.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e2abad23fbc42b3700f2f279844dc832adb2b2eb069b2df918f455c4e18cc646"
[[package]]
name = "libc"
version = "0.2.67"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eb147597cdf94ed43ab7a9038716637d2d1bf2bc571da995d0028dec06bd3018"
[[package]]
name = "memmap"
version = "0.7.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6585fd95e7bb50d6cc31e20d4cf9afb4e2ba16c5846fc76793f11218da9c475b"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "num-bigint"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "090c7f9998ee0ff65aa5b723e4009f7b217707f1fb5ea551329cc4d6231fb304"
dependencies = [
"autocfg",
"num-integer",
"num-traits",
]
[[package]]
name = "num-integer"
version = "0.1.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f6ea62e9d81a77cd3ee9a2a5b9b609447857f3d358704331e4ef39eb247fcba"
dependencies = [
"autocfg",
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c62be47e61d1842b9170f0fdeec8eba98e60e90e5446449a0545e5152acd7096"
dependencies = [
"autocfg",
]
[[package]]
name = "parity-scale-codec"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f509c5e67ca0605ee17dcd3f91ef41cadd685c75a298fb6261b781a5acb3f910"
dependencies = [
"arrayvec",
"bitvec",
"byte-slice-cast",
"parity-scale-codec-derive",
"serde",
]
[[package]]
name = "parity-scale-codec-derive"
version = "1.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5a0ec292e92e8ec7c58e576adacc1e3f399c597c8f263c42f18420abe58e7245"
dependencies = [
"proc-macro-crate",
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "ppv-lite86"
version = "0.2.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74490b50b9fbe561ac330df47c08f3f33073d2d00c150f719147d7c54522fa1b"
[[package]]
name = "primitive-types"
version = "0.6.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e4336f4f5d5524fa60bcbd6fe626f9223d8142a50e7053e979acdf0da41ab975"
dependencies = [
"fixed-hash",
"impl-codec",
"uint",
]
[[package]]
name = "proc-macro-crate"
version = "0.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e10d4b51f154c8a7fb96fd6dad097cb74b863943ec010ac94b9fd1be8861fe1e"
dependencies = [
"toml",
]
[[package]]
name = "proc-macro2"
version = "1.0.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c09721c6781493a2a492a96b5a5bf19b65917fe6728884e7c44dd0c60ca3435"
dependencies = [
"unicode-xid",
]
[[package]]
name = "quote"
version = "1.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2bdc6c187c65bca4260c9011c9e3132efe4909da44726bad24cf7572ae338d7f"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rand"
version = "0.7.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a6b1679d49b24bbfe0c803429aa1874472f50d9b363131f0e89fc356b544d03"
dependencies = [
"getrandom",
"libc",
"rand_chacha",
"rand_core",
"rand_hc",
]
[[package]]
name = "rand_chacha"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "03a2a90da8c7523f554344f921aa97283eadf6ac484a6d2a7d0212fa7f8d6853"
dependencies = [
"c2-chacha",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "90bde5296fc891b0cef12a6d03ddccc162ce7b2aff54160af9338f8d40df6d19"
dependencies = [
"getrandom",
]
[[package]]
name = "rand_hc"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ca3129af7b92a17112d59ad498c6f81eaf463253766b90396d39ea7a39d6613c"
dependencies = [
"rand_core",
]
[[package]]
name = "rustc-hex"
version = "2.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3e75f6a532d0fd9f7f13144f392b6ad56a32696bfcd9c78f797f16bbb6f072d6"
[[package]]
name = "serde"
version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "414115f25f818d7dfccec8ee535d76949ae78584fc4f79a6f45a904bf8ab4449"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.104"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "128f9e303a5a29922045a830221b8f78ec74a5f544944f3d5984f8ec3895ef64"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "sp-arithmetic"
version = "2.0.0-alpha.3"
dependencies = [
"integer-sqrt",
"num-traits",
"parity-scale-codec",
"serde",
"sp-debug-derive",
"sp-std",
]
[[package]]
name = "sp-arithmetic-fuzzer"
version = "2.0.0"
dependencies = [
"honggfuzz",
"num-bigint",
"num-traits",
"primitive-types",
"sp-arithmetic",
]
[[package]]
name = "sp-debug-derive"
version = "2.0.0-alpha.3"
dependencies = [
"proc-macro2",
"quote",
"syn",
]
[[package]]
name = "sp-std"
version = "2.0.0-alpha.3"
[[package]]
name = "static_assertions"
version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f"
[[package]]
name = "syn"
version = "1.0.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "123bd9499cfb380418d509322d7a6d52e5315f064fe4b3ad18a53d6b92c07859"
dependencies = [
"proc-macro2",
"quote",
"unicode-xid",
]
[[package]]
name = "toml"
version = "0.5.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ffc92d160b1eef40665be3a05630d003936a3bc7da7421277846c2613e92c71a"
dependencies = [
"serde",
]
[[package]]
name = "uint"
version = "0.8.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e75a4cdd7b87b28840dba13c483b9a88ee6bbf16ba5c951ee1ecfcf723078e0d"
dependencies = [
"byteorder",
"crunchy",
"rustc-hex",
"static_assertions",
]
[[package]]
name = "unicode-xid"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "826e7639553986605ec5979c7dd957c7895e93eabed50ab2ffa7f6128a75097c"
[[package]]
name = "wasi"
version = "0.9.0+wasi-snapshot-preview1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cccddf32554fecc6acb585f82a32a72e28b48f8c4c1883ddfeeeaa96f7d8e519"
[[package]]
name = "winapi"
version = "0.3.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8093091eeb260906a183e6ae1abdba2ef5ef2257a21801128899c3fc699229c6"
dependencies = [
"winapi-i686-pc-windows-gnu",
"winapi-x86_64-pc-windows-gnu",
]
[[package]]
name = "winapi-i686-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac3b87c63620426dd9b991e5ce0329eff545bccbbb34f3be09ff6fb6ab51b7b6"
[[package]]
name = "winapi-x86_64-pc-windows-gnu"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "712e227841d057c1ee1cd2fb22fa7e5a5461ae8e48fa2ca79ec42cfc1931183f"
@@ -20,6 +20,10 @@ num-traits = "0.2"
name = "biguint"
path = "src/biguint.rs"
[[bin]]
name = "per_thing_rational"
path = "src/per_thing_rational.rs"
[[bin]]
name = "rational128"
path = "src/rational128.rs"
@@ -0,0 +1,123 @@
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
//! # Running
//! Running this fuzzer can be done with `cargo hfuzz run per_thing_rational`. `honggfuzz` CLI options can
//! be used by setting `HFUZZ_RUN_ARGS`, such as `-n 4` to use 4 threads.
//!
//! # Debugging a panic
//! Once a panic is found, it can be debugged with
//! `cargo hfuzz run-debug per_thing_rational hfuzz_workspace/per_thing_rational/*.fuzz`.
use honggfuzz::fuzz;
use sp_arithmetic::{
PerThing, PerU16, Percent, Perbill, Perquintill, assert_eq_error_rate,
traits::SaturatedConversion,
};
fn main() {
loop {
fuzz!(|
data: ((u16, u16), (u32, u32), (u64, u64))
| {
let (u16_pair, u32_pair, u64_pair) = data;
// peru16
let (smaller, bigger) = (u16_pair.0.min(u16_pair.1), u16_pair.0.max(u16_pair.1));
let ratio = PerU16::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
PerU16::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u32_pair.0.min(u32_pair.1), u32_pair.0.max(u32_pair.1));
let ratio = PerU16::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
PerU16::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = PerU16::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
PerU16::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
// percent
let (smaller, bigger) = (u16_pair.0.min(u16_pair.1), u16_pair.0.max(u16_pair.1));
let ratio = Percent::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Percent::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u32_pair.0.min(u32_pair.1), u32_pair.0.max(u32_pair.1));
let ratio = Percent::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Percent::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = Percent::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Percent::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
// perbill
let (smaller, bigger) = (u32_pair.0.min(u32_pair.1), u32_pair.0.max(u32_pair.1));
let ratio = Perbill::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Perbill::from_fraction(smaller as f64 / bigger.max(1) as f64),
100,
);
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = Perbill::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Perbill::from_fraction(smaller as f64 / bigger.max(1) as f64),
100,
);
// perquintillion
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = Perquintill::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Perquintill::from_fraction(smaller as f64 / bigger.max(1) as f64),
1000,
);
})
}
}
fn assert_per_thing_equal_error<T: PerThing>(a: T, b: T, err: u128) {
let a_abs = a.deconstruct().saturated_into::<u128>();
let b_abs = b.deconstruct().saturated_into::<u128>();
let diff = a_abs.max(b_abs) - a_abs.min(b_abs);
dbg!(&diff);
assert!(diff <= err, "{:?} !~ {:?}", a, b);
}
+14 -2
View File
@@ -19,7 +19,7 @@
#![cfg_attr(not(feature = "std"), no_std)]
/// Copied from `sp-runtime` and documented there.
#[cfg(test)]
#[macro_export]
macro_rules! assert_eq_error_rate {
($x:expr, $y:expr, $error:expr $(,)?) => {
assert!(
@@ -40,5 +40,17 @@ mod fixed64;
mod rational128;
pub use fixed64::Fixed64;
pub use per_things::{PerThing, Percent, Permill, Perbill, Perquintill};
pub use per_things::{PerThing, Percent, PerU16, Permill, Perbill, Perquintill};
pub use rational128::Rational128;
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn peru16_rational_does_not_overflow() {
// A historical example that will panic only for per_thing type that are created with
// maximum capacity of their type, e.g. PerU16.
let _ = PerU16::from_rational_approximation(17424870u32, 17424870);
}
}
+288 -119
View File
@@ -17,18 +17,24 @@
#[cfg(feature = "std")]
use serde::{Serialize, Deserialize};
use sp_std::{ops, prelude::*, convert::TryInto};
use sp_std::{ops, fmt, prelude::*, convert::TryInto};
use codec::{Encode, Decode, CompactAs};
use crate::traits::{
SaturatedConversion, UniqueSaturatedInto, Saturating, BaseArithmetic,
use crate::{
traits::{SaturatedConversion, UniqueSaturatedInto, Saturating, BaseArithmetic, Bounded},
};
use sp_debug_derive::RuntimeDebug;
/// Something that implements a fixed point ration with an arbitrary granularity `X`, as _parts per
/// `X`_.
pub trait PerThing: Sized + Saturating + Copy {
pub trait PerThing:
Sized + Saturating + Copy + Default + Eq + PartialEq + Ord + PartialOrd + Bounded + fmt::Debug
{
/// The data type used to build this per-thingy.
type Inner: BaseArithmetic + Copy;
type Inner: BaseArithmetic + Copy + fmt::Debug;
/// The data type that is used to store values bigger than the maximum of this type. This must
/// at least be able to store `Self::ACCURACY * Self::ACCURACY`.
type Upper: BaseArithmetic + Copy + fmt::Debug;
/// accuracy of this type
const ACCURACY: Self::Inner;
@@ -63,12 +69,53 @@ pub trait PerThing: Sized + Saturating + Copy {
/// The computation of this approximation is performed in the generic type `N`. Given
/// `M` as the data type that can hold the maximum value of this per-thing (e.g. u32 for
/// perbill), this can only work if `N == M` or `N: From<M> + TryInto<M>`.
///
/// Note that this always rounds _down_, i.e.
///
/// ```rust
/// # use sp_arithmetic::{Percent, PerThing};
/// # fn main () {
/// // 989/100 is technically closer to 99%.
/// assert_eq!(
/// Percent::from_rational_approximation(989, 1000),
/// Percent::from_parts(98),
/// );
/// # }
/// ```
fn from_rational_approximation<N>(p: N, q: N) -> Self
where N: Clone + Ord + From<Self::Inner> + TryInto<Self::Inner> + ops::Div<N, Output=N>;
where N:
Clone + Ord + From<Self::Inner> + TryInto<Self::Inner> + TryInto<Self::Upper> +
ops::Div<N, Output=N> + ops::Rem<N, Output=N> + ops::Add<N, Output=N>;
/// A mul implementation that always rounds down, whilst the standard `Mul` implementation
/// rounds to the nearest numbers
///
/// ```rust
/// # use sp_arithmetic::{Percent, PerThing};
/// # fn main () {
/// // rounds to closest
/// assert_eq!(Percent::from_percent(34) * 10u64, 3);
/// assert_eq!(Percent::from_percent(36) * 10u64, 4);
///
/// // collapse down
/// assert_eq!(Percent::from_percent(34).mul_collapse(10u64), 3);
/// assert_eq!(Percent::from_percent(36).mul_collapse(10u64), 3);
/// # }
/// ```
fn mul_collapse<N>(self, b: N) -> N
where N: Clone + From<Self::Inner> + UniqueSaturatedInto<Self::Inner> + ops::Rem<N, Output=N>
+ ops::Div<N, Output=N> + ops::Mul<N, Output=N> + ops::Add<N, Output=N>;
}
macro_rules! implement_per_thing {
($name:ident, $test_mod:ident, [$($test_units:tt),+], $max:tt, $type:ty, $upper_type:ty, $title:expr $(,)?) => {
(
$name:ident,
$test_mod:ident,
[$($test_units:tt),+],
$max:tt, $type:ty,
$upper_type:ty,
$title:expr $(,)?
) => {
/// A fixed point representation of a number between in the range [0, 1].
///
#[doc = $title]
@@ -78,33 +125,30 @@ macro_rules! implement_per_thing {
impl PerThing for $name {
type Inner = $type;
type Upper = $upper_type;
/// The accuracy of this type.
const ACCURACY: Self::Inner = $max;
/// Nothing.
fn zero() -> Self { Self(0) }
/// `true` if this is nothing.
fn is_zero(&self) -> bool { self.0 == 0 }
/// Everything.
fn one() -> Self { Self($max) }
/// Consume self and deconstruct into a raw numeric type.
fn deconstruct(self) -> Self::Inner { self.0 }
/// From an explicitly defined number of parts per maximum of the type.
// needed only for peru16. Since peru16 is the only type in which $max ==
// $type::max_value(), rustc is being a smart-a** here by warning that the comparison
// is not needed.
#[allow(unused_comparisons)]
fn from_parts(parts: Self::Inner) -> Self {
Self([parts, $max][(parts > $max) as usize])
}
/// Converts a percent into `Self`. Equal to `x / 100`.
fn from_percent(x: Self::Inner) -> Self {
Self([x, 100][(x > 100) as usize] * ($max / 100))
Self::from_rational_approximation([x, 100][(x > 100) as usize] as $upper_type, 100)
}
/// Return the product of multiplication of this value by itself.
fn square(self) -> Self {
// both can be safely casted and multiplied.
let p: $upper_type = self.0 as $upper_type * self.0 as $upper_type;
@@ -112,39 +156,43 @@ macro_rules! implement_per_thing {
Self::from_rational_approximation(p, q)
}
/// Converts a fraction into `Self`.
#[cfg(feature = "std")]
fn from_fraction(x: f64) -> Self { Self((x * ($max as f64)) as Self::Inner) }
/// Approximate the fraction `p/q` into a per-thing fraction. This will never overflow.
///
/// The computation of this approximation is performed in the generic type `N`. Given
/// `M` as the data type that can hold the maximum value of this per-thing (e.g. u32 for
/// perbill), this can only work if `N == M` or `N: From<M> + TryInto<M>`.
fn from_rational_approximation<N>(p: N, q: N) -> Self
where N: Clone + Ord + From<Self::Inner> + TryInto<Self::Inner> + ops::Div<N, Output=N>
where N:
Clone + Ord + From<Self::Inner> + TryInto<Self::Inner> + TryInto<Self::Upper> +
ops::Div<N, Output=N> + ops::Rem<N, Output=N> + ops::Add<N, Output=N>
{
let div_ceil = |x: N, f: N| -> N {
let mut o = x.clone() / f.clone();
let r = x.rem(f.clone());
if r > N::from(0) {
o = o + N::from(1);
}
o
};
// q cannot be zero.
let q = q.max((1 as Self::Inner).into());
let q: N = q.max((1 as Self::Inner).into());
// p should not be bigger than q.
let p = p.min(q.clone());
let p: N = p.min(q.clone());
let factor = (q.clone() / $max.into()).max((1 as Self::Inner).into());
let factor: N = div_ceil(q.clone(), $max.into()).max((1 as Self::Inner).into());
// q cannot overflow: (q / (q/$max)) < 2 * $max. p < q hence p also cannot overflow.
// this implies that Self::Inner must be able to fit 2 * $max.
let q_reduce: Self::Inner = (q / factor.clone())
// q cannot overflow: (q / (q/$max)) < $max. p < q hence p also cannot overflow.
let q_reduce: $type = (q.clone() / factor.clone())
.try_into()
.map_err(|_| "Failed to convert")
.expect(
"q / (q/$max) < (2 * $max). Macro prevents any type being created that \
"q / ceil(q/$max) < $max. Macro prevents any type being created that \
does not satisfy this; qed"
);
let p_reduce: Self::Inner = (p / factor.clone())
let p_reduce: $type = (p / factor)
.try_into()
.map_err(|_| "Failed to convert")
.expect(
"q / (q/$max) < (2 * $max). Macro prevents any type being created that \
"q / ceil(q/$max) < $max. Macro prevents any type being created that \
does not satisfy this; qed"
);
@@ -157,13 +205,49 @@ macro_rules! implement_per_thing {
$name(part as Self::Inner)
}
fn mul_collapse<N>(self, b: N) -> N
where
N: Clone + From<$type> + UniqueSaturatedInto<$type> + ops::Rem<N, Output=N>
+ ops::Div<N, Output=N> + ops::Mul<N, Output=N> + ops::Add<N, Output=N>
{
let maximum: N = $max.into();
let upper_max: $upper_type = $max.into();
let part: N = self.0.into();
let rem_multiplied_divided = {
let rem = b.clone().rem(maximum.clone());
// `rem_sized` is inferior to $max, thus it fits into $type. This is assured by
// a test.
let rem_sized = rem.saturated_into::<$type>();
// `self` and `rem_sized` are inferior to $max, thus the product is less than
// $max^2 and fits into $upper_type. This is assured by a test.
let rem_multiplied_upper = rem_sized as $upper_type * self.0 as $upper_type;
// `rem_multiplied_upper` is less than $max^2 therefore divided by $max it fits
// in $type. remember that $type always fits $max.
let rem_multiplied_divided_sized =
(rem_multiplied_upper / upper_max) as $type;
// `rem_multiplied_divided_sized` is inferior to b, thus it can be converted
// back to N type
rem_multiplied_divided_sized.into()
};
(b / maximum) * part + rem_multiplied_divided
}
}
/// Implement const functions
impl $name {
/// From an explicitly defined number of parts per maximum of the type.
///
/// This can be called at compile time.
// needed only for peru16. Since peru16 is the only type in which $max ==
// $type::max_value(), rustc is being a smart-a** here by warning that the comparison
// is not needed.
#[allow(unused_comparisons)]
pub const fn from_parts(parts: $type) -> Self {
Self([parts, $max][(parts > $max) as usize])
}
@@ -202,6 +286,16 @@ macro_rules! implement_per_thing {
}
}
impl crate::traits::Bounded for $name {
fn min_value() -> Self {
<Self as PerThing>::zero()
}
fn max_value() -> Self {
<Self as PerThing>::one()
}
}
impl ops::Div for $name {
type Output = Self;
@@ -212,9 +306,10 @@ macro_rules! implement_per_thing {
}
}
/// Overflow-prune multiplication.
/// Non-overflow multiplication.
///
/// tailored to be used with a balance type.
///
impl<N> ops::Mul<N> for $name
where
N: Clone + From<$type> + UniqueSaturatedInto<$type> + ops::Rem<N, Output=N>
@@ -241,6 +336,7 @@ macro_rules! implement_per_thing {
// in $type. remember that $type always fits $max.
let mut rem_multiplied_divided_sized =
(rem_multiplied_upper / upper_max) as $type;
// fix a tiny rounding error
if rem_multiplied_upper % upper_max > upper_max / 2 {
rem_multiplied_divided_sized += 1;
@@ -261,15 +357,17 @@ macro_rules! implement_per_thing {
use super::{$name, Saturating, RuntimeDebug, PerThing};
use crate::traits::Zero;
#[test]
fn macro_expanded_correctly() {
// needed for the `from_percent` to work.
assert!($max >= 100);
assert!($max % 100 == 0);
// needed for the `from_percent` to work. UPDATE: this is no longer needed; yet note
// that tests that use percentage or fractions such as $name::from_fraction(0.2) to
// create values will most likely be inaccurate when used with per_things that are
// not multiples of 100.
// assert!($max >= 100);
// assert!($max % 100 == 0);
// needed for `from_rational_approximation`
assert!(2 * $max < <$type>::max_value());
assert!(2 * ($max as $upper_type) < <$upper_type>::max_value());
assert!(<$upper_type>::from($max) < <$upper_type>::max_value());
// for something like percent they can be the same.
@@ -298,7 +396,7 @@ macro_rules! implement_per_thing {
(63, 1),
(64, 2),
(65, 2),
(<$type>::max_value(), <$type>::max_value().encode().len() + 1)
// (<$type>::max_value(), <$type>::max_value().encode().len() + 1)
];
for &(n, l) in &tests {
let compact: codec::Compact<$name> = $name(n).into();
@@ -317,33 +415,73 @@ macro_rules! implement_per_thing {
assert_eq!($name::zero(), $name::from_parts(Zero::zero()));
assert_eq!($name::one(), $name::from_parts($max));
assert_eq!($name::ACCURACY, $max);
assert_eq!($name::from_percent(0), $name::from_parts(Zero::zero()));
assert_eq!($name::from_percent(10), $name::from_parts($max / 10));
assert_eq!($name::from_percent(100), $name::from_parts($max));
assert_eq!($name::from_fraction(0.0), $name::from_parts(Zero::zero()));
assert_eq!($name::from_fraction(0.1), $name::from_parts($max / 10));
assert_eq!($name::from_fraction(1.0), $name::from_parts($max));
}
macro_rules! u256ify {
($val:expr) => {
Into::<U256>::into($val)
};
}
macro_rules! per_thing_mul_test {
($num_type:tt) => {
// multiplication from all sort of from_percent
assert_eq!(
$name::from_percent(100) * $num_type::max_value(),
$name::from_fraction(1.0) * $num_type::max_value(),
$num_type::max_value()
);
assert_eq_error_rate!(
$name::from_percent(99) * $num_type::max_value(),
((Into::<U256>::into($num_type::max_value()) * 99u32) / 100u32).as_u128() as $num_type,
1,
);
assert_eq!(
$name::from_percent(50) * $num_type::max_value(),
$num_type::max_value() / 2,
);
assert_eq_error_rate!(
$name::from_percent(1) * $num_type::max_value(),
$num_type::max_value() / 100,
1,
);
assert_eq!($name::from_percent(0) * $num_type::max_value(), 0);
if $max % 100 == 0 {
assert_eq_error_rate!(
$name::from_percent(99) * $num_type::max_value(),
((Into::<U256>::into($num_type::max_value()) * 99u32) / 100u32).as_u128() as $num_type,
1,
);
assert_eq!(
$name::from_fraction(0.5) * $num_type::max_value(),
$num_type::max_value() / 2,
);
assert_eq_error_rate!(
$name::from_percent(1) * $num_type::max_value(),
$num_type::max_value() / 100,
1,
);
} else {
assert_eq!(
$name::from_fraction(0.99) * <$num_type>::max_value(),
(
(
u256ify!($name::from_fraction(0.99).0) *
u256ify!(<$num_type>::max_value()) /
u256ify!($max)
).as_u128()
) as $num_type,
);
assert_eq!(
$name::from_fraction(0.50) * <$num_type>::max_value(),
(
(
u256ify!($name::from_fraction(0.50).0) *
u256ify!(<$num_type>::max_value()) /
u256ify!($max)
).as_u128()
) as $num_type,
);
assert_eq!(
$name::from_fraction(0.01) * <$num_type>::max_value(),
(
(
u256ify!($name::from_fraction(0.01).0) *
u256ify!(<$num_type>::max_value()) /
u256ify!($max)
).as_u128()
) as $num_type,
);
}
assert_eq!($name::from_fraction(0.0) * $num_type::max_value(), 0);
// // multiplication with bounds
assert_eq!($name::one() * $num_type::max_value(), $num_type::max_value());
@@ -356,17 +494,20 @@ macro_rules! implement_per_thing {
use primitive_types::U256;
// accuracy test
assert_eq!($name::from_rational_approximation(1 as $type, 3) * 30 as $type, 10);
assert_eq!(
$name::from_rational_approximation(1 as $type, 3) * 30 as $type,
10,
);
$(per_thing_mul_test!($test_units);)*
}
#[test]
fn per_thing_mul_rounds_to_nearest_number() {
assert_eq!($name::from_percent(33) * 10u64, 3);
assert_eq!($name::from_percent(34) * 10u64, 3);
assert_eq!($name::from_percent(35) * 10u64, 3);
assert_eq!($name::from_percent(36) * 10u64, 4);
assert_eq!($name::from_fraction(0.33) * 10u64, 3);
assert_eq!($name::from_fraction(0.34) * 10u64, 3);
assert_eq!($name::from_fraction(0.35) * 10u64, 3);
assert_eq!($name::from_fraction(0.36) * 10u64, 4);
}
#[test]
@@ -398,31 +539,32 @@ macro_rules! implement_per_thing {
);
assert_eq!(
$name::from_rational_approximation(1 as $num_type, 10),
$name::from_percent(10),
$name::from_fraction(0.10),
);
assert_eq!(
$name::from_rational_approximation(1 as $num_type, 4),
$name::from_percent(25),
$name::from_fraction(0.25),
);
assert_eq!(
$name::from_rational_approximation(1 as $num_type, 4),
$name::from_rational_approximation(2 as $num_type, 8),
);
// no accurate anymore but won't overflow.
assert_eq!(
assert_eq_error_rate!(
$name::from_rational_approximation(
$num_type::max_value() - 1,
$num_type::max_value()
),
$name::one(),
).0 as $upper_type,
$name::one().0 as $upper_type,
2,
);
assert_eq_error_rate!(
$name::from_rational_approximation(
$num_type::max_value() / 3,
$num_type::max_value()
).0,
$name::from_parts($max / 3).0,
2
).0 as $upper_type,
$name::from_parts($max / 3).0 as $upper_type,
2,
);
assert_eq!(
$name::from_rational_approximation(1, $num_type::max_value()),
@@ -436,13 +578,14 @@ macro_rules! implement_per_thing {
// This is just to make sure something like Percent which _might_ get built from a
// u8 does not overflow in the context of this test.
let max_value = <$upper_type>::from($max);
// almost at the edge
assert_eq!(
$name::from_rational_approximation($max - 1, $max + 1),
$name::from_rational_approximation(max_value - 1, max_value + 1),
$name::from_parts($max - 2),
);
assert_eq!(
$name::from_rational_approximation(1, $max-1),
$name::from_rational_approximation(1, $max - 1),
$name::from_parts(1),
);
assert_eq!(
@@ -450,76 +593,83 @@ macro_rules! implement_per_thing {
$name::from_parts(1),
);
assert_eq!(
$name::from_rational_approximation(2, 2 * $max - 1),
$name::from_rational_approximation(2, 2 * max_value - 1),
$name::from_parts(1),
);
assert_eq!(
$name::from_rational_approximation(1, $max+1),
$name::from_rational_approximation(1, max_value + 1),
$name::zero(),
);
assert_eq!(
$name::from_rational_approximation(3 * max_value / 2, 3 * max_value),
$name::from_percent(50),
$name::from_fraction(0.5),
);
$(per_thing_from_rationale_approx_test!($test_units);)*
}
#[test]
fn per_things_mul_operates_in_output_type() {
// assert_eq!($name::from_percent(50) * 100u32, 50u32);
assert_eq!($name::from_percent(50) * 100u64, 50u64);
assert_eq!($name::from_percent(50) * 100u128, 50u128);
// assert_eq!($name::from_fraction(0.5) * 100u32, 50u32);
assert_eq!($name::from_fraction(0.5) * 100u64, 50u64);
assert_eq!($name::from_fraction(0.5) * 100u128, 50u128);
}
#[test]
fn per_thing_saturating_op_works() {
assert_eq!(
$name::from_percent(50).saturating_add($name::from_percent(40)),
$name::from_percent(90)
assert_eq_error_rate!(
$name::from_fraction(0.5).saturating_add($name::from_fraction(0.4)).0 as $upper_type,
$name::from_fraction(0.9).0 as $upper_type,
2,
);
assert_eq_error_rate!(
$name::from_fraction(0.5).saturating_add($name::from_fraction(0.5)).0 as $upper_type,
$name::one().0 as $upper_type,
2,
);
assert_eq!(
$name::from_percent(50).saturating_add($name::from_percent(50)),
$name::from_percent(100)
);
assert_eq!(
$name::from_percent(60).saturating_add($name::from_percent(50)),
$name::from_percent(100)
$name::from_fraction(0.6).saturating_add($name::from_fraction(0.5)),
$name::one(),
);
assert_eq!(
$name::from_percent(60).saturating_sub($name::from_percent(50)),
$name::from_percent(10)
assert_eq_error_rate!(
$name::from_fraction(0.6).saturating_sub($name::from_fraction(0.5)).0 as $upper_type,
$name::from_fraction(0.1).0 as $upper_type,
2,
);
assert_eq!(
$name::from_percent(60).saturating_sub($name::from_percent(60)),
$name::from_percent(0)
$name::from_fraction(0.6).saturating_sub($name::from_fraction(0.6)),
$name::from_fraction(0.0),
);
assert_eq!(
$name::from_percent(60).saturating_sub($name::from_percent(70)),
$name::from_percent(0)
$name::from_fraction(0.6).saturating_sub($name::from_fraction(0.7)),
$name::from_fraction(0.0),
);
assert_eq!(
$name::from_percent(50).saturating_mul($name::from_percent(50)),
$name::from_percent(25)
assert_eq_error_rate!(
$name::from_fraction(0.5).saturating_mul($name::from_fraction(0.5)).0 as $upper_type,
$name::from_fraction(0.25).0 as $upper_type,
2,
);
assert_eq!(
$name::from_percent(20).saturating_mul($name::from_percent(20)),
$name::from_percent(4)
assert_eq_error_rate!(
$name::from_fraction(0.2).saturating_mul($name::from_fraction(0.2)).0 as $upper_type,
$name::from_fraction(0.04).0 as $upper_type,
2,
);
assert_eq!(
$name::from_percent(10).saturating_mul($name::from_percent(10)),
$name::from_percent(1)
assert_eq_error_rate!(
$name::from_fraction(0.1).saturating_mul($name::from_fraction(0.1)).0 as $upper_type,
$name::from_fraction(0.01).0 as $upper_type,
1,
);
}
#[test]
fn per_thing_square_works() {
assert_eq!($name::from_percent(100).square(), $name::from_percent(100));
assert_eq!($name::from_percent(50).square(), $name::from_percent(25));
assert_eq!($name::from_percent(10).square(), $name::from_percent(1));
assert_eq!($name::from_fraction(1.0).square(), $name::from_fraction(1.0));
assert_eq!($name::from_fraction(0.5).square(), $name::from_fraction(0.25));
assert_eq!($name::from_fraction(0.1).square(), $name::from_fraction(0.01));
assert_eq!(
$name::from_percent(2).square(),
$name::from_fraction(0.02).square(),
$name::from_parts((4 * <$upper_type>::from($max) / 100 / 100) as $type)
);
}
@@ -527,22 +677,32 @@ macro_rules! implement_per_thing {
#[test]
fn per_things_div_works() {
// normal
assert_eq!($name::from_percent(10) / $name::from_percent(20),
$name::from_percent(50)
assert_eq_error_rate!(
($name::from_fraction(0.1) / $name::from_fraction(0.20)).0 as $upper_type,
$name::from_fraction(0.50).0 as $upper_type,
2,
);
assert_eq!($name::from_percent(10) / $name::from_percent(10),
$name::from_percent(100)
assert_eq_error_rate!(
($name::from_fraction(0.1) / $name::from_fraction(0.10)).0 as $upper_type,
$name::from_fraction(1.0).0 as $upper_type,
2,
);
assert_eq!($name::from_percent(10) / $name::from_percent(0),
$name::from_percent(100)
assert_eq_error_rate!(
($name::from_fraction(0.1) / $name::from_fraction(0.0)).0 as $upper_type,
$name::from_fraction(1.0).0 as $upper_type,
2,
);
// will not overflow
assert_eq!($name::from_percent(10) / $name::from_percent(5),
$name::from_percent(100)
assert_eq_error_rate!(
($name::from_fraction(0.10) / $name::from_fraction(0.05)).0 as $upper_type,
$name::from_fraction(1.0).0 as $upper_type,
2,
);
assert_eq!($name::from_percent(100) / $name::from_percent(50),
$name::from_percent(100)
assert_eq_error_rate!(
($name::from_fraction(1.0) / $name::from_fraction(0.5)).0 as $upper_type,
$name::from_fraction(1.0).0 as $upper_type,
2,
);
}
}
@@ -558,6 +718,15 @@ implement_per_thing!(
u16,
"_Percent_",
);
implement_per_thing!(
PerU16,
test_peru16,
[u32, u64, u128],
65535_u16,
u16,
u32,
"_Parts per 65535_",
);
implement_per_thing!(
Permill,
test_permill,