Offchain Phragmén BREAKING. (#4517)

* Initial skeleton for offchain phragmen

* Basic compact encoding decoding for results

* add compact files

* Bring back Self::ensure_storage_upgraded();

* Make staking use compact stuff.

* First seemingly working version of reduce, full of todos

* Everything phragmen related works again.

* Signing made easier, still issues.

* Signing from offchain compile fine 😎

* make compact work with staked asssignment

* Evaluation basics are in place.

* Move reduce into crate. Document stuff

* move reduce into no_std

* Add files

* Remove other std deps. Runtime compiles

* Seemingly it is al stable; cycle implemented but not integrated.

* Add fuzzing code.

* Cleanup reduce a bit more.

* a metric ton of tests for staking; wip 🔨

* Implement a lot more of the tests.

* wip getting the unsigned stuff to work

* A bit gleanup for unsigned debug

* Clean and finalize compact code.

* Document reduce.

* Still problems with signing

* We officaly duct taped the transaction submission stuff. 🤓

* Deadlock with keys again

* Runtime builds

* Unsigned test works 🙌

* Some cleanups

* Make all the tests compile and stuff

* Minor cleanup

* fix more merge stuff

* Most tests work again.

* a very nasty bug in reduce

* Fix all integrations

* Fix more todos

* Revamp everything and everything

* Remove bogus test

* Some review grumbles.

* Some fixes

* Fix doc test

* loop for submission

* Fix cli, keyring etc.

* some cleanup

* Fix staking tests again

* fix per-things; bring patches from benchmarking

* better score prediction

* Add fuzzer, more patches.

* Some fixes

* More docs

* Remove unused generics

* Remove max-nominator footgun

* Better fuzzer

* Disable it 

* Bump.

* Another round of self-review

* Refactor a lot

* More major fixes in perThing

* Add new fuzz file

* Update lock

* fix fuzzing code.

* Fix nominator retain test

* Add slashing check

* Update frame/staking/src/tests.rs

Co-Authored-By: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>

* Some formatting nits

* Review comments.

* Fix cargo file

* Almost all tests work again

* Update frame/staking/src/tests.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Fix review comments

* More review stuff

* Some nits

* Fix new staking / session / babe relation

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Update primitives/phragmen/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Update primitives/phragmen/compact/src/lib.rs

Co-Authored-By: thiolliere <gui.thiolliere@gmail.com>

* Some doc updates to slashing

* Fix derive

* Remove imports

* Remove unimplemented tests

* nits

* Remove dbg

* Better fuzzing params

* Remove unused pref map

* Deferred Slashing/Offence for offchain Phragmen  (#5151)

* Some boilerplate

* Add test

* One more test

* Review comments

* Fix build

* review comments

* fix more

* fix build

* Some cleanups and self-reviews

* More minor self reviews

* Final nits

* Some merge fixes.

* opt comment

* Fix build

* Fix build again.

* Update frame/staking/fuzz/fuzz_targets/submit_solution.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update frame/staking/src/slashing.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Update frame/staking/src/offchain_election.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

* Fix review comments

* fix test

* === 🔑 Revamp without staking key.

* final round of changes.

* Fix cargo-deny

* Update frame/staking/src/lib.rs

Co-Authored-By: Gavin Wood <gavin@parity.io>

Co-authored-by: Joshy Orndorff <JoshOrndorff@users.noreply.github.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Gavin Wood <gavin@parity.io>
This commit is contained in:
Kian Paimani
2020-03-26 15:37:40 +01:00
committed by GitHub
parent 2a67e6c437
commit 970c5f94f2
64 changed files with 11953 additions and 892 deletions
@@ -0,0 +1,123 @@
// Copyright 2019-2020 Parity Technologies (UK) Ltd.
// This file is part of Substrate.
// Substrate is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// Substrate is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with Substrate. If not, see <http://www.gnu.org/licenses/>.
//! # Running
//! Running this fuzzer can be done with `cargo hfuzz run per_thing_rational`. `honggfuzz` CLI options can
//! be used by setting `HFUZZ_RUN_ARGS`, such as `-n 4` to use 4 threads.
//!
//! # Debugging a panic
//! Once a panic is found, it can be debugged with
//! `cargo hfuzz run-debug per_thing_rational hfuzz_workspace/per_thing_rational/*.fuzz`.
use honggfuzz::fuzz;
use sp_arithmetic::{
PerThing, PerU16, Percent, Perbill, Perquintill, assert_eq_error_rate,
traits::SaturatedConversion,
};
fn main() {
loop {
fuzz!(|
data: ((u16, u16), (u32, u32), (u64, u64))
| {
let (u16_pair, u32_pair, u64_pair) = data;
// peru16
let (smaller, bigger) = (u16_pair.0.min(u16_pair.1), u16_pair.0.max(u16_pair.1));
let ratio = PerU16::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
PerU16::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u32_pair.0.min(u32_pair.1), u32_pair.0.max(u32_pair.1));
let ratio = PerU16::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
PerU16::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = PerU16::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
PerU16::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
// percent
let (smaller, bigger) = (u16_pair.0.min(u16_pair.1), u16_pair.0.max(u16_pair.1));
let ratio = Percent::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Percent::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u32_pair.0.min(u32_pair.1), u32_pair.0.max(u32_pair.1));
let ratio = Percent::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Percent::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = Percent::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Percent::from_fraction(smaller as f64 / bigger.max(1) as f64),
1,
);
// perbill
let (smaller, bigger) = (u32_pair.0.min(u32_pair.1), u32_pair.0.max(u32_pair.1));
let ratio = Perbill::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Perbill::from_fraction(smaller as f64 / bigger.max(1) as f64),
100,
);
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = Perbill::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Perbill::from_fraction(smaller as f64 / bigger.max(1) as f64),
100,
);
// perquintillion
let (smaller, bigger) = (u64_pair.0.min(u64_pair.1), u64_pair.0.max(u64_pair.1));
let ratio = Perquintill::from_rational_approximation(smaller, bigger);
assert_per_thing_equal_error(
ratio,
Perquintill::from_fraction(smaller as f64 / bigger.max(1) as f64),
1000,
);
})
}
}
fn assert_per_thing_equal_error<T: PerThing>(a: T, b: T, err: u128) {
let a_abs = a.deconstruct().saturated_into::<u128>();
let b_abs = b.deconstruct().saturated_into::<u128>();
let diff = a_abs.max(b_abs) - a_abs.min(b_abs);
dbg!(&diff);
assert!(diff <= err, "{:?} !~ {:?}", a, b);
}