Introduce on_runtime_upgrade (#5058)

* Initial idea of `on_runtime_upgrade`

* Runtime storage for module version

* Gui shawntabrizi runtime upgrade (#5118)

* adding unleash to ci (#5020)

* adding unleash to ci

* fixing formatting

* with a dot please

* alpha.3 now

* do not publish testing helpers

* remove old test-helpers cruft

* fix cargo.lock

* with alpha 4

* do not publish runtime-interface-test either

* disable more test crates from publishing

* switch to alpha.5

* replace tempdir with tempfile

* update lru

* switch to bytes 0.5

* release script fixes

* switch on and to latest alpha

* BUT THE SPACES

* Fix: CI failing for some CLI tests (#5043)

* Initial commit

Forked at: ad90ab7ec9
Parent branch: origin/master

* Increase killing grace period of CLI tests and display more info

* Use --dev everywhere possible

* Put pruning mode to its own params struct

* Add pruning params to export-blocks command

* Added missing file

* Removed not-dev mode in tests

* Add pruning mode to the revert command

* Decrease killing grace period again

* Move back unsafe_pruning to import_params

* Applied proposed changes

* aura: remove unused tx pool (#5046)

* aura: remove unused transaction pool parameter

* node-template: remove transaction pool from aura

* aura: fix tests

* Extend rust nightly detection in `wasm-builder` (#5021)

Instead of just testing `cargo` and `rustup run nightly`, we now test
the `CARGO` env variable and also scan non default nightlies. The user
is also now able to select the toolchain with `WASM_BUILD_TOOLCHAIN`.

* Add steps setting to benchmarking CLI (#5033)

* Add steps setting to CLI, use max value to hit worst case.

* Bump impl_version.

* Apply review suggestion.

* Remove indices from node-template (#5025)

* Remove indices from node-template

* Use identity lookup instead

* Bump impl

* clean cargo.toml

* Fix documentation for "BlockBuilder::push_trusted" (#5051)

* fix doc

* rephrase

* do not check unleash on every PR, only master and tags (#5054)

* do not check unleash on every PR, only master and tags

* move scripts folder

* add signed-tag check to CI

* remove publish-to-crates-io dependencies

Co-authored-by: s3krit <pugh@s3kr.it>

* prepare version to alpha.1 (#5055)

bump version to -alpha.1

* Sync: validate block responses for required data (#5052)

* Less verbose state-db logging

* Validate block responses for block bodies

* Update client/network/src/protocol.rs

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

* Added validation test

* Disconnect on missing header as well

* Typo

Co-Authored-By: André Silva <andre.beat@gmail.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: André Silva <andre.beat@gmail.com>

* Make these chainspecs fields private (#5031)

* Fix dockerfile (#5059)

* Adds documentation for `wipe` and `commit` (#5053)

* Adds documentation for `wipe` and `commit`

This adds documentation to `wipe` and `commit` of `Externalities`.
Besides that it removes the default implementation that would just panic
and requires that all implementers of the trait implement the functions.

* Update primitives/externalities/src/lib.rs

Co-Authored-By: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>

* Fix the issue with `trybuild`'s `ui` tests (#4992)

* repro ui bug

* fix the tests

* test with the new image

* test without CARGO_HOME

* test without fixes

* test again

* fix trybuild old versions

* bump CArgo.lock

* fix trybuild newest versions

* bump Cargo.lock

* trying on the latest image

* bump Cargo.lock

* run with the old image

* ci will be green on the image from 2020-02-19 [skip ci]

* bump Cargo.lock

* Activate publishing of draft releases... (#5062)

* Activate publishing of draft releases...

... And fix the message sending (missing parameter).

* publish_draft_release.sh now checks latest...

... release on github rather than just a tag

* Fix/div by zero (#5041)

* Handle gas_price being zero separately

* Bump spec_version

* Add a unit & integration tests for gas price = 0

* set missing metadata fields, prepping alpha.2 (#5067)

* setting first batch of descriptions

* fix what I just broke

* next batch

* and pallets, too

* last batch

* set cargo.lock

* keep'em dev-deps

* bump version to alpha.2

* Fix revalidation not revalidating multiple times (#5065)

* removes use of sc_client::Client from sc_finality_grandpa (#5030)

* removes use of sc_client::Client from sc_finality_grandpa

* code formatting

* code formatting

* removes use of sc_client::Client from sc_finality_grandpa

* Remove deprecated host functions (#5038)

Sadly we need to keep one function `ext_blake2_256`. This function is
manually defined in `sp-core`.

* removes use of sc_client::Client from sc_basic_authorship (#5050)

* removes use of sc-client from sc-basic-authorship

* refactor use of ProposerFactory

* correct dep path

* pallet-transaction-payment clean up (#5070)

* Formatting clean up

* Introduce separate setters for the fees.

* *: Rename prometheus-exporter crate to substrate-prometheus-end… (#5076)

This patch renames the crate for the following two reasons:

1. The prometheus-exporter crate introduces native in-process Prometheus
style instrumentation to the Substrate project. Within the Prometheus
ecosystem the term "exporter" is used for external processes exposing
metrics for e.g. the Linux Kernel. In-process exposition would be
described via the term "endpoint".

2. "prometheus-exporter" is generic and ignores the fact that it is only
usable within the context of Substrate. In addition the name
"prometheus-exporter" is already taken on crates.io.

* rename `browser-utils` to `substrate-browser-utils` (#5079)

* prepping for Alpha.3 (#5080)

* Bump to alpha.3

* update gitlab-ci

* Propagate DispatchError for benchmarks. (#5075)

* Propagate DispatchError for benchmarks.

* Apply review suggestions.

* Use RuntimeString.

* fix expect

Co-Authored-By: Bastian Köcher <bkchr@users.noreply.github.com>

Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>

* Add options to overwrite range bounds in benchmark command. (#5072)

* Add --mins --maxs to benchmark command.

* Apply review suggestions.

* Update yamux to version 0.4.4. (#5086)

* Remove more instances of futures01 (#4633)

* Start removing last few instances of futures01

* Use to_poll on wasm

* Revert "Use to_poll on wasm"

This reverts commit 1c61728f10d520df5f9b28c415a0db68e478b9c7.

* Fix fg test

* Upgrade network test futures

* Update offchain hyper version

* Update service test

* bump tokio to 0.2.10

* Removed some unneeded tokios

* fixes

* fix run_until_all_full

* Make service test debuggable

* Update client/offchain/src/api/http.rs

Co-Authored-By: Demi Obenour <48690212+DemiMarie-parity@users.noreply.github.com>

* Add service_test to test-int output

* nitpicking

* Finally fix test

* Give up and revert client/serviec/test

* Revert gitlab ci too

Co-authored-by: Demi Obenour <demi@parity.io>

* Make export blocks default to json on stdout (#5090)

* Make export blocks default to json on stdout

* Multiline instead of single line to stay under 100 cols

* Change --json flag to --binary, defaulting to json

* Offence reporting returns a result (#5082)

* Offence reporting returns a result

* Bump spec_version

* Use unwrap instead of assertions

* Fix more review grumbles

* Update to libp2p 0.16.2 (#5088)

* Remove request ID from the new protocol (#5049)

* Make sure we remove a peer on disconnect in gossip (#5104)

* Make sure we remove peers on disconnect in gossip state machine

* Clear up the code

* Add a comment

* Expose `state-db` memory info (#5110)

This exposes memory statistics from the state-db.

* Change extrinsic_count to extrinsic_index in pallet-utility (#5044)

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>

* Add more metrics to prometheus (#5034)

* Add a few things

* Add finality_grandpa_round

* fix fg tests

* Nitpicks

* Nitpicks

* Fix name of prometheus crate

* Update to SCALE 1.2.0 (#5113)

This updates `parity-scale-codec` to `1.2.0`, which includes multiple
performance improvements and a fix that bounds the capacity of a vector
at decoding.

* Lazy payouts (#4474)

* TODOs

* Remove superfluous:

* partial implementation

* full implementation

* fix preferences

* update comments

* upgrade test WIP

* fix more tests

* fix cutoff

* fix saturation

* comment

* upgrade mock

* upgrade test

* WIP migration

* WIP migration

* remove slot stake stuff

* fix merge

* migration of ledger

* remove equalize from test

* add test

* fix

* update doc

* fix compilation

* improve test readibility

* improve doc

* fix most todo

* fix migration and test

* remove println

* WIP

* add test and spec

* weight

* update doc

* safer end_era

* fix exposure of conversion

* Revert "safer end_era"

This reverts commit 72ff737d27be67d87308514b13e2574bc5f09fce.

* fix useless put

* exposure clipped

* doc

* fix payout with clipped

* fix node runtime

* add doc

* pluggable and generalized staking module

* remove print

* update doc

* refactor

* improve documentation and implementation

* fix test

* Fix test

* fix test

* fix test

* fix remove lowest stake from exposure, not biggest.

* nomination index arguments in nominator_payout

* add test

* try to fix offence

* apply slashed and bond eras until active era

* doc

* update spec version

* add test upgrade from previous test environment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* nominators upgrade has been cleaned

* dynamic history depth implementation

* make current_era - history_depth included

* Change equality check to start era to less than or equal

* Use era specific validator prefs

* Add print statement and comment about start era if <

* fix next_reward overflow

* make more check for bad era claim for zero cost

* small refactor

* code refactor + fix use of deprecated storage

* fix wasm build

* add comment

* Fix tests

* remove outdated comment

* Apply suggestions from code review

Co-Authored-By: Shawn Tabrizi <shawntabrizi@gmail.com>

* gather active era information into one storage

Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>

* impl on_runtime_upgrade

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>

* make compile

* Add some tests

* docs

* Remove "useless" code

* Fix merge and use n + 1 block number

* Fix tests

* unfix ui tests

* Update on_initialize.stderr

* fix test

* Fix test

* Bump spec

* Remove `on_finalise` and `on_initialise`

* Use bool for tracking runtime upgraded

* typo

* Support runtime upgrade with `set_storage`

* Refactor migration code location

* add trailing newlines

* Remove old `IsUpgraded` flag

* Update state root

* Exhaustive match statement

* Apply suggestions from code review

Co-Authored-By: Kian Paimani <5588131+kianenigma@users.noreply.github.com>

Co-authored-by: Benjamin Kampmann <ben.kampmann@googlemail.com>
Co-authored-by: Cecile Tonglet <cecile.tonglet@cecton.com>
Co-authored-by: André Silva <andre.beat@gmail.com>
Co-authored-by: Bastian Köcher <bkchr@users.noreply.github.com>
Co-authored-by: Marcio Diaz <marcio@parity.io>
Co-authored-by: Nikolay Volf <nikvolf@gmail.com>
Co-authored-by: s3krit <pugh@s3kr.it>
Co-authored-by: Arkadiy Paronyan <arkady.paronyan@gmail.com>
Co-authored-by: Pierre Krieger <pierre.krieger1708@gmail.com>
Co-authored-by: Chevdor <chevdor@users.noreply.github.com>
Co-authored-by: joe petrowski <25483142+joepetrowski@users.noreply.github.com>
Co-authored-by: Denis Pisarev <denis.pisarev@parity.io>
Co-authored-by: Eric <e.lim0322@gmail.com>
Co-authored-by: Seun Lanlege <seunlanlege@gmail.com>
Co-authored-by: Sergei Pepyakin <s.pepyakin@gmail.com>
Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Ashley <ashley.ruglys@gmail.com>
Co-authored-by: Toralf Wittner <tw@dtex.org>
Co-authored-by: Demi Obenour <demi@parity.io>
Co-authored-by: pscott <30843220+pscott@users.noreply.github.com>
Co-authored-by: Fedor Sakharov <fedor.sakharov@gmail.com>
Co-authored-by: Gavin Wood <github@gavwood.com>
Co-authored-by: thiolliere <gui.thiolliere@gmail.com>
Co-authored-by: Kian Paimani <5588131+kianenigma@users.noreply.github.com>
This commit is contained in:
Gavin Wood
2020-03-05 13:44:18 +01:00
committed by GitHub
parent 906e08ee2a
commit df77c4bf37
21 changed files with 811 additions and 513 deletions
-218
View File
@@ -16,8 +16,6 @@
//! Tests for the module.
mod test_upgrade_from_master_dataset;
use super::*;
use mock::*;
use sp_runtime::{assert_eq_error_rate, traits::{OnInitialize, BadOrigin}};
@@ -26,10 +24,8 @@ use frame_support::{
assert_ok, assert_noop,
traits::{Currency, ReservableCurrency},
StorageMap,
storage::migration::{put_storage_value, get_storage_value},
};
use pallet_balances::Error as BalancesError;
use sp_io::hashing::blake2_256;
use substrate_test_utils::assert_eq_uvec;
use crate::Store;
@@ -2843,97 +2839,6 @@ fn claim_reward_at_the_last_era_and_no_double_claim_and_invalid_claim() {
});
}
#[test]
fn upgrade_works() {
ExtBuilder::default().build().execute_with(|| {
start_era(3);
assert_eq!(Session::validators(), vec![21, 11]);
// Insert fake data to check the migration
put_storage_value::<Vec<AccountId>>(b"Staking", b"CurrentElected", b"", vec![21, 31]);
put_storage_value::<SessionIndex>(b"Staking", b"CurrentEraStartSessionIndex", b"", 5);
put_storage_value::<MomentOf<Test>>(b"Staking", b"CurrentEraStart", b"", 777);
put_storage_value(
b"Staking", b"Stakers", &blake2_256(&11u64.encode()),
Exposure::<AccountId, Balance> {
total: 10,
own: 10,
others: vec![],
}
);
put_storage_value(
b"Staking", b"Stakers", &blake2_256(&21u64.encode()),
Exposure::<AccountId, Balance> {
total: 20,
own: 20,
others: vec![],
}
);
put_storage_value(
b"Staking", b"Stakers", &blake2_256(&31u64.encode()),
Exposure::<AccountId, Balance> {
total: 30,
own: 30,
others: vec![],
}
);
put_storage_value::<(u32, Vec<u32>)>(b"Staking", b"CurrentEraPointsEarned", b"", (12, vec![2, 10]));
<Staking as Store>::ErasStakers::remove_all();
<Staking as Store>::ErasStakersClipped::remove_all();
<Staking as Store>::IsUpgraded::put(false);
// Perform upgrade
Staking::ensure_storage_upgraded();
assert_eq!(<Staking as Store>::IsUpgraded::get(), true);
// Check migration
assert_eq!(<Staking as Store>::ErasStartSessionIndex::get(3).unwrap(), 5);
assert_eq!(<Staking as Store>::ErasRewardPoints::get(3), EraRewardPoints {
total: 12,
individual: vec![(21, 2), (31, 10)].into_iter().collect(),
});
assert_eq!(<Staking as Store>::ActiveEra::get().unwrap().index, 3);
assert_eq!(<Staking as Store>::ActiveEra::get().unwrap().start, Some(777));
assert_eq!(<Staking as Store>::CurrentEra::get().unwrap(), 3);
assert_eq!(<Staking as Store>::ErasStakers::get(3, 11), Exposure {
total: 0,
own: 0,
others: vec![],
});
assert_eq!(<Staking as Store>::ErasStakers::get(3, 21), Exposure {
total: 20,
own: 20,
others: vec![],
});
assert_eq!(<Staking as Store>::ErasStakers::get(3, 31), Exposure {
total: 30,
own: 30,
others: vec![],
});
assert_eq!(<Staking as Store>::ErasStakersClipped::get(3, 11), Exposure {
total: 0,
own: 0,
others: vec![],
});
assert_eq!(<Staking as Store>::ErasStakersClipped::get(3, 21), Exposure {
total: 20,
own: 20,
others: vec![],
});
assert_eq!(<Staking as Store>::ErasStakersClipped::get(3, 31), Exposure {
total: 30,
own: 30,
others: vec![],
});
assert_eq!(<Staking as Store>::ErasValidatorPrefs::get(3, 21), Staking::validators(21));
assert_eq!(<Staking as Store>::ErasValidatorPrefs::get(3, 31), Staking::validators(31));
assert_eq!(<Staking as Store>::ErasTotalStake::get(3), 50);
})
}
#[test]
fn zero_slash_keeps_nominators() {
ExtBuilder::default().build().execute_with(|| {
@@ -3071,129 +2976,6 @@ fn test_max_nominator_rewarded_per_validator_and_cant_steal_someone_else_reward(
});
}
// Test that an upgrade from previous test environment works.
#[test]
fn test_upgrade_from_master_works() {
let data_sets = &[
test_upgrade_from_master_dataset::_0,
test_upgrade_from_master_dataset::_1,
test_upgrade_from_master_dataset::_2,
test_upgrade_from_master_dataset::_3,
test_upgrade_from_master_dataset::_4,
test_upgrade_from_master_dataset::_5,
test_upgrade_from_master_dataset::_6,
test_upgrade_from_master_dataset::_7,
test_upgrade_from_master_dataset::_8,
];
for data_set in data_sets.iter() {
let mut storage = sp_runtime::Storage::default();
for (key, value) in data_set.iter() {
storage.top.insert(key.to_vec(), value.to_vec());
}
let mut ext = sp_io::TestExternalities::from(storage);
ext.execute_with(|| {
let old_stakers =
get_storage_value::<Vec<AccountId>>(b"Staking", b"CurrentElected", b"").unwrap();
let old_staker_0 = old_stakers[0];
let old_staker_1 = old_stakers[1];
let old_current_era =
get_storage_value::<EraIndex>(b"Staking", b"CurrentEra", b"").unwrap();
let old_staker_0_exposure = get_storage_value::<Exposure<AccountId, Balance>>(
b"Staking", b"Stakers", &blake2_256(&old_staker_0.encode())
).unwrap();
let old_staker_1_exposure = get_storage_value::<Exposure<AccountId, Balance>>(
b"Staking", b"Stakers", &blake2_256(&old_staker_1.encode())
).unwrap();
let (
old_era_points_earned_total,
old_era_points_earned_individual
) = get_storage_value::<(u32, Vec<u32>)>(b"Staking", b"CurrentEraPointsEarned", b"")
.unwrap_or((0, vec![]));
Staking::ensure_storage_upgraded();
assert!(<Staking as Store>::IsUpgraded::get());
// Check ActiveEra and CurrentEra
let active_era = Staking::active_era().unwrap().index;
let current_era = Staking::current_era().unwrap();
assert!(current_era == active_era);
assert!(current_era == old_current_era);
// Check ErasStartSessionIndex
let active_era_start = Staking::eras_start_session_index(active_era).unwrap();
let current_era_start = Staking::eras_start_session_index(current_era).unwrap();
let current_session_index = Session::current_index();
assert!(current_era_start == active_era_start);
assert!(active_era_start <= current_session_index);
assert_eq!(<Staking as Store>::ErasStartSessionIndex::iter().count(), 1);
// Check ErasStakers
assert_eq!(<Staking as Store>::ErasStakers::iter().count(), 2);
assert_eq!(
<Staking as Store>::ErasStakers::get(current_era, old_staker_0),
old_staker_0_exposure
);
assert_eq!(
<Staking as Store>::ErasStakers::get(current_era, old_staker_1),
old_staker_1_exposure
);
// Check ErasStakersClipped
assert_eq!(<Staking as Store>::ErasStakersClipped::iter().count(), 2);
assert!(<Staking as Store>::ErasStakersClipped::iter().all(|exposure_clipped| {
let max = <Test as Trait>::MaxNominatorRewardedPerValidator::get() as usize;
exposure_clipped.others.len() <= max
}));
assert_eq!(
<Staking as Store>::ErasStakersClipped::get(current_era, old_staker_0),
old_staker_0_exposure
);
assert_eq!(
<Staking as Store>::ErasStakersClipped::get(current_era, old_staker_1),
old_staker_1_exposure
);
// Check ErasValidatorPrefs
assert_eq!(<Staking as Store>::ErasValidatorPrefs::iter().count(), 2);
assert_eq!(
<Staking as Store>::ErasValidatorPrefs::get(current_era, old_staker_0),
Staking::validators(old_staker_0)
);
assert_eq!(
<Staking as Store>::ErasValidatorPrefs::get(current_era, old_staker_1),
Staking::validators(old_staker_1)
);
// Check ErasTotalStake
assert_eq!(<Staking as Store>::ErasTotalStake::iter().count(), 1);
assert_eq!(
<Staking as Store>::ErasTotalStake::get(current_era),
old_staker_0_exposure.total + old_staker_1_exposure.total
);
// Check ErasRewardPoints
assert_eq!(<Staking as Store>::ErasRewardPoints::iter().count(), 1);
let mut individual = BTreeMap::new();
if let Some(p) = old_era_points_earned_individual.get(0) {
individual.insert(old_staker_0, p.clone());
}
if let Some(p) = old_era_points_earned_individual.get(1) {
individual.insert(old_staker_1, p.clone());
}
assert_eq!(
<Staking as Store>::ErasRewardPoints::get(current_era),
EraRewardPoints {
total: old_era_points_earned_total,
individual,
}
);
// Check ErasValidatorReward
assert_eq!(<Staking as Store>::ErasValidatorReward::iter().count(), 0);
});
}
}
#[test]
fn set_history_depth_works() {
ExtBuilder::default().build().execute_with(|| {