Files
pezkuwi-subxt/substrate/bin/utils/chain-spec-builder/src/main.rs
T
Rakan Alhneiti 3aa4bfacfc Async keystore + Authority-Discovery async/await (#7000)
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
2020-10-08 20:56:35 +00:00

282 lines
7.6 KiB
Rust

// This file is part of Substrate.
// Copyright (C) 2019-2020 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: GPL-3.0-or-later WITH Classpath-exception-2.0
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
use std::{fs, path::{Path, PathBuf}, sync::Arc};
use ansi_term::Style;
use rand::{Rng, distributions::Alphanumeric, rngs::OsRng};
use structopt::StructOpt;
use sc_keystore::LocalKeystore;
use node_cli::chain_spec::{self, AccountId};
use sp_core::{
sr25519,
crypto::{Public, Ss58Codec},
};
use sp_keystore::{SyncCryptoStorePtr, SyncCryptoStore};
/// A utility to easily create a testnet chain spec definition with a given set
/// of authorities and endowed accounts and/or generate random accounts.
#[derive(StructOpt)]
#[structopt(rename_all = "kebab-case")]
enum ChainSpecBuilder {
/// Create a new chain spec with the given authorities, endowed and sudo
/// accounts.
New {
/// Authority key seed.
#[structopt(long, short, required = true)]
authority_seeds: Vec<String>,
/// Endowed account address (SS58 format).
#[structopt(long, short)]
endowed_accounts: Vec<String>,
/// Sudo account address (SS58 format).
#[structopt(long, short)]
sudo_account: String,
/// The path where the chain spec should be saved.
#[structopt(long, short, default_value = "./chain_spec.json")]
chain_spec_path: PathBuf,
},
/// Create a new chain spec with the given number of authorities and endowed
/// accounts. Random keys will be generated as required.
Generate {
/// The number of authorities.
#[structopt(long, short)]
authorities: usize,
/// The number of endowed accounts.
#[structopt(long, short, default_value = "0")]
endowed: usize,
/// The path where the chain spec should be saved.
#[structopt(long, short, default_value = "./chain_spec.json")]
chain_spec_path: PathBuf,
/// Path to use when saving generated keystores for each authority.
///
/// At this path, a new folder will be created for each authority's
/// keystore named `auth-$i` where `i` is the authority index, i.e.
/// `auth-0`, `auth-1`, etc.
#[structopt(long, short)]
keystore_path: Option<PathBuf>,
},
}
impl ChainSpecBuilder {
/// Returns the path where the chain spec should be saved.
fn chain_spec_path(&self) -> &Path {
match self {
ChainSpecBuilder::New { chain_spec_path, .. } =>
chain_spec_path.as_path(),
ChainSpecBuilder::Generate { chain_spec_path, .. } =>
chain_spec_path.as_path(),
}
}
}
fn genesis_constructor(
authority_seeds: &[String],
endowed_accounts: &[AccountId],
sudo_account: &AccountId,
) -> chain_spec::GenesisConfig {
let authorities = authority_seeds
.iter()
.map(AsRef::as_ref)
.map(chain_spec::authority_keys_from_seed)
.collect::<Vec<_>>();
let enable_println = true;
chain_spec::testnet_genesis(
authorities,
sudo_account.clone(),
Some(endowed_accounts.to_vec()),
enable_println,
)
}
fn generate_chain_spec(
authority_seeds: Vec<String>,
endowed_accounts: Vec<String>,
sudo_account: String,
) -> Result<String, String> {
let parse_account = |address: &String| {
AccountId::from_string(address)
.map_err(|err| format!("Failed to parse account address: {:?}", err))
};
let endowed_accounts = endowed_accounts
.iter()
.map(parse_account)
.collect::<Result<Vec<_>, String>>()?;
let sudo_account = parse_account(&sudo_account)?;
let chain_spec = chain_spec::ChainSpec::from_genesis(
"Custom",
"custom",
sc_chain_spec::ChainType::Live,
move || genesis_constructor(&authority_seeds, &endowed_accounts, &sudo_account),
vec![],
None,
None,
None,
Default::default(),
);
chain_spec.as_json(false).map_err(|err| err)
}
fn generate_authority_keys_and_store(
seeds: &[String],
keystore_path: &Path,
) -> Result<(), String> {
for (n, seed) in seeds.into_iter().enumerate() {
let keystore: SyncCryptoStorePtr = Arc::new(LocalKeystore::open(
keystore_path.join(format!("auth-{}", n)),
None,
).map_err(|err| err.to_string())?);
let (_, _, grandpa, babe, im_online, authority_discovery) =
chain_spec::authority_keys_from_seed(seed);
let insert_key = |key_type, public| {
SyncCryptoStore::insert_unknown(
&*keystore,
key_type,
&format!("//{}", seed),
public,
).map_err(|_| format!("Failed to insert key: {}", grandpa))
};
insert_key(
sp_core::crypto::key_types::BABE,
babe.as_slice(),
)?;
insert_key(
sp_core::crypto::key_types::GRANDPA,
grandpa.as_slice(),
)?;
insert_key(
sp_core::crypto::key_types::IM_ONLINE,
im_online.as_slice(),
)?;
insert_key(
sp_core::crypto::key_types::AUTHORITY_DISCOVERY,
authority_discovery.as_slice(),
)?;
}
Ok(())
}
fn print_seeds(
authority_seeds: &[String],
endowed_seeds: &[String],
sudo_seed: &str,
) {
let header = Style::new().bold().underline();
let entry = Style::new().bold();
println!("{}", header.paint("Authority seeds"));
for (n, seed) in authority_seeds.iter().enumerate() {
println!("{} //{}",
entry.paint(format!("auth-{}:", n)),
seed,
);
}
println!();
if !endowed_seeds.is_empty() {
println!("{}", header.paint("Endowed seeds"));
for (n, seed) in endowed_seeds.iter().enumerate() {
println!("{} //{}",
entry.paint(format!("endowed-{}:", n)),
seed,
);
}
println!();
}
println!("{}", header.paint("Sudo seed"));
println!("//{}", sudo_seed);
}
fn main() -> Result<(), String> {
#[cfg(build_type="debug")]
println!(
"The chain spec builder builds a chain specification that includes a Substrate runtime compiled as WASM. To \
ensure proper functioning of the included runtime compile (or run) the chain spec builder binary in \
`--release` mode.\n",
);
let builder = ChainSpecBuilder::from_args();
let chain_spec_path = builder.chain_spec_path().to_path_buf();
let (authority_seeds, endowed_accounts, sudo_account) = match builder {
ChainSpecBuilder::Generate { authorities, endowed, keystore_path, .. } => {
let authorities = authorities.max(1);
let rand_str = || -> String {
OsRng.sample_iter(&Alphanumeric)
.take(32)
.collect()
};
let authority_seeds = (0..authorities).map(|_| rand_str()).collect::<Vec<_>>();
let endowed_seeds = (0..endowed).map(|_| rand_str()).collect::<Vec<_>>();
let sudo_seed = rand_str();
print_seeds(
&authority_seeds,
&endowed_seeds,
&sudo_seed,
);
if let Some(keystore_path) = keystore_path {
generate_authority_keys_and_store(
&authority_seeds,
&keystore_path,
)?;
}
let endowed_accounts = endowed_seeds.iter().map(|seed| {
chain_spec::get_account_id_from_seed::<sr25519::Public>(seed)
.to_ss58check()
}).collect();
let sudo_account = chain_spec::get_account_id_from_seed::<sr25519::Public>(&sudo_seed)
.to_ss58check();
(authority_seeds, endowed_accounts, sudo_account)
},
ChainSpecBuilder::New { authority_seeds, endowed_accounts, sudo_account, .. } => {
(authority_seeds, endowed_accounts, sudo_account)
},
};
let json = generate_chain_spec(
authority_seeds,
endowed_accounts,
sudo_account,
)?;
fs::write(chain_spec_path, json).map_err(|err| err.to_string())
}