pezkuwichain/pezkuwi-subxt
1
0
Fork 0
mirror of https://github.com/pezkuwichain/pezkuwi-subxt.git synced 2026-06-09 21:21:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

Async keystore + Authority-Discovery async/await (#7000)

Browse Source 
* Asyncify sign_with

* Asyncify generate/get keys

* Complete BareCryptoStore asyncification

* Cleanup

* Rebase

* Add Proxy

* Inject keystore proxy into extensions

* Implement some methods

* Await on send

* Cleanup

* Send result over the oneshot channel sender

* Process one future at a time

* Fix cargo stuff

* Asyncify sr25519_vrf_sign

* Cherry-pick and fix changes

* Introduce SyncCryptoStore

* SQUASH ME WITH THE first commit

* Implement into SyncCryptoStore

* Implement BareCryptoStore for KeystoreProxyAdapter

* authority-discovery

* AURA

* BABE

* finality-grandpa

* offchain-workers

* benchmarking-cli

* sp_io

* test-utils

* application-crypto

* Extensions and RPC

* Client Service

* bin

* Update cargo.lock

* Implement BareCryptoStore on proxy directly

* Simplify proxy setup

* Fix authority-discover

* Pass async keystore to authority-discovery

* Fix tests

* Use async keystore in authority-discovery

* Rename BareCryptoStore to CryptoStore

* WIP

* Remote mutable borrow in CryptoStore trait

* Implement Keystore with backends

* Remove Proxy implementation

* Fix service builder and keystore user-crates

* Fix tests

* Rework authority-discovery after refactoring

* futures::select!

* Fix multiple mut borrows in authority-discovery

* Merge fixes

* Require sync

* Restore Cargo.lock

* PR feedback - round 1

* Remove Keystore and use LocalKeystore directly

Also renamed KeystoreParams to KeystoreContainer

* Join

* Remove sync requirement

* Fix keystore tests

* Fix tests

* client/authority-discovery: Remove event stream dynamic dispatching

With authority-discovery moving from a poll based future to an `async`
future Rust has difficulties propagating the `Sync` trade through the
generated state machine.

Instead of using dynamic dispatching, use a trait parameter to specify
the DHT event stream.

* Make it compile

* Fix submit_transaction

* Fix block_on issue

* Use await in async context

* Fix manual seal keystore

* Fix authoring_blocks test

* fix aura authoring_blocks

* Try to fix tests for auth-discovery

* client/authority-discovery: Fix lookup_throttling test

* client/authority-discovery: Fix triggers_dht_get_query test

* Fix epoch_authorship_works

* client/authority-discovery: Remove timing assumption in unit test

* client/authority-discovery: Revert changes to termination test

* PR feedback

* Remove deadcode and mark test code

* Fix test_sync

* Use the correct keyring type

* Return when from_service stream is closed

* Convert SyncCryptoStore to a trait

* Fix line width

* Fix line width - take 2

* Remove unused import

* Fix keystore instantiation

* PR feedback

* Remove KeystoreContainer

* Revert "Remove KeystoreContainer"

This reverts commit ea4a37c7d74f9772b93d974e05e4498af6192730.

* Take a ref of keystore

* Move keystore to dev-dependencies

* Address some PR feedback

* Missed one

* Pass keystore reference - take 2

* client/finality-grandpa: Use `Arc<dyn CryptoStore>` instead of SyncXXX

Instead of using `SyncCryptoStorePtr` within `client/finality-grandpa`,
which is a type alias for `Arc<dyn SyncCryptoStore>`, use `Arc<dyn
CryptoStore>`. Benefits are:

1. No additional mental overhead of a `SyncCryptoStorePtr`.

2. Ability for new code to use the asynchronous methods of `CryptoStore`
instead of the synchronous `SyncCryptoStore` methods within
`client/finality-granpa` without the need for larger refactorings.

Note: This commit uses `Arc<dyn CryptoStore>` instead of
`CryptoStorePtr`, as I find the type signature more descriptive. This is
subjective and in no way required.

* Remove SyncCryptoStorePtr

* Remove KeystoreContainer & SyncCryptoStorePtr

* PR feedback

* *: Use CryptoStorePtr whereever possible

* *: Define SyncCryptoStore as a pure extension trait of CryptoStore

* Follow up to SyncCryptoStore extension trait

* Adjust docs for SyncCryptoStore as Ben suggested

* Cleanup unnecessary requirements

* sp-keystore

* Use async_std::task::block_on in keystore

* Fix block_on std requirement

* Update primitives/keystore/src/lib.rs

Co-authored-by: Max Inden <mail@max-inden.de>

* Fix wasm build

* Remove unused var

* Fix wasm compilation - take 2

* Revert async-std in keystore

* Fix indent

* Fix version and copyright

* Cleanup feature = "std"

* Auth Discovery: Ignore if from_service is cloed

* Max's suggestion

* Revert async-std usage for block_on

* Address PR feedback

* Fix example offchain worker build

* Address PR feedback

* Update Cargo.lock

* Move unused methods to test helper functions

* Restore accidentally deleted cargo.lock files

* Fix unused imports

Co-authored-by: Max Inden <mail@max-inden.de>
Co-authored-by: Shawn Tabrizi <shawntabrizi@gmail.com>
This commit is contained in:
Rakan Alhneiti
2020-10-08 22:56:35 +02:00
committed by GitHub
parent db8a0cafa9
commit 3aa4bfacfc
70 changed files with 2394 additions and 1762 deletions
Download Patch File Download Diff File Expand all files Collapse all files
substrate/client/keystore/src/lib.rs
+6 -516
View File
@@ -17,19 +17,13 @@
//! Keystore (and session key management) for ed25519 based chains like Polkadot.
#![warn(missing_docs)]
use std::{collections::{HashMap, HashSet}, path::PathBuf, fs::{self, File}, io::{self, Write}, sync::Arc};
use sp_core::{
crypto::{IsWrappedBy, CryptoTypePublicPair, KeyTypeId, Pair as PairT, ExposeSecret, SecretString, Public},
traits::{BareCryptoStore, Error as TraitError},
sr25519::{Public as Sr25519Public, Pair as Sr25519Pair},
vrf::{VRFTranscriptData, VRFSignature, make_transcript},
Encode,
};
use sp_application_crypto::{AppKey, AppPublic, AppPair, ed25519, sr25519, ecdsa};
use parking_lot::RwLock;
use std::io;
use sp_core::crypto::KeyTypeId;
use sp_keystore::Error as TraitError;
/// Keystore pointer
pub type KeyStorePtr = Arc<RwLock<Store>>;
/// Local keystore implementation
mod local;
pub use local::LocalKeystore;
/// Keystore error.
#[derive(Debug, derive_more::Display, derive_more::From)]
@@ -86,507 +80,3 @@ impl std::error::Error for Error {
}
}
/// Key store.
///
/// Stores key pairs in a file system store + short lived key pairs in memory.
///
/// Every pair that is being generated by a `seed`, will be placed in memory.
pub struct Store {
path: Option<PathBuf>,
/// Map over `(KeyTypeId, Raw public key)` -> `Key phrase/seed`
additional: HashMap<(KeyTypeId, Vec<u8>), String>,
password: Option<SecretString>,
}
impl Store {
/// Open the store at the given path.
///
/// Optionally takes a password that will be used to encrypt/decrypt the keys.
pub fn open<T: Into<PathBuf>>(path: T, password: Option<SecretString>) -> Result<KeyStorePtr> {
let path = path.into();
fs::create_dir_all(&path)?;
let instance = Self { path: Some(path), additional: HashMap::new(), password };
Ok(Arc::new(RwLock::new(instance)))
}
/// Create a new in-memory store.
pub fn new_in_memory() -> KeyStorePtr {
Arc::new(RwLock::new(Self {
path: None,
additional: HashMap::new(),
password: None
}))
}
/// Get the key phrase for the given public key and key type from the in-memory store.
fn get_additional_pair(
&self,
public: &[u8],
key_type: KeyTypeId,
) -> Option<&String> {
let key = (key_type, public.to_vec());
self.additional.get(&key)
}
/// Insert the given public/private key pair with the given key type.
///
/// Does not place it into the file system store.
fn insert_ephemeral_pair<Pair: PairT>(&mut self, pair: &Pair, seed: &str, key_type: KeyTypeId) {
let key = (key_type, pair.public().to_raw_vec());
self.additional.insert(key, seed.into());
}
/// Insert a new key with anonymous crypto.
///
/// Places it into the file system store.
fn insert_unknown(&self, key_type: KeyTypeId, suri: &str, public: &[u8]) -> Result<()> {
if let Some(path) = self.key_file_path(public, key_type) {
let mut file = File::create(path).map_err(Error::Io)?;
serde_json::to_writer(&file, &suri).map_err(Error::Json)?;
file.flush().map_err(Error::Io)?;
}
Ok(())
}
/// Insert a new key.
///
/// Places it into the file system store.
pub fn insert_by_type<Pair: PairT>(&self, key_type: KeyTypeId, suri: &str) -> Result<Pair> {
let pair = Pair::from_string(
suri,
self.password()
).map_err(|_| Error::InvalidSeed)?;
self.insert_unknown(key_type, suri, pair.public().as_slice())
.map_err(|_| Error::Unavailable)?;
Ok(pair)
}
/// Insert a new key.
///
/// Places it into the file system store.
pub fn insert<Pair: AppPair>(&self, suri: &str) -> Result<Pair> {
self.insert_by_type::<Pair::Generic>(Pair::ID, suri).map(Into::into)
}
/// Generate a new key.
///
/// Places it into the file system store.
pub fn generate_by_type<Pair: PairT>(&self, key_type: KeyTypeId) -> Result<Pair> {
let (pair, phrase, _) = Pair::generate_with_phrase(self.password());
if let Some(path) = self.key_file_path(pair.public().as_slice(), key_type) {
let mut file = File::create(path)?;
serde_json::to_writer(&file, &phrase)?;
file.flush()?;
}
Ok(pair)
}
/// Generate a new key.
///
/// Places it into the file system store.
pub fn generate<Pair: AppPair>(&self) -> Result<Pair> {
self.generate_by_type::<Pair::Generic>(Pair::ID).map(Into::into)
}
/// Create a new key from seed.
///
/// Does not place it into the file system store.
pub fn insert_ephemeral_from_seed_by_type<Pair: PairT>(
&mut self,
seed: &str,
key_type: KeyTypeId,
) -> Result<Pair> {
let pair = Pair::from_string(seed, None).map_err(|_| Error::InvalidSeed)?;
self.insert_ephemeral_pair(&pair, seed, key_type);
Ok(pair)
}
/// Create a new key from seed.
///
/// Does not place it into the file system store.
pub fn insert_ephemeral_from_seed<Pair: AppPair>(&mut self, seed: &str) -> Result<Pair> {
self.insert_ephemeral_from_seed_by_type::<Pair::Generic>(seed, Pair::ID).map(Into::into)
}
/// Get the key phrase for a given public key and key type.
fn key_phrase_by_type(&self, public: &[u8], key_type: KeyTypeId) -> Result<String> {
if let Some(phrase) = self.get_additional_pair(public, key_type) {
return Ok(phrase.clone())
}
let path = self.key_file_path(public, key_type).ok_or_else(|| Error::Unavailable)?;
let file = File::open(path)?;
serde_json::from_reader(&file).map_err(Into::into)
}
/// Get a key pair for the given public key and key type.
pub fn key_pair_by_type<Pair: PairT>(&self,
public: &Pair::Public,
key_type: KeyTypeId,
) -> Result<Pair> {
let phrase = self.key_phrase_by_type(public.as_slice(), key_type)?;
let pair = Pair::from_string(
&phrase,
self.password(),
).map_err(|_| Error::InvalidPhrase)?;
if &pair.public() == public {
Ok(pair)
} else {
Err(Error::InvalidPassword)
}
}
/// Get a key pair for the given public key.
pub fn key_pair<Pair: AppPair>(&self, public: &<Pair as AppKey>::Public) -> Result<Pair> {
self.key_pair_by_type::<Pair::Generic>(IsWrappedBy::from_ref(public), Pair::ID).map(Into::into)
}
/// Get public keys of all stored keys that match the key type.
///
/// This will just use the type of the public key (a list of which to be returned) in order
/// to determine the key type. Unless you use a specialized application-type public key, then
/// this only give you keys registered under generic cryptography, and will not return keys
/// registered under the application type.
pub fn public_keys<Public: AppPublic>(&self) -> Result<Vec<Public>> {
self.raw_public_keys(Public::ID)
.map(|v| {
v.into_iter()
.map(|k| Public::from_slice(k.as_slice()))
.collect()
})
}
/// Returns the file path for the given public key and key type.
fn key_file_path(&self, public: &[u8], key_type: KeyTypeId) -> Option<PathBuf> {
let mut buf = self.path.as_ref()?.clone();
let key_type = hex::encode(key_type.0);
let key = hex::encode(public);
buf.push(key_type + key.as_str());
Some(buf)
}
/// Returns a list of raw public keys filtered by `KeyTypeId`
fn raw_public_keys(&self, id: KeyTypeId) -> Result<Vec<Vec<u8>>> {
let mut public_keys: Vec<Vec<u8>> = self.additional.keys()
.into_iter()
.filter_map(|k| if k.0 == id { Some(k.1.clone()) } else { None })
.collect();
if let Some(path) = &self.path {
for entry in fs::read_dir(&path)? {
let entry = entry?;
let path = entry.path();
// skip directories and non-unicode file names (hex is unicode)
if let Some(name) = path.file_name().and_then(|n| n.to_str()) {
match hex::decode(name) {
Ok(ref hex) if hex.len() > 4 => {
if &hex[0..4] != &id.0 {
continue;
}
let public = hex[4..].to_vec();
public_keys.push(public);
}
_ => continue,
}
}
}
}
Ok(public_keys)
}
}
impl BareCryptoStore for Store {
fn keys(
&self,
id: KeyTypeId
) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
let raw_keys = self.raw_public_keys(id)?;
Ok(raw_keys.into_iter()
.fold(Vec::new(), |mut v, k| {
v.push(CryptoTypePublicPair(sr25519::CRYPTO_ID, k.clone()));
v.push(CryptoTypePublicPair(ed25519::CRYPTO_ID, k.clone()));
v.push(CryptoTypePublicPair(ecdsa::CRYPTO_ID, k));
v
}))
}
fn supported_keys(
&self,
id: KeyTypeId,
keys: Vec<CryptoTypePublicPair>
) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
let all_keys = self.keys(id)?.into_iter().collect::<HashSet<_>>();
Ok(keys.into_iter()
.filter(|key| all_keys.contains(key))
.collect::<Vec<_>>())
}
fn sign_with(
&self,
id: KeyTypeId,
key: &CryptoTypePublicPair,
msg: &[u8],
) -> std::result::Result<Vec<u8>, TraitError> {
match key.0 {
ed25519::CRYPTO_ID => {
let pub_key = ed25519::Public::from_slice(key.1.as_slice());
let key_pair: ed25519::Pair = self
.key_pair_by_type::<ed25519::Pair>(&pub_key, id)
.map_err(|e| TraitError::from(e))?;
Ok(key_pair.sign(msg).encode())
}
sr25519::CRYPTO_ID => {
let pub_key = sr25519::Public::from_slice(key.1.as_slice());
let key_pair: sr25519::Pair = self
.key_pair_by_type::<sr25519::Pair>(&pub_key, id)
.map_err(|e| TraitError::from(e))?;
Ok(key_pair.sign(msg).encode())
},
ecdsa::CRYPTO_ID => {
let pub_key = ecdsa::Public::from_slice(key.1.as_slice());
let key_pair: ecdsa::Pair = self
.key_pair_by_type::<ecdsa::Pair>(&pub_key, id)
.map_err(|e| TraitError::from(e))?;
Ok(key_pair.sign(msg).encode())
}
_ => Err(TraitError::KeyNotSupported(id))
}
}
fn sr25519_public_keys(&self, key_type: KeyTypeId) -> Vec<sr25519::Public> {
self.raw_public_keys(key_type)
.map(|v| {
v.into_iter()
.map(|k| sr25519::Public::from_slice(k.as_slice()))
.collect()
})
.unwrap_or_default()
}
fn sr25519_generate_new(
&mut self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<sr25519::Public, TraitError> {
let pair = match seed {
Some(seed) => self.insert_ephemeral_from_seed_by_type::<sr25519::Pair>(seed, id),
None => self.generate_by_type::<sr25519::Pair>(id),
}.map_err(|e| -> TraitError { e.into() })?;
Ok(pair.public())
}
fn ed25519_public_keys(&self, key_type: KeyTypeId) -> Vec<ed25519::Public> {
self.raw_public_keys(key_type)
.map(|v| {
v.into_iter()
.map(|k| ed25519::Public::from_slice(k.as_slice()))
.collect()
})
.unwrap_or_default()
}
fn ed25519_generate_new(
&mut self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<ed25519::Public, TraitError> {
let pair = match seed {
Some(seed) => self.insert_ephemeral_from_seed_by_type::<ed25519::Pair>(seed, id),
None => self.generate_by_type::<ed25519::Pair>(id),
}.map_err(|e| -> TraitError { e.into() })?;
Ok(pair.public())
}
fn ecdsa_public_keys(&self, key_type: KeyTypeId) -> Vec<ecdsa::Public> {
self.raw_public_keys(key_type)
.map(|v| {
v.into_iter()
.map(|k| ecdsa::Public::from_slice(k.as_slice()))
.collect()
})
.unwrap_or_default()
}
fn ecdsa_generate_new(
&mut self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<ecdsa::Public, TraitError> {
let pair = match seed {
Some(seed) => self.insert_ephemeral_from_seed_by_type::<ecdsa::Pair>(seed, id),
None => self.generate_by_type::<ecdsa::Pair>(id),
}.map_err(|e| -> TraitError { e.into() })?;
Ok(pair.public())
}
fn insert_unknown(&mut self, key_type: KeyTypeId, suri: &str, public: &[u8])
-> std::result::Result<(), ()>
{
Store::insert_unknown(self, key_type, suri, public).map_err(|_| ())
}
fn password(&self) -> Option<&str> {
self.password.as_ref()
.map(|p| p.expose_secret())
.map(|p| p.as_str())
}
fn has_keys(&self, public_keys: &[(Vec<u8>, KeyTypeId)]) -> bool {
public_keys.iter().all(|(p, t)| self.key_phrase_by_type(&p, *t).is_ok())
}
fn sr25519_vrf_sign(
&self,
key_type: KeyTypeId,
public: &Sr25519Public,
transcript_data: VRFTranscriptData,
) -> std::result::Result<VRFSignature, TraitError> {
let transcript = make_transcript(transcript_data);
let pair = self.key_pair_by_type::<Sr25519Pair>(public, key_type)
.map_err(|e| TraitError::PairNotFound(e.to_string()))?;
let (inout, proof, _) = pair.as_ref().vrf_sign(transcript);
Ok(VRFSignature {
output: inout.to_output(),
proof,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
use sp_core::{testing::SR25519, crypto::Ss58Codec};
use std::str::FromStr;
#[test]
fn basic_store() {
let temp_dir = TempDir::new().unwrap();
let store = Store::open(temp_dir.path(), None).unwrap();
assert!(store.read().public_keys::<ed25519::AppPublic>().unwrap().is_empty());
let key: ed25519::AppPair = store.write().generate().unwrap();
let key2: ed25519::AppPair = store.read().key_pair(&key.public()).unwrap();
assert_eq!(key.public(), key2.public());
assert_eq!(store.read().public_keys::<ed25519::AppPublic>().unwrap()[0], key.public());
}
#[test]
fn test_insert_ephemeral_from_seed() {
let temp_dir = TempDir::new().unwrap();
let store = Store::open(temp_dir.path(), None).unwrap();
let pair: ed25519::AppPair = store
.write()
.insert_ephemeral_from_seed("0x3d97c819d68f9bafa7d6e79cb991eebcd77d966c5334c0b94d9e1fa7ad0869dc")
.unwrap();
assert_eq!(
"5DKUrgFqCPV8iAXx9sjy1nyBygQCeiUYRFWurZGhnrn3HJCA",
pair.public().to_ss58check()
);
drop(store);
let store = Store::open(temp_dir.path(), None).unwrap();
// Keys generated from seed should not be persisted!
assert!(store.read().key_pair::<ed25519::AppPair>(&pair.public()).is_err());
}
#[test]
fn password_being_used() {
let password = String::from("password");
let temp_dir = TempDir::new().unwrap();
let store = Store::open(
temp_dir.path(),
Some(FromStr::from_str(password.as_str()).unwrap()),
).unwrap();
let pair: ed25519::AppPair = store.write().generate().unwrap();
assert_eq!(
pair.public(),
store.read().key_pair::<ed25519::AppPair>(&pair.public()).unwrap().public(),
);
// Without the password the key should not be retrievable
let store = Store::open(temp_dir.path(), None).unwrap();
assert!(store.read().key_pair::<ed25519::AppPair>(&pair.public()).is_err());
let store = Store::open(
temp_dir.path(),
Some(FromStr::from_str(password.as_str()).unwrap()),
).unwrap();
assert_eq!(
pair.public(),
store.read().key_pair::<ed25519::AppPair>(&pair.public()).unwrap().public(),
);
}
#[test]
fn public_keys_are_returned() {
let temp_dir = TempDir::new().unwrap();
let store = Store::open(temp_dir.path(), None).unwrap();
let mut public_keys = Vec::new();
for i in 0..10 {
public_keys.push(store.write().generate::<ed25519::AppPair>().unwrap().public());
public_keys.push(store.write().insert_ephemeral_from_seed::<ed25519::AppPair>(
&format!("0x3d97c819d68f9bafa7d6e79cb991eebcd7{}d966c5334c0b94d9e1fa7ad0869dc", i),
).unwrap().public());
}
// Generate a key of a different type
store.write().generate::<sr25519::AppPair>().unwrap();
public_keys.sort();
let mut store_pubs = store.read().public_keys::<ed25519::AppPublic>().unwrap();
store_pubs.sort();
assert_eq!(public_keys, store_pubs);
}
#[test]
fn store_unknown_and_extract_it() {
let temp_dir = TempDir::new().unwrap();
let store = Store::open(temp_dir.path(), None).unwrap();
let secret_uri = "//Alice";
let key_pair = sr25519::AppPair::from_string(secret_uri, None).expect("Generates key pair");
store.write().insert_unknown(
SR25519,
secret_uri,
key_pair.public().as_ref(),
).expect("Inserts unknown key");
let store_key_pair = store.read().key_pair_by_type::<sr25519::AppPair>(
&key_pair.public(),
SR25519,
).expect("Gets key pair from keystore");
assert_eq!(key_pair.public(), store_key_pair.public());
}
#[test]
fn store_ignores_files_with_invalid_name() {
let temp_dir = TempDir::new().unwrap();
let store = Store::open(temp_dir.path(), None).unwrap();
let file_name = temp_dir.path().join(hex::encode(&SR25519.0[..2]));
fs::write(file_name, "test").expect("Invalid file is written");
assert!(
store.read().sr25519_public_keys(SR25519).is_empty(),
);
}
}
substrate/client/keystore/src/local.rs
+647
View File
@@ -0,0 +1,647 @@
// This file is part of Substrate.
// Copyright (C) 2019-2020 Parity Technologies (UK) Ltd.
// SPDX-License-Identifier: Apache-2.0
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//
//! Local keystore implementation
use std::{
collections::{HashMap, HashSet},
fs::{self, File},
io::Write,
path::PathBuf,
sync::Arc,
};
use async_trait::async_trait;
use parking_lot::RwLock;
use sp_core::{
crypto::{CryptoTypePublicPair, KeyTypeId, Pair as PairT, ExposeSecret, SecretString, Public},
sr25519::{Public as Sr25519Public, Pair as Sr25519Pair},
Encode,
};
use sp_keystore::{
CryptoStore,
SyncCryptoStorePtr,
Error as TraitError,
SyncCryptoStore,
vrf::{VRFTranscriptData, VRFSignature, make_transcript},
};
use sp_application_crypto::{ed25519, sr25519, ecdsa};
use crate::{Result, Error};
/// A local based keystore that is either memory-based or filesystem-based.
pub struct LocalKeystore(RwLock<KeystoreInner>);
impl LocalKeystore {
/// Create a local keystore from filesystem.
pub fn open<T: Into<PathBuf>>(path: T, password: Option<SecretString>) -> Result<Self> {
let inner = KeystoreInner::open(path, password)?;
Ok(Self(RwLock::new(inner)))
}
/// Create a local keystore in memory.
pub fn in_memory() -> Self {
let inner = KeystoreInner::new_in_memory();
Self(RwLock::new(inner))
}
}
#[async_trait]
impl CryptoStore for LocalKeystore {
async fn keys(&self, id: KeyTypeId) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
SyncCryptoStore::keys(self, id)
}
async fn sr25519_public_keys(&self, id: KeyTypeId) -> Vec<sr25519::Public> {
SyncCryptoStore::sr25519_public_keys(self, id)
}
async fn sr25519_generate_new(
&self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<sr25519::Public, TraitError> {
SyncCryptoStore::sr25519_generate_new(self, id, seed)
}
async fn ed25519_public_keys(&self, id: KeyTypeId) -> Vec<ed25519::Public> {
SyncCryptoStore::ed25519_public_keys(self, id)
}
async fn ed25519_generate_new(
&self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<ed25519::Public, TraitError> {
SyncCryptoStore::ed25519_generate_new(self, id, seed)
}
async fn ecdsa_public_keys(&self, id: KeyTypeId) -> Vec<ecdsa::Public> {
SyncCryptoStore::ecdsa_public_keys(self, id)
}
async fn ecdsa_generate_new(
&self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<ecdsa::Public, TraitError> {
SyncCryptoStore::ecdsa_generate_new(self, id, seed)
}
async fn insert_unknown(&self, id: KeyTypeId, suri: &str, public: &[u8]) -> std::result::Result<(), ()> {
SyncCryptoStore::insert_unknown(self, id, suri, public)
}
async fn has_keys(&self, public_keys: &[(Vec<u8>, KeyTypeId)]) -> bool {
SyncCryptoStore::has_keys(self, public_keys)
}
async fn supported_keys(
&self,
id: KeyTypeId,
keys: Vec<CryptoTypePublicPair>,
) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
SyncCryptoStore::supported_keys(self, id, keys)
}
async fn sign_with(
&self,
id: KeyTypeId,
key: &CryptoTypePublicPair,
msg: &[u8],
) -> std::result::Result<Vec<u8>, TraitError> {
SyncCryptoStore::sign_with(self, id, key, msg)
}
async fn sr25519_vrf_sign(
&self,
key_type: KeyTypeId,
public: &sr25519::Public,
transcript_data: VRFTranscriptData,
) -> std::result::Result<VRFSignature, TraitError> {
SyncCryptoStore::sr25519_vrf_sign(self, key_type, public, transcript_data)
}
}
impl SyncCryptoStore for LocalKeystore {
fn keys(
&self,
id: KeyTypeId
) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
let raw_keys = self.0.read().raw_public_keys(id)?;
Ok(raw_keys.into_iter()
.fold(Vec::new(), |mut v, k| {
v.push(CryptoTypePublicPair(sr25519::CRYPTO_ID, k.clone()));
v.push(CryptoTypePublicPair(ed25519::CRYPTO_ID, k.clone()));
v.push(CryptoTypePublicPair(ecdsa::CRYPTO_ID, k));
v
}))
}
fn supported_keys(
&self,
id: KeyTypeId,
keys: Vec<CryptoTypePublicPair>
) -> std::result::Result<Vec<CryptoTypePublicPair>, TraitError> {
let all_keys = SyncCryptoStore::keys(self, id)?
.into_iter()
.collect::<HashSet<_>>();
Ok(keys.into_iter()
.filter(|key| all_keys.contains(key))
.collect::<Vec<_>>())
}
fn sign_with(
&self,
id: KeyTypeId,
key: &CryptoTypePublicPair,
msg: &[u8],
) -> std::result::Result<Vec<u8>, TraitError> {
match key.0 {
ed25519::CRYPTO_ID => {
let pub_key = ed25519::Public::from_slice(key.1.as_slice());
let key_pair: ed25519::Pair = self.0.read()
.key_pair_by_type::<ed25519::Pair>(&pub_key, id)
.map_err(|e| TraitError::from(e))?;
Ok(key_pair.sign(msg).encode())
}
sr25519::CRYPTO_ID => {
let pub_key = sr25519::Public::from_slice(key.1.as_slice());
let key_pair: sr25519::Pair = self.0.read()
.key_pair_by_type::<sr25519::Pair>(&pub_key, id)
.map_err(|e| TraitError::from(e))?;
Ok(key_pair.sign(msg).encode())
},
ecdsa::CRYPTO_ID => {
let pub_key = ecdsa::Public::from_slice(key.1.as_slice());
let key_pair: ecdsa::Pair = self.0.read()
.key_pair_by_type::<ecdsa::Pair>(&pub_key, id)
.map_err(|e| TraitError::from(e))?;
Ok(key_pair.sign(msg).encode())
}
_ => Err(TraitError::KeyNotSupported(id))
}
}
fn sr25519_public_keys(&self, key_type: KeyTypeId) -> Vec<sr25519::Public> {
self.0.read().raw_public_keys(key_type)
.map(|v| {
v.into_iter()
.map(|k| sr25519::Public::from_slice(k.as_slice()))
.collect()
})
.unwrap_or_default()
}
fn sr25519_generate_new(
&self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<sr25519::Public, TraitError> {
let pair = match seed {
Some(seed) => self.0.write().insert_ephemeral_from_seed_by_type::<sr25519::Pair>(seed, id),
None => self.0.write().generate_by_type::<sr25519::Pair>(id),
}.map_err(|e| -> TraitError { e.into() })?;
Ok(pair.public())
}
fn ed25519_public_keys(&self, key_type: KeyTypeId) -> Vec<ed25519::Public> {
self.0.read().raw_public_keys(key_type)
.map(|v| {
v.into_iter()
.map(|k| ed25519::Public::from_slice(k.as_slice()))
.collect()
})
.unwrap_or_default()
}
fn ed25519_generate_new(
&self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<ed25519::Public, TraitError> {
let pair = match seed {
Some(seed) => self.0.write().insert_ephemeral_from_seed_by_type::<ed25519::Pair>(seed, id),
None => self.0.write().generate_by_type::<ed25519::Pair>(id),
}.map_err(|e| -> TraitError { e.into() })?;
Ok(pair.public())
}
fn ecdsa_public_keys(&self, key_type: KeyTypeId) -> Vec<ecdsa::Public> {
self.0.read().raw_public_keys(key_type)
.map(|v| {
v.into_iter()
.map(|k| ecdsa::Public::from_slice(k.as_slice()))
.collect()
})
.unwrap_or_default()
}
fn ecdsa_generate_new(
&self,
id: KeyTypeId,
seed: Option<&str>,
) -> std::result::Result<ecdsa::Public, TraitError> {
let pair = match seed {
Some(seed) => self.0.write().insert_ephemeral_from_seed_by_type::<ecdsa::Pair>(seed, id),
None => self.0.write().generate_by_type::<ecdsa::Pair>(id),
}.map_err(|e| -> TraitError { e.into() })?;
Ok(pair.public())
}
fn insert_unknown(&self, key_type: KeyTypeId, suri: &str, public: &[u8])
-> std::result::Result<(), ()>
{
self.0.write().insert_unknown(key_type, suri, public).map_err(|_| ())
}
fn has_keys(&self, public_keys: &[(Vec<u8>, KeyTypeId)]) -> bool {
public_keys.iter().all(|(p, t)| self.0.read().key_phrase_by_type(&p, *t).is_ok())
}
fn sr25519_vrf_sign(
&self,
key_type: KeyTypeId,
public: &Sr25519Public,
transcript_data: VRFTranscriptData,
) -> std::result::Result<VRFSignature, TraitError> {
let transcript = make_transcript(transcript_data);
let pair = self.0.read().key_pair_by_type::<Sr25519Pair>(public, key_type)
.map_err(|e| TraitError::PairNotFound(e.to_string()))?;
let (inout, proof, _) = pair.as_ref().vrf_sign(transcript);
Ok(VRFSignature {
output: inout.to_output(),
proof,
})
}
}
impl Into<SyncCryptoStorePtr> for LocalKeystore {
fn into(self) -> SyncCryptoStorePtr {
Arc::new(self)
}
}
impl Into<Arc<dyn CryptoStore>> for LocalKeystore {
fn into(self) -> Arc<dyn CryptoStore> {
Arc::new(self)
}
}
/// A local key store.
///
/// Stores key pairs in a file system store + short lived key pairs in memory.
///
/// Every pair that is being generated by a `seed`, will be placed in memory.
struct KeystoreInner {
path: Option<PathBuf>,
/// Map over `(KeyTypeId, Raw public key)` -> `Key phrase/seed`
additional: HashMap<(KeyTypeId, Vec<u8>), String>,
password: Option<SecretString>,
}
impl KeystoreInner {
/// Open the store at the given path.
///
/// Optionally takes a password that will be used to encrypt/decrypt the keys.
pub fn open<T: Into<PathBuf>>(path: T, password: Option<SecretString>) -> Result<Self> {
let path = path.into();
fs::create_dir_all(&path)?;
let instance = Self { path: Some(path), additional: HashMap::new(), password };
Ok(instance)
}
/// Get the password for this store.
fn password(&self) -> Option<&str> {
self.password.as_ref()
.map(|p| p.expose_secret())
.map(|p| p.as_str())
}
/// Create a new in-memory store.
pub fn new_in_memory() -> Self {
Self {
path: None,
additional: HashMap::new(),
password: None
}
}
/// Get the key phrase for the given public key and key type from the in-memory store.
fn get_additional_pair(
&self,
public: &[u8],
key_type: KeyTypeId,
) -> Option<&String> {
let key = (key_type, public.to_vec());
self.additional.get(&key)
}
/// Insert the given public/private key pair with the given key type.
///
/// Does not place it into the file system store.
fn insert_ephemeral_pair<Pair: PairT>(&mut self, pair: &Pair, seed: &str, key_type: KeyTypeId) {
let key = (key_type, pair.public().to_raw_vec());
self.additional.insert(key, seed.into());
}
/// Insert a new key with anonymous crypto.
///
/// Places it into the file system store.
pub fn insert_unknown(&self, key_type: KeyTypeId, suri: &str, public: &[u8]) -> Result<()> {
if let Some(path) = self.key_file_path(public, key_type) {
let mut file = File::create(path).map_err(Error::Io)?;
serde_json::to_writer(&file, &suri).map_err(Error::Json)?;
file.flush().map_err(Error::Io)?;
}
Ok(())
}
/// Generate a new key.
///
/// Places it into the file system store.
pub fn generate_by_type<Pair: PairT>(&self, key_type: KeyTypeId) -> Result<Pair> {
let (pair, phrase, _) = Pair::generate_with_phrase(self.password());
if let Some(path) = self.key_file_path(pair.public().as_slice(), key_type) {
let mut file = File::create(path)?;
serde_json::to_writer(&file, &phrase)?;
file.flush()?;
}
Ok(pair)
}
/// Create a new key from seed.
///
/// Does not place it into the file system store.
pub fn insert_ephemeral_from_seed_by_type<Pair: PairT>(
&mut self,
seed: &str,
key_type: KeyTypeId,
) -> Result<Pair> {
let pair = Pair::from_string(seed, None).map_err(|_| Error::InvalidSeed)?;
self.insert_ephemeral_pair(&pair, seed, key_type);
Ok(pair)
}
/// Get the key phrase for a given public key and key type.
fn key_phrase_by_type(&self, public: &[u8], key_type: KeyTypeId) -> Result<String> {
if let Some(phrase) = self.get_additional_pair(public, key_type) {
return Ok(phrase.clone())
}
let path = self.key_file_path(public, key_type).ok_or_else(|| Error::Unavailable)?;
let file = File::open(path)?;
serde_json::from_reader(&file).map_err(Into::into)
}
/// Get a key pair for the given public key and key type.
pub fn key_pair_by_type<Pair: PairT>(&self,
public: &Pair::Public,
key_type: KeyTypeId,
) -> Result<Pair> {
let phrase = self.key_phrase_by_type(public.as_slice(), key_type)?;
let pair = Pair::from_string(
&phrase,
self.password(),
).map_err(|_| Error::InvalidPhrase)?;
if &pair.public() == public {
Ok(pair)
} else {
Err(Error::InvalidPassword)
}
}
/// Returns the file path for the given public key and key type.
fn key_file_path(&self, public: &[u8], key_type: KeyTypeId) -> Option<PathBuf> {
let mut buf = self.path.as_ref()?.clone();
let key_type = hex::encode(key_type.0);
let key = hex::encode(public);
buf.push(key_type + key.as_str());
Some(buf)
}
/// Returns a list of raw public keys filtered by `KeyTypeId`
fn raw_public_keys(&self, id: KeyTypeId) -> Result<Vec<Vec<u8>>> {
let mut public_keys: Vec<Vec<u8>> = self.additional.keys()
.into_iter()
.filter_map(|k| if k.0 == id { Some(k.1.clone()) } else { None })
.collect();
if let Some(path) = &self.path {
for entry in fs::read_dir(&path)? {
let entry = entry?;
let path = entry.path();
// skip directories and non-unicode file names (hex is unicode)
if let Some(name) = path.file_name().and_then(|n| n.to_str()) {
match hex::decode(name) {
Ok(ref hex) if hex.len() > 4 => {
if &hex[0..4] != &id.0 {
continue;
}
let public = hex[4..].to_vec();
public_keys.push(public);
}
_ => continue,
}
}
}
}
Ok(public_keys)
}
}
#[cfg(test)]
mod tests {
use super::*;
use tempfile::TempDir;
use sp_core::{
Pair,
crypto::{IsWrappedBy, Ss58Codec},
testing::SR25519,
};
use sp_application_crypto::{ed25519, sr25519, AppPublic, AppKey, AppPair};
use std::{
fs,
str::FromStr,
};
/// Generate a new key.
///
/// Places it into the file system store.
fn generate<Pair: AppPair>(store: &KeystoreInner) -> Result<Pair> {
store.generate_by_type::<Pair::Generic>(Pair::ID).map(Into::into)
}
/// Create a new key from seed.
///
/// Does not place it into the file system store.
fn insert_ephemeral_from_seed<Pair: AppPair>(store: &mut KeystoreInner, seed: &str) -> Result<Pair> {
store.insert_ephemeral_from_seed_by_type::<Pair::Generic>(seed, Pair::ID).map(Into::into)
}
/// Get public keys of all stored keys that match the key type.
///
/// This will just use the type of the public key (a list of which to be returned) in order
/// to determine the key type. Unless you use a specialized application-type public key, then
/// this only give you keys registered under generic cryptography, and will not return keys
/// registered under the application type.
fn public_keys<Public: AppPublic>(store: &KeystoreInner) -> Result<Vec<Public>> {
store.raw_public_keys(Public::ID)
.map(|v| {
v.into_iter()
.map(|k| Public::from_slice(k.as_slice()))
.collect()
})
}
/// Get a key pair for the given public key.
fn key_pair<Pair: AppPair>(store: &KeystoreInner, public: &<Pair as AppKey>::Public) -> Result<Pair> {
store.key_pair_by_type::<Pair::Generic>(IsWrappedBy::from_ref(public), Pair::ID).map(Into::into)
}
#[test]
fn basic_store() {
let temp_dir = TempDir::new().unwrap();
let store = KeystoreInner::open(temp_dir.path(), None).unwrap();
assert!(public_keys::<ed25519::AppPublic>(&store).unwrap().is_empty());
let key: ed25519::AppPair = generate(&store).unwrap();
let key2: ed25519::AppPair = key_pair(&store, &key.public()).unwrap();
assert_eq!(key.public(), key2.public());
assert_eq!(public_keys::<ed25519::AppPublic>(&store).unwrap()[0], key.public());
}
#[test]
fn test_insert_ephemeral_from_seed() {
let temp_dir = TempDir::new().unwrap();
let mut store = KeystoreInner::open(temp_dir.path(), None).unwrap();
let pair: ed25519::AppPair = insert_ephemeral_from_seed(
&mut store,
"0x3d97c819d68f9bafa7d6e79cb991eebcd77d966c5334c0b94d9e1fa7ad0869dc"
).unwrap();
assert_eq!(
"5DKUrgFqCPV8iAXx9sjy1nyBygQCeiUYRFWurZGhnrn3HJCA",
pair.public().to_ss58check()
);
drop(store);
let store = KeystoreInner::open(temp_dir.path(), None).unwrap();
// Keys generated from seed should not be persisted!
assert!(key_pair::<ed25519::AppPair>(&store, &pair.public()).is_err());
}
#[test]
fn password_being_used() {
let password = String::from("password");
let temp_dir = TempDir::new().unwrap();
let store = KeystoreInner::open(
temp_dir.path(),
Some(FromStr::from_str(password.as_str()).unwrap()),
).unwrap();
let pair: ed25519::AppPair = generate(&store).unwrap();
assert_eq!(
pair.public(),
key_pair::<ed25519::AppPair>(&store, &pair.public()).unwrap().public(),
);
// Without the password the key should not be retrievable
let store = KeystoreInner::open(temp_dir.path(), None).unwrap();
assert!(key_pair::<ed25519::AppPair>(&store, &pair.public()).is_err());
let store = KeystoreInner::open(
temp_dir.path(),
Some(FromStr::from_str(password.as_str()).unwrap()),
).unwrap();
assert_eq!(
pair.public(),
key_pair::<ed25519::AppPair>(&store, &pair.public()).unwrap().public(),
);
}
#[test]
fn public_keys_are_returned() {
let temp_dir = TempDir::new().unwrap();
let mut store = KeystoreInner::open(temp_dir.path(), None).unwrap();
let mut keys = Vec::new();
for i in 0..10 {
keys.push(generate::<ed25519::AppPair>(&store).unwrap().public());
keys.push(insert_ephemeral_from_seed::<ed25519::AppPair>(
&mut store,
&format!("0x3d97c819d68f9bafa7d6e79cb991eebcd7{}d966c5334c0b94d9e1fa7ad0869dc", i),
).unwrap().public());
}
// Generate a key of a different type
generate::<sr25519::AppPair>(&store).unwrap();
keys.sort();
let mut store_pubs = public_keys::<ed25519::AppPublic>(&store).unwrap();
store_pubs.sort();
assert_eq!(keys, store_pubs);
}
#[test]
fn store_unknown_and_extract_it() {
let temp_dir = TempDir::new().unwrap();
let store = KeystoreInner::open(temp_dir.path(), None).unwrap();
let secret_uri = "//Alice";
let key_pair = sr25519::AppPair::from_string(secret_uri, None).expect("Generates key pair");
store.insert_unknown(
SR25519,
secret_uri,
key_pair.public().as_ref(),
).expect("Inserts unknown key");
let store_key_pair = store.key_pair_by_type::<sr25519::AppPair>(
&key_pair.public(),
SR25519,
).expect("Gets key pair from keystore");
assert_eq!(key_pair.public(), store_key_pair.public());
}
#[test]
fn store_ignores_files_with_invalid_name() {
let temp_dir = TempDir::new().unwrap();
let store = LocalKeystore::open(temp_dir.path(), None).unwrap();
let file_name = temp_dir.path().join(hex::encode(&SR25519.0[..2]));
fs::write(file_name, "test").expect("Invalid file is written");
assert!(
SyncCryptoStore::sr25519_public_keys(&store, SR25519).is_empty(),
);
}
}