diff --git a/pdf/grandpa.tex b/pdf/grandpa.tex index 9e02cb5..1891b62 100644 --- a/pdf/grandpa.tex +++ b/pdf/grandpa.tex @@ -63,7 +63,7 @@ An alternative to probablistic finality is having \emph{provable finality} that %This is useful to prove what happened to light clients, who do not have the full chain or are not actively listening to the network, and to communicate with other chains, possibly as part of a scalability solution, where not anyone receives or stores all the data in the system. -New generation protocols~\cite{Tendermint,AlgorandAgreement,kokoris16enhancing,decker16bitcoin,pass16hybrid} propose the complete opposite, where every blocks are finalized one by one that forfeits liveness when finality is not readily achievable. This gives provable finality immediately.% However this is slow if we have a large set of participants in the Byzantine agreement. +New generation protocols~\cite{Tendermint,kokoris16enhancing,decker16bitcoin,pass16hybrid} propose the complete opposite, where every blocks are finalized one by one that forfeits liveness when finality is not readily achievable. This gives provable finality immediately.% However this is slow if we have a large set of participants in the Byzantine agreement. Protocols that finalize blocks one by one has the shortcomings that many consensus particpants leads to slow performance. Hence, they need to put a limit on the number of consensus participats which might lead to centralization. @@ -690,224 +690,6 @@ Then either all honest participants finalise $B$ before time $t_r+6T$ or no hone -\section{Optimized version of GRANDPA} - - There are a few ways we can optimise the GRANDPA protocol. - Firstly, a participant that is offline for many rounds should be able to catch up to the latest round by only seeing recent messages. - Secondly, we shouldn't need to actively use many rounds worth of votes, only needing old rounds for challenges for accountable safety and not finalising blocks. - Thirdly, We should wait $2T$ as little as possible. Conversely if communication is faster than block production, we shouldn't be running many rounds before a new block arrives. - - To achieve this, we need to have more complicated conditions for when to perform each step of the protocol. Here is the resulting protocol: - - To enter a round $r$, $v$ needs that round $r-1$ is completable and that $E_{r-2,v}$ is finalised. - If $v$ sees messages that give this for a future round $r$, even if $v$ are not in round $r-1$, $v$ jumps straight to round $r$. - (when checking this condition, for the finalisation, we need to relax not finalising using precommits from future rounds to all rounds $< r$). - - \noindent \fbox{\parbox{6.3in}{ - \begin{enumerate} - \item If $v$ is the primary, it broadcast $E_{r-1,v}$ at the start time $t_{r,v}$ - - \item We prevote when one of the folowing conditions tells us to. - \begin{itemize} - %\item[(i)] If it is impossible for $V_{r-1,v}$ to have a supermajority for any children of $E_{r-1,v}$, then $v$ prevotes for the best chain containing $E_{r-1,v}$ - \item[(i)] If $v$ has received $B$ from the primary, $v$ prevotes for the head of the best chain containing $B$ as soon as one of the following holds: - - \begin{itemize} - \item[(a)] $g(V_{r-1,v}) \geq B \geq E_{r-1,v}$ - \item[(b)] The best chain containing $B$ is also the best chain containing $E_{r-1,v}$ - (equivalently if we evaluate the best chain containing the eariler of the two blocks, then it contains the other) - \end{itemize} - \item[(ii)] If round $r$ is completable and $E_{r,v} \geq E_{r-1,v}$, then we prevote for $E_{r,v}$. - \item[(iii)] if we have reached time $t_{r,v}+2T$ then if we have not recieved a message from the primary or (i) (a) does not hold, then $v$ prevotes for the head of best chain containing $E_{r-1,v}$ anyway. - \end{itemize} - - \item After prevoting, we wait until $g(V_{r,v}) \geq E_{r-1,v}$, then when one of the following holds, we precommit $g(V_{r,v})$ - \begin{itemize} - \item[(i)] if round $r$ is completable - \item[(ii)] if $v$ has seen a child of the last finalised block and it is impossible for $V_{r,v}$ to have a supermajority for any child of $g(V_{r,v})$ . - \item[(iii)] If $v$ has seen a child of the last finalised block and we have reached time $t_{r,v}+4T$. - \end{itemize} - \end{enumerate} - }} - - We claim that all results we proved about the protocol described in Section \ref{sec:grandpa} apply to this protocol. the stronger properties this satisifies are that $v$ does not need to store votes from before round $r-1$ (except to answer challenges for accountable safety, which should be rare) and that if we have seen no descendants of the last finalised block, we pause until we do. - - - - -\section{Practicalities} - -\subsection{Changing the voter set on-chain in an asynchronously safe way} - -\subsubsection{Changing the voter set in an asynchronously safe way} - -Suppose we have an on-chain protocol that decides we need a different voter set. Once everyone finalises the block, they know that we need to change the set. The protocol can cope with changing the voter set from some round $r$. -The main difficulty is that the chain has no idea what the current round number is and even if we have a block that instructs us to change the voter set at round $r$, we might only finalise the block after round $r$. -So instead we will not take advantage of the ability to change set from one round to the next. - -A block $B$ can contain an instruction that we should change to the voter set to some other set after some integer $m \geq 0$ blocks. If our best chain for a prevote contains such a block $B$, then we do not prevote for more than $m$ blocks after $B$, even if our best chain is longer. -Thus if the current voter set has $n-f$ honest voters, they will only finalise $m$ blocks after such a $B$. We only accept votes and commit messages up top $m$ blocks after $B$ from the current set of voters. - -When some block $B'$ that is $m$ blocks after $B$ has been finalised, then the new voter set starts again at round $1$ with $E_{0}=B'$. Votes will need to contain additional metadata that indicates the voter set somehow. - -\subsubsection{Unsafe fallback for changing the voter set after stalling} - -In extreme circumstances, we may need to deal with $1/3$ of voters being offline. -There is no asynchronously safe way of doing this. It also breaks the chain of signed statements by the existing set of voters saying who the future set of voters should be. And it means we may be vulnerable to being cut of by Byzantine participants. -However if we are in a state when many voters go offline but the network is not partitioned, then we want a way to agree on a set of new voters to restart the finality gadget. - -Every 100 blocks or so, we should put a valid commit message on chain. Honest block producers should put the most recent message on the chain, provided that there is one for a more recent block than 100 blocks ago. -Then if a participant sees that their best chain has not had such a message for 1000 blocks and are not aware of any more recent blocks being finalised, then they set a new voter set to be one determined by the 900th block since the last commit message on chain. - -The protocol for selecting voters should require recent messages on chain signed by those voters so that this is likely to give a set of voters very few of whom are offline. - -We should consider having to manually approve finality agreed upon by this new set to alleviate the security concerns above. But this still gives a way to canonically agree on a new set, in the event of WW3 or bad initialisation of a new chain. - -If we do not want to put commit messages on chain, then we can alternatively do the following. Every block producer puts the highest block number that they see as finalised in their block. - -Then any participant sees that if there is an $n$ such that -\begin{itemize} -\item[(i)] their best chain is at least length $n+100$ -\item[(ii)] the indicators the last finalised block height of blocks $n-100$ to $n$ in their best chain have median at most $n-1050$ and -\item[(iii]) $n$ is the minimum that satisifies (i) and (ii) -\end{itemize} -then they switch to the best voter set given by block $n$. If the same block at height $n$ is on everyone's best chain, which can be shown to occur with high probability given (i) for many block production mechanisms, then everyone will eventually agree that we should switch to the voter set given by that block. If any $100$ consectutive blocks of the best chain are produced by honest and synchronised block producers then this will only happen if GRANDPA fails to finalise any block in the time it took to produce $1000$ blocks. - -\subsection{Alternatives to the last block hash} - -The danger with voting for the last blockhash in the best chain is that maybe no one else will have seen and processed the next block. It would also be nice to make the most of BLS multisig/aggregation, which allows a single signature for many messages/signers than can be checked in time proportional to the number of different messages signed. - -To get round the first alone, it might be better to vote for a block 3/4 along (rounding further) the unfinalised chain , rather than for th head. - -But the second suggests that maybe we should be including signatures for several of the latest blocks in a chain. We could include that last 2 or 3. We could also do e.g. the the blocks with block numbers with the last 2 multiples of each power of two since the last finalised block, which gives log unfinalised chain length messages but should have many blocks in common. - -When presented with a vote that includes many blocks, we should interpret them as being for the last block we've seen if any. Then we need to be able to update that vote to a later block when that is seen. This retains monotonicity of a supermajority for/ it is impossible to have a supermajority for over time. - -It does not matter if some of the votes are for a block that does not exist as everyone will ignore that part of the vote. But including votes for block that are seen but are not on a chain is an equivocation and is slashable. We need to count such votes as votes for the had of every chain in the vote (as someone might interpret them as for any one of them). - -Then if we need to BLS aggregate votes that are $\geq B$ for a commit message or query response, it is OK to use any vote that is $\geq B$, not necessarily the vote for the head. This should reduce the number of blockhashs sign, in the optimistic case down to 1. - -\subsection{ Block production rule} - -If we adopt that rule that block producers should build on the best chain including the last finalised block, then if we don't finalise another block this will eventually include some prefix beyond the last finalised block, and therefore the protocol is live by Lemma \ref{lem:honest-recent-validity}. - -But the issue is that if agreement is much slower than block production, then we might have a prevote for a short chain on the last finalised block, then the best chain does not include that block and we build a long chain that is eventually never finalised. This could be fixed by building on $E_{r-1}$ or $E_r$. But if we do that, and these change very quickly, then we may never come to agreement on the best chain. - -So we have two possible chain selection rules for block producers: - -\begin{enumerate} -\item Build on the best chain including the last finalise block B. -\item Build on best chain including whichever of $\{E_r,E_{r-1},B\}$ is latest and $\geq B$. -\end{enumerate} - -1 is better if finalisation is happening quickly compared to block production and 2 is best if block production is much faster. We could also consider hybrid rules like adopt 1 unless we see that the protocol is stuck or slow, then we switch to 2. -\com{ -\section{The asynchronous finality gadget problem} - -Here we give an extension of the \cite{flp} result that shows the impossibility of having an asynchronous and deterministic finality gadget protocol and give an asynchronous protocol that uses a common coin primitive. - -\subsection{1/5 BFT finality gadget using a common coin} - -In this section, we will assume the asynchronous gossip network model. By the previous impossibility result, we will need to use randomness to get a finality gadget in this model. We assume that we have access to a common coin protocol. - -For every vote, We have $n$ voters , at most $f$ of which are Byzantine and $n = 5f+1$. For a voter $v$, Let $V_{r,v}$, $C_{r,v}$ be the set of prevotes and precommits from round $r$ that $v$ has seen. - -\begin{enumerate} -\item Everyone prevotes for the best chain including the block they were locked to last round. -\item Wait until $V_{v,r}$ contains prevotes from $n-f$ voters. -\item Precommit $g_{3/5}(V_{r,v})$ -\item Call a precommit for $B$ justified if $B \leq g_{3/5}(V_{r,v})$ and if $B < g_{3/5}(V_{r,v})$ then the child $B'$ of $B$ on the chain of $g_{3/5}(V_{r,v})$ has that there are votes from $f+1$ voters in $V_{r,v}$ that are not $\geq B'$. -Wait until $C_{r,v}$ has justified precommits from $n-f$ voters. -\item Call the common coin, $s_r$ -\item If $s_r=1$, finalise $g_{4/5}(C_r)$ -\item lock to $g_{(4-3s_r)/5}(C_r)$ for next round. -\end{enumerate} - -The common coin is a (secure cryptographic implementation of) the following protocol. It does not return a coin until more than $4f+1$ voters (for the prevote vote in the next round in case of ambiguity) call it. It returns at the latest shortly after all honest voters call it. - When it does, it returns an $s_r$ sampled uniformly from $\{0,1\}$, identical for all who called it, and before $4f+1$ called it, no-one has any information about the result. - -Here $g_{t}(S)$ is the $t$-GHOST function defined as follows. We construct a chain starting with the genesis block and adding the child of the current block such that most voters have votes $\geq$ it until there are $nt$ or less votes for any child of the current block, when we return the current block. - -The idea behind the proof of asynchronous liveness is that for a particular block $B'$, some value of the common coin, either all the honest voters who received $4/5$ of precommits before the common coin was decided lock to $B'$ or none do. -If we had a fixed threshold for locking, an adversarial choice of the number of precommits for $B'$ or its descendants could lead to some voters locking to it and some not (and indeed there would be runs that do this indefinitely as this is how the impossibility result works for this type of algorithm.) - -Firstly we note that much of the machinery of section $\ref{sec:prelims}$ carries over to the $1/5$ byzantine case. - -\begin{lemma} \label{lem:ghost-monotonicity-general} -Let $T$ be a set of votes such that at most $f$ voters have multiple votes in $T$. Let $t \geq (n+f)/2n$ Then -\begin{enumerate} -\item The above definition uniquely defines $g_t(T)$. -\item If $S \subseteq T$ has $g_t(S) \neq$ nil, then $g_t(S) \leq g_t(T)$ for $t \geq (n+f)/2n$. -\item If $S_i \subseteq T$ for $1 \leq i \leq n$ then all non-nil $g_t(S_i)$ are on a single chain with head $g(T)$. -\item If $r \leq s$, then $g_r(T) \geq g_s(T)$. -\end{enumerate} -\end{lemma} -So with $n=5f+1$, $g_{3/5}$ is sufficient for uniqueness. - -First we need to show that the protocol is deadlock free. -As long as all honest voters prevote and precommit, any participant eventually sees prevotes and precomits from $n-f$ voters. We just need to show that honest prevotes are eventually seen as justified. -\begin{lemma} Suppose that an honest voter $v$ precommits $B$ in round $r$. If $V'_{r,v}$ is the set of prevotes they saw at the time they precommited and another participant $v'$ sees all these prevotes i.e. $V_{r,v'} \supseteq V'_{r,v}$, then $v'$ sees $v$'s precommit for $B$ as justified. \end{lemma} -\begin{proof} $v$ precommits $B = g_{3/5}(V'_{r,v})$. Since $V_{r,v'} \supseteq V'_{r,v}$, $B \leq g_{3/5}(V_{r,v'})$ by Lemma \ref{lem:ghost-monotonicity-general} 2. -So we just need to show that if $B < g_{3/5}(V_{r,v'})$, $V_{r,v'}$ contains votes from $f+1$ voters that are not $\geq B'$ where $B'$ is the child of $B$ in the chain of $g_{3/5}(V_{r,v'})$. -Since $B = g_{3/5}(V'_{r,v})$, from the definition of $g$, $B'$, like any child of $B$, does not have votes from $3f+1$ voters $\geq B'$ in $V'_{r,v}$. -Since $V'_{r,v}$ contains votes from $4f+1$ voters, there are votes from at least $f+1$ voters that are $\not \geq B'$ in $V'_{r,v}$ and so also in $V_{r,v'}$. -\end{proof} - -Our network assumption and a simple induction shows that we do not deadlock. -\begin{corollary} All honest voters eventually prevote and precommit in every round and all honest participants reach every round.\end{corollary} - -\begin{lemma} \label{lem:possibly-final-implies-permanent-lock} -If there are enough precommits to finalise a block $B$ in round $r$, then all honest voters who prevote in future rounds will be locked to $B$ or its descendants when they do. At the end of the next round $r' > r$ with $s_{r'}=1$, all participants will have finalised $B$. -\end{lemma} -\begin{proof} -For $B$ to be finalised in round $r$, there need to be votes from more than $n-f$ voters that are $\geq B$ and $s_r=1$. Any honest participant $v$ also sees that $s_r=1$ and so they lock $g_{1/5}(C_{r,v})$. $C_{r,v}$ contains votes from at least $4f+1$ voters. At most $f$ voters can have votes $\not\geq B$ in $C_{r,v}$ if they also voted $\geq B$ and at most $f$ voters do not have votes in $C_{r,v}$. - Thus at least $2f+1$ voters have votes $\geq B$ in $C_{r,v}$. Because $g_{1/5}$ is not unique in general, to show that $g_{1/5}(C_{r,v}) \geq B$, we also need to show that no block $B' \nsim B$ has $f+1$ voters have votes $\geq B'$ in $C_{r,v}$. If this holds then the procedure to calculate $g_{1/5}$ will not follow chain that does not include $B$ and so it will return a block $\geq B$. - Letting $V_r$ be the set of prevotes ever cast, note that any honest voter $v'$ prevotes for a block $g_{3/5}(V_{r,v'}) \leq g_{3/5}{V_r}$ and so as before honest voters precommit to blocks in one chain. - Since many honest voters precommit $\geq B$, all precommit $\sim B$, and so if $f+1$ voters have votes $\geq B'$ in $B$ then since at least one of those are honest $B' \sim B$. Thus we have $g_{1/5}(C_{r,v}) \geq B$. - -Since all honest voters prevote $\geq B$ in round $r+1$, any participant who waits for votes from $4f+1$ voters will see $g_{3/5}(V_{r+1}) \geq B$ and so all honest voters precommit $\geq B$ in round $r+1$. Since only at most $f$ voters vote $\not \geq B$, only precommits $\geq B$ are ever seen as justified by honest participants. Therefore all honest participants will see $g_{345}(C_{r+1}) \geq -B$. -If $s_r=1$, this is enough to finalise $B$.Since $g_{1/5}(C_{r+1}) \geq g_{4/5}(C_{r+1}) \geq -B$, whatever the common coin, all honest particupants lock $\geq B$. By induction, this holds for all future rounds. - -\end{proof} - -We want to show that this is asynchronously live: - -\begin{proposition} Suppose that block $B$ is finalised before round $r$. With probability at least $1/2$ over the common coin in round $r$, if all voters agree that the best chain including the last finalised block $B$ includes a decedent $B''$, at the prevote step of rounds $r+1$ and $r+2$, then a descendant of $B$ is finalised the next time $s_r=1$ after round $r+2$ or earlier. -\end{proposition} - -\begin{proof} By the Lemma \ref{lem:possibly-final-implies-permanent-lock}, all honest voters prevote in round $r$ for $B$ or its descendants and so all honest voters precommit to $B$ or its descendants. - -Let $V_r$ be the set of prevotes of all voters. Using Lemma \ref{lem:ghost-monotonicity-general}, all honest voters precommit $g_{3/5}(V_r)$ or its ancestors. Since some must precommit $\geq B$ for it to be finalised, $g_{3/5}(V_r) \geq B$. - -For the case $g_{3/5}(V_r)=B$, all honest voters precommit $B$ and so any honest participant sees that -$B = g_{1/5}(C_r) = g_{4/5}(C_r)$. Thus all honest participants -lock $B$ and so are free to prevote for $B''$ or its descendants in round $r+1$. Thus we finalise $B''$ in round $r+1$ or the next round when $s_r=1$ after that. - -Otherwise, let $B'$ be the child of $B$ in the chain of $g_{3/5}(V_r)$. We seek to show that we finalise either $B'$ or $B''$. - -Let $S$ be the set of honest voters who precommit in round $r$ before $4f+1$ voters call the common coin. Let $S'$ be the set of honest voters who call the common coin before it is decided. % Note that $S' \subset S$. -Since $4f+1$ voters call the coin before it decided and honest voters who do so saw precommits from $4f+1$ voters, $S'$ and $S$ each contain at least $3f+1$ voters. - -Let $h$ be the number of voters in $S$ that precommit $B'$ or its descendants. Note that the other $|S|-h$ voters just precommit $B$. - -Now consider a particular voter $v$ and the set $C_{r,v}$ of precommits they received in step 4. the number of voters with precommits in $C_{r,v}$ is at least $4f+1$. -If $v \in S'$, -All the honest voters -with precommits in $C_{r,v}$ are in $S$. In this case we have that the number of votes for $B'$ or its descendants in $C_{r,v}$, $m_v$ has $h-f \leq m_v < h+f$. For $v \notin S'$, since $f$ honest vali8dators can be outside $S$, we have $h-2f \leq m_v \leq h+2f$ - -Since any descendant of $B$ that is not $B'$ or its descendants receives less than $f$ precommits for it or its descendants, we have that either $g_{1/5}(C_{r,v})=B$ or $g_{1/5}(C_{r,v})\geq B'$ and similarly for $g_{4/5}(C_{r,v})$. Now note that if $h \geq 3f+1$, $m_v \geq f+1$ and so $g_{1/5}(C_{r,v}) \geq B'$. On the other hand if $h < 3f+1$, for $v \in S'$, $m_v < 4f+1$ and so $g_{4/5}(C_{r,v})=B$. - -If $h \geq 3f+1$ and $s_r=1$, then every honest voter locks a block $\geq B'$. Thus is round $r+1$, they all prevote $\geq B'$. - By similar reasoning to Lemma \ref{lem:possibly-final-implies-permanent-lock}, we finalise $B'$, the next round $r' > r$ that we have $s_{r'}=1$. - -If $h < 3f+1$ and $s_r=0$, then every $v \in S'$ locks only $B$. But then all such $v$ will prevote their best chain containing $B$ and so a block $\geq B''$. There are only at most $2f$ voters who might not do this, the Byantine voters and the honest voters outside of $|S|$ who prevote $\geq B$. Thus any honest voter who has seen prevotes from $n-f$ voters either sees $g_{3/5}(V_{r+1,v})=B$ or $g_{3/5}(V_{r+1,v}) \geq B'$. Since all honest precommits are either $B$ or $\geq B''$, evry honest voter locks either $B$ or $\geq B''$. Since in round $r+2$, all honest voters see that the best chain including $B$ also includes $B''$, this time they all prevote $\geq B''$. By similar reasoning to Lemma \ref{lem:possibly-final-implies-permanent-lock}, we finalise $B''$, by the next round $r' > r+1$ that we have $s_{r'}=1$. - -Crucially note that $h$ depends only on $S$, which is determined when $4f+1$ voters call the common coin and before it is flipped. Thus $s_r$ is independent of $h$. If $h < 3f+1$ then $s_r=0$ with probability $1/2$ and if $h \geq 3f+1$ then $s_r=1$ with probability $1/2$. So with probability $1/2$, we have either both $h < 3f+1$ and $s_r=0$ or both $h \geq 3f+1$ and $s_r=1$. Thus with probability at least $1/2$, we finalise $B'$ or $B''$ before the next round after $r+1$ when $s_r=1$. -\end{proof} -} - -\bibliography{net,os,sec,soc,theory} +\bibliography{net,os,sec,soc,theory, grandpa} \end{document}