From cb9bc9dd9d6c856e6496bcf6570ac0452fd60bc6 Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Sun, 17 Dec 2023 20:24:09 -0500 Subject: [PATCH 1/9] init --- docs/modules/ROOT/nav.adoc | 1 + .../ROOT/pages/guides/weights_fees.adoc | 77 +++++++++++++++++++ 2 files changed, 78 insertions(+) create mode 100644 docs/modules/ROOT/pages/guides/weights_fees.adoc diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc index 774a7e4..0d59e2e 100644 --- a/docs/modules/ROOT/nav.adoc +++ b/docs/modules/ROOT/nav.adoc @@ -1,4 +1,5 @@ * General Guides +** xref:guides/weights_fees.adoc[weights_fees] * Runtime Descriptions * Pallet Specifications ** xref:pallets/proxy.adoc[pallet_proxy] diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc new file mode 100644 index 0000000..44e0e0e --- /dev/null +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -0,0 +1,77 @@ +:source-highlighter: highlight.js +:highlightjs-languages: rust +:github-icon: pass:[] + += Weights & Fees + +Weight provides a metric to estimate the time it takes to execute code. + +By assigning a weight to every transaction, Substrate uses this metric to bound the time it takes to execute and verify each batch of transactions organized into a block. + +By converting the transaction's weight to fees, Substrate charges the transaction's caller with fees in proportion to the cost of execution. + +Table of Contents: +** Limit and bound non-linear complexity +** Weights should be conservative +** Underestimating weight exposes DDOS +** Steps To Generate Runtime-Specific Weights +** References + +== Limit and bound non-linear complexity + +TODO + +== Weights should be conservative + +To limit the execution time for each block, runtime constraints on Weight implicitly enforce upper bounds on computation and storage changes per block. + +TODO: which runtime constraints and how are they enforced? + +TODO + +== Underestimating weight exposes DDOS + +* Important that benchmarking hardware is not significantly more high performance than the hardware used by nodes participating in network consensus (i.e. validators, collators) + +* Using worse hardware (i.e. developer hardware) does take longer and, therefore, indirectly underestimates weights. + + +== Steps to Generate Runtime-Specific Weights + +TODO: why are runtime-specific weights important + +1. Ensure Compilation With Runtime Benchmarks +2. Generate Weights Using Production Hardware +3. Write Weights to Runtime +4. Automate via Bench Bot + +=== Ensure Compilation With Runtime Benchmarks + +* compile `cargo build --features runtime-benchmarks` +* compile `cargo build --release --features runtime-benchmarks` +* Enforce with CI + +=== Generate Weights Using Production Hardware + +Technically, *Weight* represents the time it takes to execute code on *production hardware* used by nodes actively participating in the network's consensus. + +``` +1 unit of weight = 1 picosecond of execution time on target reference hardware +``` + +https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i.[Polkadot Reference Hardware Docs] provides more information on Polkadot validator hardware requirements. + +It is important to ONLY generate weights by running `cargo build --features runtime-benchmarks --release` on *production hardware*. + +=== Write Weights to Runtime. + +TODO: where/why + +=== Automate via Bench Bot + +* examples and references + +== References + +** *https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi]* + From 3a9da886de2fa4edec6e0bdf23de85413e1e7647 Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Sun, 17 Dec 2023 21:34:09 -0500 Subject: [PATCH 2/9] clean --- .../ROOT/pages/guides/weights_fees.adoc | 31 ++++--------------- 1 file changed, 6 insertions(+), 25 deletions(-) diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index 44e0e0e..d430b5c 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -6,36 +6,17 @@ Weight provides a metric to estimate the time it takes to execute code. -By assigning a weight to every transaction, Substrate uses this metric to bound the time it takes to execute and verify each batch of transactions organized into a block. +In Substrate, every transaction has a weight. By default, the block production algorithm selects the set of transactions from the transaction pool that achieve block saturation without exceeding `MaximumBlockWeight`. The `AvailableBlockRatio` ensures only a fraction of `MaximumBlockWeight` is used for regular transactions, but system-critical operations (operational transactions) may use all remaining block weight. -By converting the transaction's weight to fees, Substrate charges the transaction's caller with fees in proportion to the cost of execution. - -Table of Contents: -** Limit and bound non-linear complexity -** Weights should be conservative -** Underestimating weight exposes DDOS -** Steps To Generate Runtime-Specific Weights -** References - -== Limit and bound non-linear complexity - -TODO - -== Weights should be conservative - -To limit the execution time for each block, runtime constraints on Weight implicitly enforce upper bounds on computation and storage changes per block. - -TODO: which runtime constraints and how are they enforced? - -TODO - -== Underestimating weight exposes DDOS +By converting each transaction's weight to fees, Substrate charges every transaction's caller with fees in proportion to the cost of execution. +TODO: organize +* weights should be conservative +* Limit and bound non-linear complexity +* Underestimating weight exposes DDOS * Important that benchmarking hardware is not significantly more high performance than the hardware used by nodes participating in network consensus (i.e. validators, collators) - * Using worse hardware (i.e. developer hardware) does take longer and, therefore, indirectly underestimates weights. - == Steps to Generate Runtime-Specific Weights TODO: why are runtime-specific weights important From 36c41645adf78b2d3413b061d2adbed496594e69 Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Mon, 18 Dec 2023 12:57:59 -0500 Subject: [PATCH 3/9] clean --- docs/modules/ROOT/nav.adoc | 2 +- .../ROOT/pages/guides/weights_fees.adoc | 46 ++++++------------- 2 files changed, 15 insertions(+), 33 deletions(-) diff --git a/docs/modules/ROOT/nav.adoc b/docs/modules/ROOT/nav.adoc index 0d59e2e..2b33619 100644 --- a/docs/modules/ROOT/nav.adoc +++ b/docs/modules/ROOT/nav.adoc @@ -1,5 +1,5 @@ * General Guides -** xref:guides/weights_fees.adoc[weights_fees] +** xref:guides/weights_fees.adoc[Weights & Fees] * Runtime Descriptions * Pallet Specifications ** xref:pallets/proxy.adoc[pallet_proxy] diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index d430b5c..ee0c45f 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -4,35 +4,27 @@ = Weights & Fees -Weight provides a metric to estimate the time it takes to execute code. +Weight is a metric to estimate the time it takes to execute code. -In Substrate, every transaction has a weight. By default, the block production algorithm selects the set of transactions from the transaction pool that achieve block saturation without exceeding `MaximumBlockWeight`. The `AvailableBlockRatio` ensures only a fraction of `MaximumBlockWeight` is used for regular transactions, but system-critical operations (operational transactions) may use all remaining block weight. +In Substrate, every transaction has a weight. The block production algorithm selects the set of transactions from the transaction pool that achieve block saturation without exceeding `MaximumBlockWeight`. The `AvailableBlockRatio` ensures only a fraction of `MaximumBlockWeight` is used for regular transactions while system-critical operations (operational transactions) may use all remaining block weight. -By converting each transaction's weight to fees, Substrate charges every transaction's caller with fees in proportion to the cost of execution. +Substrate charges every transaction's caller with fees in proportion to the cost of execution. It does so by converting each transaction's weight to fees via `WeightToFee`. -TODO: organize -* weights should be conservative -* Limit and bound non-linear complexity -* Underestimating weight exposes DDOS -* Important that benchmarking hardware is not significantly more high performance than the hardware used by nodes participating in network consensus (i.e. validators, collators) -* Using worse hardware (i.e. developer hardware) does take longer and, therefore, indirectly underestimates weights. +Weights which underestimate cost of execution expose a DDOS vulnerability. If a transaction takes up less space than it should, too many transactions may be included in the block and the block's execution time may exceed expected block time. For parachains, this DDOS vulnerability can be critical; it may prevent future block production by bricking the chain until the parachain state is reset on the relay chain. -== Steps to Generate Runtime-Specific Weights +It is imperative for runtime developers to be conservative when approximating weight. Moreover, runtime complexity that is non-linear must be limited and bounded. -TODO: why are runtime-specific weights important +== Runtime-Specific Weights -1. Ensure Compilation With Runtime Benchmarks -2. Generate Weights Using Production Hardware -3. Write Weights to Runtime -4. Automate via Bench Bot +Production runtimes should never set `WeightInfo = ()` in production because this configuration underestimates weight. Instead, every production runtime should generate and set its own weights specific to its runtime. -=== Ensure Compilation With Runtime Benchmarks +Steps to generate Runtime-Specific Weights: +1. Ensure compilation for runtime benchmarks by enforcing a passing CI status for the production benchmarking command `cargo build --release --features runtime-benchmarks`. +2. Generate weights on https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i.[Production Hardware]. +3. Write weights to the Runtime. +4. Automate via a Benchmarking Bot. For an example, see https://github.com/paritytech/command-bot[Command Bot] and https://github.com/paritytech/pipeline-scripts/issues/54[an overview of its design]. -* compile `cargo build --features runtime-benchmarks` -* compile `cargo build --release --features runtime-benchmarks` -* Enforce with CI - -=== Generate Weights Using Production Hardware +== More Reading Technically, *Weight* represents the time it takes to execute code on *production hardware* used by nodes actively participating in the network's consensus. @@ -44,15 +36,5 @@ https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:t It is important to ONLY generate weights by running `cargo build --features runtime-benchmarks --release` on *production hardware*. -=== Write Weights to Runtime. - -TODO: where/why - -=== Automate via Bench Bot - -* examples and references - -== References - +References: ** *https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi]* - From 6c5d32b9d588a0c34ad671d1bf557722ad664a57 Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Mon, 18 Dec 2023 13:35:09 -0500 Subject: [PATCH 4/9] address review comments --- .../ROOT/pages/guides/weights_fees.adoc | 36 ++++++++++++++++--- 1 file changed, 31 insertions(+), 5 deletions(-) diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index ee0c45f..ef100e8 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -19,10 +19,35 @@ It is imperative for runtime developers to be conservative when approximating we Production runtimes should never set `WeightInfo = ()` in production because this configuration underestimates weight. Instead, every production runtime should generate and set its own weights specific to its runtime. Steps to generate Runtime-Specific Weights: -1. Ensure compilation for runtime benchmarks by enforcing a passing CI status for the production benchmarking command `cargo build --release --features runtime-benchmarks`. -2. Generate weights on https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i.[Production Hardware]. -3. Write weights to the Runtime. -4. Automate via a Benchmarking Bot. For an example, see https://github.com/paritytech/command-bot[Command Bot] and https://github.com/paritytech/pipeline-scripts/issues/54[an overview of its design]. +1. Ensure passing status for benchmarking compilation +``` +cargo build --features runtime-benchmarks +``` + + +2. Run the benchmarking command and write the output to the runtime. Here is the command via bash script: +``` +#!/bin/sh + +# Build release runtime benchmarks +cargo build --release --features=runtime-benchmarks + +# Collect all pallets needed for benchmarking +# Makes the assumption all pallets are present at: /pallets/$PALLET_NAME +pallets=$(ls pallets/) + +# Generate weights +for pallet_name in $pallets; do + ./target/release/node benchmark pallet \ + --pallet pallet_$pallet_name \ + --extrinsic "*" \ + --steps 50 \ + --repeat 20 \ + --output ./runtime/src/weights/$pallet_name.rs +done +``` + +3. Enforce (1) in the CI, and automate (2) via a Benchmarking Bot. For example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. == More Reading @@ -37,4 +62,5 @@ https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:t It is important to ONLY generate weights by running `cargo build --features runtime-benchmarks --release` on *production hardware*. References: -** *https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi]* +** https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi] + From f4afc88e2913ae281df8d47a06ea93e3726d0ead Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Mon, 18 Dec 2023 13:37:49 -0500 Subject: [PATCH 5/9] fix number formatting --- docs/modules/ROOT/pages/guides/weights_fees.adoc | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index ef100e8..8a8e2ac 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -18,14 +18,12 @@ It is imperative for runtime developers to be conservative when approximating we Production runtimes should never set `WeightInfo = ()` in production because this configuration underestimates weight. Instead, every production runtime should generate and set its own weights specific to its runtime. -Steps to generate Runtime-Specific Weights: -1. Ensure passing status for benchmarking compilation +The first step is to ensure passing status for benchmarking compilation ``` cargo build --features runtime-benchmarks ``` - -2. Run the benchmarking command and write the output to the runtime. Here is the command via bash script: +The second step is to run the benchmarking command and write its output to the runtime. Here is the command via bash script: ``` #!/bin/sh @@ -47,7 +45,7 @@ for pallet_name in $pallets; do done ``` -3. Enforce (1) in the CI, and automate (2) via a Benchmarking Bot. For example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. +The final step is to automate the steps above. the first step may be automated by enforcing the command in the CI. The second step is automated by integrating a Benchmarking Bot. For an example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. == More Reading From 6e2bd3a24604b33aeb1b61357278102dea6c3f37 Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Mon, 18 Dec 2023 13:40:29 -0500 Subject: [PATCH 6/9] clean --- docs/modules/ROOT/pages/guides/weights_fees.adoc | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index 8a8e2ac..a0a6e55 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -45,7 +45,7 @@ for pallet_name in $pallets; do done ``` -The final step is to automate the steps above. the first step may be automated by enforcing the command in the CI. The second step is automated by integrating a Benchmarking Bot. For an example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. +The final step is to automate the steps above. The first step may be automated by enforcing the command in the CI. The second step is automated by integrating a Benchmarking Bot. For an example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. == More Reading @@ -55,9 +55,7 @@ Technically, *Weight* represents the time it takes to execute code on *productio 1 unit of weight = 1 picosecond of execution time on target reference hardware ``` -https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i.[Polkadot Reference Hardware Docs] provides more information on Polkadot validator hardware requirements. - -It is important to ONLY generate weights by running `cargo build --features runtime-benchmarks --release` on *production hardware*. +It is important to ONLY generate weights on *production hardware*. https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i.[Polkadot Reference Hardware Docs] provides more information on Polkadot validator hardware requirements. References: ** https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi] From 9c8f4f4b57da7c92476f9c9171d4d51a31c518e8 Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Mon, 18 Dec 2023 19:30:34 -0500 Subject: [PATCH 7/9] format --- docs/modules/ROOT/pages/guides/weights_fees.adoc | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index a0a6e55..e49a9fb 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -18,12 +18,12 @@ It is imperative for runtime developers to be conservative when approximating we Production runtimes should never set `WeightInfo = ()` in production because this configuration underestimates weight. Instead, every production runtime should generate and set its own weights specific to its runtime. -The first step is to ensure passing status for benchmarking compilation +. Ensure passing status for benchmarking compilation by running this command: ``` cargo build --features runtime-benchmarks ``` -The second step is to run the benchmarking command and write its output to the runtime. Here is the command via bash script: +. Run the benchmarking command and write its output to the runtime. Here is the command via bash script: ``` #!/bin/sh @@ -45,7 +45,7 @@ for pallet_name in $pallets; do done ``` -The final step is to automate the steps above. The first step may be automated by enforcing the command in the CI. The second step is automated by integrating a Benchmarking Bot. For an example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. +. Automate the benchmarking pipeline. The first step may be automated by enforcing the command in the CI. The second step may be automated by integrating a benchmarking github bot. For an example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. == More Reading @@ -55,7 +55,7 @@ Technically, *Weight* represents the time it takes to execute code on *productio 1 unit of weight = 1 picosecond of execution time on target reference hardware ``` -It is important to ONLY generate weights on *production hardware*. https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i.[Polkadot Reference Hardware Docs] provides more information on Polkadot validator hardware requirements. +It is important to ONLY generate weights on *production hardware*. https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i[Polkadot Reference Hardware Docs] provides more information on Polkadot validator hardware requirements. References: ** https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi] From fcb7015025b4dc8f2ca92fc23b2e613b527a334c Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Mon, 18 Dec 2023 19:32:55 -0500 Subject: [PATCH 8/9] fmt --- docs/modules/ROOT/pages/guides/weights_fees.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index e49a9fb..46385c8 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -22,7 +22,7 @@ Production runtimes should never set `WeightInfo = ()` in production because thi ``` cargo build --features runtime-benchmarks ``` - +[start=2] . Run the benchmarking command and write its output to the runtime. Here is the command via bash script: ``` #!/bin/sh @@ -44,7 +44,7 @@ for pallet_name in $pallets; do --output ./runtime/src/weights/$pallet_name.rs done ``` - +[start=3] . Automate the benchmarking pipeline. The first step may be automated by enforcing the command in the CI. The second step may be automated by integrating a benchmarking github bot. For an example, see https://github.com/paritytech/command-bot[Parity's Command Bot]. == More Reading From 840b919921a595bb8d6e5738a292dd3442168856 Mon Sep 17 00:00:00 2001 From: 4meta5 Date: Mon, 18 Dec 2023 19:35:12 -0500 Subject: [PATCH 9/9] format --- docs/modules/ROOT/pages/guides/weights_fees.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/modules/ROOT/pages/guides/weights_fees.adoc b/docs/modules/ROOT/pages/guides/weights_fees.adoc index 46385c8..3bf177e 100644 --- a/docs/modules/ROOT/pages/guides/weights_fees.adoc +++ b/docs/modules/ROOT/pages/guides/weights_fees.adoc @@ -57,6 +57,6 @@ Technically, *Weight* represents the time it takes to execute code on *productio It is important to ONLY generate weights on *production hardware*. https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot#:~:text=Reference%20Hardware%E2%80%8B,instance%20on%20GCP%20and%20c6i[Polkadot Reference Hardware Docs] provides more information on Polkadot validator hardware requirements. -References: -** https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi] +=== References +https://www.shawntabrizi.com/blog/substrate/substrate-weight-and-fees/[Substrate Weight & Fees] by https://github.com/shawntabrizi/[Shawn Tabrizi]