New Github Workflow to check extrinsic ordering (#3620)

* ci: new workflow for exrtinsic ordering checks from bin

* fix workflow

* sanitize output and show result

* add context to the output

* Add default url

* fix envs

* fix output generation

* Fix the release todos with a link to the gha

* fix typo

polkadot/.github/ISSUE_TEMPLATE/release.md

@@ -105,26 +105,15 @@ functions. Compare the metadata of the current and new runtimes and ensure that
 the `module index, call index` tuples map to the same set of functions. In case
 of a breaking change, increase `transaction_version`.
 
-To verify the order has not changed:
+To verify the order has not changed, you may manually start the following [Github Action](https://github.com/paritytech/polkadot/actions/workflows/extrinsic-ordering-check-from-bin.yml). It takes around a minute to run and will produce the report as artifact you need to manually check.
 
-1. Download the latest release-candidate binary either from the draft-release
+The things to look for in the output are lines like:
-on Github, or
-[AWS](https://releases.parity.io/polkadot/x86_64-debian:stretch/{{ env.VERSION }}-rc1/polkadot)
-(adjust the rc in this URL as necessary).
-2. Run the release-candidate binary using a local chain:
-`./polkadot --chain=polkadot-local` or `./polkadot --chain=kusama-local`
-3. Use [`polkadot-js-tools`](https://github.com/polkadot-js/tools) to compare
-the metadata:
-  - For Polkadot: `docker run --network host jacogr/polkadot-js-tools metadata wss://rpc.polkadot.io ws://localhost:9944`
-  - For Kusama: `docker run --network host jacogr/polkadot-js-tools metadata wss://kusama-rpc.polkadot.io ws://localhost:9944`
-4. Things to look for in the output are lines like:
   - `[Identity] idx 28 -> 25 (calls 15)` - indicates the index for `Identity` has changed
   - `[+] Society, Recovery` - indicates the new version includes 2 additional modules/pallets.
   - If no indices have changed, every modules line should look something like `[Identity] idx 25 (calls 15)`
 
 Note: Adding new functions to the runtime does not constitute a breaking change
-as long as they are added to the end of a pallet (i.e., does not break any
+as long as the indexes did not change.
-other call index).
 
 ### Proxy Filtering
 

polkadot/.github/workflows/extrinsic-ordering-check-from-bin.yml

@@ -0,0 +1,69 @@
+# This workflow performs the Extrinsic Ordering Check on demand using a binary
+
+name: Extrinsic Ordering Check from Binary
+on:
+  workflow_dispatch:
+    inputs:
+      reference_url:
+        description: The WebSocket url of the reference node
+        default: wss://rpc.polkadot.io
+        required: true
+      binary_url:
+        description: A url to a Linux binary for the node containing the runtime to test
+        default: https://releases.parity.io/polkadot/x86_64-debian:stretch/v0.9.9-rc1/polkadot
+        required: true
+      chain:
+        description: The name of the chain under test. Usually, you would pass a local chain
+        default: polkadot-local
+        required: true
+
+jobs:
+  check:
+    name: Run check
+    runs-on: ubuntu-latest
+    env:
+      CHAIN: ${{github.event.inputs.chain}}
+      BIN_URL: ${{github.event.inputs.binary_url}}
+      REF_URL: ${{github.event.inputs.reference_url}}
+
+    steps:
+      - name: Fetch binary
+        run: |
+          echo Fetching $BIN_URL
+          wget $BIN_URL
+          chmod a+x polkadot
+          ./polkadot --version
+
+      - name: Start local node
+        run: |
+          echo Running on $CHAIN
+          ./polkadot --chain=$CHAIN &
+
+      - name: Prepare output
+        run: |
+          VERSION=$(./polkadot --version)
+          echo "Metadata comparison:" >> output.txt
+          echo "Date: $(date)" >> output.txt
+          echo "Reference: $REF_URL" >> output.txt
+          echo "Target version: $VERSION" >> output.txt
+          echo "-------------------------------------------" >> output.txt
+
+      - name: Compare the metadata
+        run: |
+          CMD="docker run --network host jacogr/polkadot-js-tools metadata $REF_URL ws://localhost:9944"
+          echo -e "Running:\n$CMD"
+          $CMD >> output.txt
+          sed -z -i 's/\n\n/\n/g' output.txt
+
+      - name: Show result
+        run: cat output.txt
+
+      - name: Stop our local node
+        run: pkill polkadot
+
+      - name: Save output as artifact
+        uses: actions/upload-artifact@v2
+        with:
+          name: ${{ env.CHAIN }}
+          path: |
+            output.txt