Fix markdown and guideline issues

.circleci/config.yml

@@ -11,9 +11,6 @@ executors:
   node_executor:
     docker:
       - image: circleci/node:12.9.0-stretch
-  ruby_executor:
-    docker:
-      - image: circleci/ruby:2-stretch
 
 commands:
   prepare:
@@ -43,7 +40,7 @@ jobs:
           command: |
             python3 -m venv venv
             . venv/bin/activate
-            pip install -r ./common/build_tools/requirements.txt
+            pip install -q -r ./common/build_tools/requirements.txt
       - save_cache:
           paths:
             - ./venv
@@ -63,10 +60,14 @@ jobs:
           name: Install dependencies
           command: |
             curl -sfL https://install.goreleaser.com/github.com/ValeLint/vale.sh | sh -s v1.7.1
+      - run:
+          name: Cat config
+          command: |
+            cat ./common/build_tools/vale/.vale.ini .circleci/vale.ini > .circleci/cat-vale.ini
       - run:
           name: Run Vale
           command: |
-            ./bin/vale --config ./common/build_tools/vale/.vale.ini --glob='*.{md}' . | tee ./vale.out
+            ./bin/vale --config .circleci/cat-vale.ini --glob='*.{md}' . | tee ./vale.out
       - notify
       - store_artifacts:
           path: ./vale.out
@@ -97,23 +98,30 @@ jobs:
           destination: ./linkchecker.out
 
   markdownlint:
-    executor: ruby_executor
+    executor: node_executor
     steps:
       - prepare
+      - restore_cache:
+          keys:
+            - deps-{{ checksum "./common/build_tools/package-lock.json" }}
       - run:
           name: Install dependencies
           command: |
-            gem install mdl
+            npm ci
       - run:
           shell: /bin/bash #this is a non breaking command so it will always return success
           name: Run Markdownlint info checks
           command: |
-            mdl --ignore-front-matter --style ./common/build_tools/markdownlint/info_style.rb . | tee ./markdownlint_info.out
+            npm run test:markdown:info
       - run:
           name: Run Markdownlint
           command: |
-            mdl --ignore-front-matter --style ./common/build_tools/markdownlint/style.rb . | tee ./markdownlint.out
+            npm run test:markdown
       - notify
+      - save_cache:
+          paths:
+            - ./node_modules
+          key: deps-{{ checksum "./common/build_tools/package-lock.json" }}
       - store_artifacts:
           path: ./markdownlint.out
           destination: ./markdownlint.out

.circleci/vale.ini

@@ -0,0 +1,4 @@
+# Disable guidelines check for auto generated interfaces
+[docs/Reference/Plugins/*/interface.md]
+BasedOnStyles =
+vale.Repetition = NO

.gitignore

@@ -4,6 +4,7 @@ env/
 *.out
 node_modules/
 .circleci/process.yml
+.circleci/cat-vale.ini
 .DS_Store
 **/.DS_Store
-*.iml
+*.iml

.markdownlintignore

@@ -0,0 +1 @@
+docs/Reference/Plugins/**/*terface.md

docs/Concepts/AccountManagement.md

@@ -1,13 +1,13 @@
 # Account/Key Management
 
-Cryptographic keys are an essential component of a GoQuorum network.  GoQuorum uses keys to create digital
-signatures which verify a sender's identity and prevent message tampering.  The Privacy Manager uses keys to encrypt private transaction data.
+Cryptographic keys are an essential component of a GoQuorum network. GoQuorum uses keys to create digital
+signatures which verify a sender's identity and prevent message tampering. The Privacy Manager uses keys to encrypt private transaction data.
 
-Both GoQuorum and the Privacy Manager use user-provided asymmetric key pairs.  Each key pair consists
-of a public key and a private key.  The public key can be shared freely, but **the private key should never be shared**.
+Both GoQuorum and the Privacy Manager use user-provided asymmetric key pairs. Each key pair consists
+of a public key and a private key. The public key can be shared freely, but **the private key should never be shared**.
 
 * GoQuorum derives the account address from the public key by taking the last 20 bytes of its keccak256 hash
-* The Privacy Manager uses the public key as an identifier for the target nodes of a private transaction (i.e. the `privateFor` transaction field)
+* The Privacy Manager uses the public key as an identifier for the target nodes of a private transaction (that is the `privateFor` transaction field)
 
 Key management determines how [GoQuorum](../HowTo/ManageKeys/ManagingKeys.md) and [Tessera](https://docs.tessera.consensys.net)
-store and use private keys.  
+store and use private keys.

docs/Concepts/Architecture.md

@@ -1,10 +1,10 @@
 ---
-description: Overview of GoQuorum Architecture 
+description: Overview of GoQuorum Architecture
 ---
 
-# Architecture 
+# Architecture
 
-![](../images/Quorum%20Design.png)
+![GoQuorum Architecture diagram](../images/Quorum%20Design.png)
 
 ## Differences from geth
 
@@ -13,12 +13,12 @@ GoQuorum is updated as geth releases occur.
 
 GoQuorum includes the following modifications to geth:
 
- * Consensus is achieved with the Raft or Istanbul BFT consensus algorithms instead of Proof-of-Work.
- * The P2P layer is modified to only allow connections between permissioned nodes.
- * The block generation logic is modified to replace the `global state root` check with a `global public state root` check.
- * The block validation logic is modified to replace the ‘global state root’ in the block header with the ‘global public state root’
- * The State Patricia trie has been split into two: a public state trie and a private state trie.
- * Block validation logic is modified to handle private transactions. 
- * Transaction creation is modified to allow for transaction data to be replaced by encrypted hashes
-   to preserve private data where required. 
- * The pricing of gas is removed. Gas itself remains. 
+* Consensus is achieved with the Raft or Istanbul BFT consensus algorithms instead of Proof-of-Work.
+* The P2P layer is modified to only allow connections between permissioned nodes.
+* The block generation logic is modified to replace the `global state root` check with a `global public state root` check.
+* The block validation logic is modified to replace the ‘global state root’ in the block header with the ‘global public state root’
+* The State Patricia trie has been split into two: a public state trie and a private state trie.
+* Block validation logic is modified to handle private transactions.
+* Transaction creation is modified to allow for transaction data to be replaced by encrypted hashes
+    to preserve private data where required.
+* The pricing of gas is removed. Gas itself remains.

docs/Concepts/Cakeshop.md

@@ -21,10 +21,10 @@ It provides tools for managing a local blockchain node, setting up clusters,
 exploring the state of the chain, and working with contracts.
 
 The Cakeshop package includes [Tessera](https://docs.tessera.consensys.net), a [Solidity](https://solidity.readthedocs.org/en/latest/)
-compiler, and all dependencies. 
+compiler, and all dependencies.
 
 Cakeshop downloads the latest version of [GoQuorum](https://github.com/ConsenSys/quorum)
 and bootnode from [geth](https://github.com/ethereum/go-ethereum). To use a different version, see
-[here](https://github.com/ConsenSys/cakeshop/blob/master/docs/configuration.md#custom-quorum-binaries). 
+[here](https://github.com/ConsenSys/cakeshop/blob/master/docs/configuration.md#custom-quorum-binaries).
 
 [Get started with Cakeshop.](../HowTo/GetStarted/Cakeshop.md)

docs/Concepts/Consensus/IBFT.md

@@ -1,8 +1,14 @@
+---
+description: IBFT Consensus Overview
+---
+
 # IBFT Consensus Overview
 
 ## Introduction
 
-Istanbul Byzantine Fault Tolerant (IBFT) consensus is inspired by Castro-Liskov 99 [paper](http://pmg.csail.mit.edu/papers/osdi99.pdf). IBFT inherits from the original PBFT by using a 3-phase consensus, `PRE-PREPARE`, `PREPARE` and `COMMIT`. The system can tolerate at most `F` faulty nodes in a `N` validator network, where `N = 3F + 1`.
+Istanbul Byzantine Fault Tolerant (IBFT) consensus is inspired by Castro-Liskov 99 [paper](http://pmg.csail.mit.edu/papers/osdi99.pdf).
+
+IBFT inherits from the original PBFT by using a 3-phase consensus, `PRE-PREPARE`, `PREPARE` and `COMMIT`. The system can tolerate at most `F` faulty nodes in a `N` validator network, where `N = 3F + 1`.
 
 ## Implementation
 
@@ -19,11 +25,13 @@ Istanbul Byzantine Fault Tolerant (IBFT) consensus is inspired by Castro-Liskov
 - `Snapshot`: The validator voting state from last epoch.
 
 ### Consensus
+
 Istanbul BFT Consensus protocol begins at Round `0` with the validators picking a proposer from themselves in a round robin fashion. The proposer will then propose a new block proposal and broadcast it along with the `PRE-PREPARE` message. Upon receiving the `PRE-PREPARE` message from the proposer, other validators validate the incoming proposal and enter the state of `PRE-PREPARED` and broadcast `PREPARE` message. This step is to make sure all validators are working on the same sequence and on the same round. When `ceil(2N/3)` of `PREPARE` messages is received by the validator from other validators, the validator switches to the state of `PREPARED` and broadcasts `COMMIT` message. This step is to inform other validators that it accepts the proposed block and is going to insert the block to the chain. Lastly, validators wait for `ceil(2N/3)` of `COMMIT` messages to enter `COMMITTED` state and then append the block to the chain.
 
-Blocks in Istanbul BFT protocol are final, which means that there are no forks and any valid block must be somewhere in the main chain. To prevent a faulty node from generating a totally different chain from the main chain, each validator appends `ceil(2N/3)` of received `COMMIT` signatures to `extraData` field in the header before inserting it into the chain. Thus all blocks are self-verifiable. However, the dynamic `extraData` would cause an issue on block hash calculation. Since the same block from different validators can have different set of `COMMIT` signatures, the same block can have different block hashes as well. To solve this, we calculate the block hash by excluding the `COMMIT` signatures part. Therefore, we can still keep the block/block hash consistency as well as put the consensus proof in the block header.
+Blocks in Istanbul BFT protocol are final, which means that there are no forks and any valid block must be somewhere in the main chain. To prevent a faulty node from generating a totally different chain from the main chain, each validator appends `ceil(2N/3)` of received `COMMIT` signatures to `extraData` field in the header before inserting it into the chain. Thus all blocks are self-verifiable. However, the dynamic `extraData` would cause an issue on block hash calculation. Since the same block from different validators can have different set of `COMMIT` signatures, the same block can have different block hashes as well. To solve this, we calculate the block hash by excluding the `COMMIT` signatures part. Therefore, we can still keep the block/block hash consistency and put the consensus proof in the block header.
 
 #### Consensus States
+
 Istanbul BFT is a state machine replication algorithm. Each validator maintains a state machine replica in order to reach block consensus. Various states in IBFT consensus are,
 
 - `NEW ROUND`: Proposer to send new block proposal. Validators wait for `PRE-PREPARE` message.
@@ -33,7 +41,8 @@ Istanbul BFT is a state machine replication algorithm. Each validator maintains
 - `FINAL COMMITTED`: A new block is successfully inserted into the blockchain and the validator is ready for the next round.
 - `ROUND CHANGE`: A validator is waiting for `ceil(2N/3)` of `ROUND CHANGE` messages on the same proposed round number.
 
-**State Transitions**:
+##### State Transitions
+
 ![State Transitions](../../images/IBFTStateTransition.png)
 
 - `NEW ROUND` -> `PRE-PREPARED`:
@@ -63,7 +72,7 @@ Istanbul BFT is a state machine replication algorithm. Each validator maintains
 
 #### Round change flow
 
-- There are three conditions that would trigger `ROUND CHANGE`:
+- Three conditions can trigger `ROUND CHANGE`:
     - Round change timer expires.
     - Invalid `PREPREPARE` message.
     - Block insertion fails.
@@ -92,13 +101,14 @@ For all transactions blocks:
 - Votes are tallied live as the chain progresses (concurrent proposals allowed).
 - Proposals reaching majority consensus `VALIDATOR_LIMIT` come into effect immediately.
 - Invalid proposals are not to be penalized for client implementation simplicity.
-- A proposal coming into effect entails discarding all pending votes for that proposal (both for and against) and starts with a clean slate.
+- A proposal coming into effect entails discarding all pending votes for that proposal (both for and against).
 
 #### Future message and backlog
 
 In an asynchronous network environment, one may receive future messages which cannot be processed in the current state. For example, a validator can receive `COMMIT` messages on `NEW ROUND`. We call this kind of message a "future message." When a validator receives a future message, it will put the message into its **backlog** and try to process later whenever possible.
 
 #### Constants
+
 Istanbul BFT define the following constants
 
 - `EPOCH_LENGTH`: Default: 30000 blocks. Number of blocks after which to checkpoint and reset the pending votes.
@@ -108,7 +118,7 @@ Istanbul BFT define the following constants
 - `ISTANBUL_DIGEST`: Fixed magic number `0x63746963616c2062797a616e74696e65206661756c7420746f6c6572616e6365` of `mixDigest` in block header for Istanbul block identification.
 - `DEFAULT_DIFFICULTY`: Default block difficulty, which is set to `0x0000000000000001`.
 - `EXTRA_VANITY`: Fixed number of extra-data prefix bytes reserved for proposer vanity.
-    - Suggested `32` bytes to retain the current extra-data allowance and/or use.
+    - Suggested `32` bytes to retain the current extra-data allowance.
 - `NONCE_AUTH`: Magic nonce number `0xffffffffffffffff` to vote on adding a validator.
 - `NONCE_DROP`: Magic nonce number `0x0000000000000000` to vote on removing a validator.
 - `UNCLE_HASH`: Always `Keccak256(RLP([]))` as uncles are meaningless outside of PoW.
@@ -120,6 +130,7 @@ Istanbul BFT define the following constants
     - Must be `floor(N / 2) + 1` to enforce majority consensus on a chain.
 
 #### Block Header
+
 Istanbul BFT does not add new block header fields. Instead, it follows Clique in repurposing the `ethash` header fields as follows:
 
 - `nonce`: Proposer proposal regarding the account defined by the beneficiary field.
@@ -131,13 +142,15 @@ Istanbul BFT does not add new block header fields. Instead, it follows Clique in
 - `timestamp`: Must be at least the parent timestamp + `BLOCK_PERIOD`
 - `difficulty`: Must be filled with `0x0000000000000001`.
 - `extraData`: Combined field for signer vanity and RLP encoded Istanbul extra data, where Istanbul extra data contains validator list, proposer seal, and commit seals. Istanbul extra data is defined as follows:
-    ```
+
+    ```text
     type IstanbulExtra struct {
-            Validators    []common.Address 	//Validator addresses
-            Seal          []byte	        //Proposer seal 65 bytes
+            Validators    []common.Address  //Validator addresses
+            Seal          []byte            //Proposer seal 65 bytes
             CommittedSeal [][]byte          //Committed seal, 65 * len(Validators) bytes
     }
     ```
+
     Thus the `extraData` would be in the form of `EXTRA_VANITY | ISTANBUL_EXTRA` where `|` represents a fixed index to separate vanity and Istanbul extra data (not an actual character for separator).
 
     - First `EXTRA_VANITY` bytes (fixed) may contain arbitrary proposer vanity data.
@@ -147,14 +160,16 @@ Istanbul BFT does not add new block header fields. Instead, it follows Clique in
         - `CommittedSeal`: The list of commitment signature seals as consensus proof.
 
 #### Block hash, proposer seal and committed seals
+
 The Istanbul block hash calculation is different from the `ethash` block hash calculation due to the following reasons:
 
 1. The proposer needs to put proposer's seal in `extraData` to prove the block is signed by the chosen proposer.
-2. The validators need to put `ceil(2N/3)` of committed seals as consensus proof in `extraData` to prove the block has gone through consensus.
+1. The validators need to put `ceil(2N/3)` of committed seals as consensus proof in `extraData` to prove the block has gone through consensus.
 
 The calculation is still similar to the `ethash` block hash calculation, with the exception that we need to deal with `extraData`. We calculate the fields as follows:
 
 ##### Proposer seal calculation
+
 By the time of proposer seal calculation, the committed seals are still unknown, so we calculate the seal with those unknowns empty. The calculation is as follows:
 
 - `Proposer seal`: `SignECDSA(Keccak256(RLP(Header)), PrivateKey)`
@@ -163,12 +178,14 @@ By the time of proposer seal calculation, the committed seals are still unknown,
 - `extraData`: `vanity | RLP(IstanbulExtra)`, where in the `IstanbulExtra`, `CommittedSeal` and `Seal` are empty arrays.
 
 ##### Block hash calculation
+
 While calculating block hash, we need to exclude committed seals since that data is dynamic between different validators. Therefore, we make `CommittedSeal` an empty array while calculating the hash. The calculation is:
 
 - `Header`: Same as `ethash` header only with a different `extraData`.
 - `extraData`: `vanity | RLP(IstanbulExtra)`, where in the `IstanbulExtra`, `CommittedSeal` is an empty array.
 
 ##### Consensus proof
+
 Before inserting a block into the blockchain, each validator needs to collect `ceil(2N/3)` of committed seals from other validators to compose a consensus proof. Once it receives enough committed seals, it will fill the `CommittedSeal` in `IstanbulExtra`, recalculate the `extraData`, and then insert the block into the blockchain. **Note** that since committed seals can differ by different sources, we exclude that part while calculating the block hash as in the previous section.
 
 Committed seal calculation:
@@ -179,6 +196,6 @@ Committed seal is calculated by each of the validators signing the hash along wi
 - `CONCAT(Hash, COMMIT_MSG_CODE)`: Concatenate block hash and `COMMIT_MSG_CODE` bytes.
 - `PrivateKey`: Signing validator's private key.
 
-
 ## Provenance
+
 Istanbul BFT implementation in GoQuorum is based on [EIP 650](https://github.com/ethereum/EIPs/issues/650). It has been updated since the EIP was opened to resolve safety issues by introducing locking.

docs/Concepts/Consensus/Overview.md

@@ -1,19 +1,19 @@
-# Consensus Mechanisms 
+# Consensus Mechanisms
 
 With no need for Proof of Work (PoW) in a permissioned network, GoQuorum instead implements multiple
-consensus mechanisms that are more appropriate for consortium chains:  
+consensus mechanisms that are more appropriate for consortium chains:
 
 * Raft-based
-   
-     A consensus model for faster blocktimes, transaction finality, and on-demand block creation.  
+
+     A consensus model for faster blocktimes, transaction finality, and on-demand block creation.
   See [Raft-based documentation](Raft.md) for more information
 
 * Istanbul BFT (Byzantine Fault Tolerance)
-    
+
      A PBFT-inspired consensus algorithm with immediate transaction finality, by AMIS.
     See [IBFT documentation](IBFT.md), the [RPC API](../../Reference/Consensus/IBFT-RPC-API.md),
-    and this [technical web article](https://medium.com/getamis/istanbul-bft-ibft-c2758b7fe6ff). 
+    and this [technical web article](https://medium.com/getamis/istanbul-bft-ibft-c2758b7fe6ff).
 
 * Clique Proof of Authority (PoA)
-     
-      A default POA consensus algorithm bundled with Go Ethereum.  See [Clique POA Consensus Documentation](https://github.com/ethereum/EIPs/issues/225) and a [guide to setup clique json](https://hackernoon.com/hands-on-creating-your-own-local-private-geth-node-beginner-friendly-3d45902cc612) with [puppeth](https://blog.ethereum.org/2017/04/14/geth-1-6-puppeth-master/)
+
+      A default POA consensus algorithm bundled with Go Ethereum. See [Clique POA Consensus Documentation](https://github.com/ethereum/EIPs/issues/225) and a [guide to setup clique json](https://hackernoon.com/hands-on-creating-your-own-local-private-geth-node-beginner-friendly-3d45902cc612) with [puppeth](https://blog.ethereum.org/2017/04/14/geth-1-6-puppeth-master/)