New Protocol Features

Network Longevity

LIP0008: Remove pre-hashing for block and transaction signatures

The hash-then-sign paradigm for block and transaction signatures in the Lisk protocol has now been removed. That means the block or transaction data is signed instead of the hash digest of a block or transaction. The main motivation for removing pre-hashing is that pre-hashing adds minor disadvantages with respect to security and performance, and unnecessary complexity to the protocol and implementation

LIP0018: Use base32 encoding of long hash of public keys plus checksum for address

A new format for addresses has now been implemented. On the protocol level, the proposed address format uses 160 bits of the SHA-256 hash of the public key. On the user interface level, a 30-bit checksum generated by the BCH code proposed in BIP 173 is added, and a customized Base32 encoding is used.

The new address format makes the "initialization process" (performing at least one transaction per account to associate a public key to an address) obsolete. Moreover, the collision resistance is significantly increased, and accidentally sending funds to a wrong address becomes almost impossible due to the checksum.

LIP0019: Use full SHA-256 hash of transaction as transactionID

The full SHA-256 hash of a transaction is now used as a transaction ID to avoid potential collisions of transaction IDs. This allows to safely use transaction IDs as unique identifiers and avoids transaction rejection due to unwanted collisions.

LIP0020: Use full SHA-256 hash of block header as blockID

The full SHA-256 hash of the block header is now utilized as the block ID. This increases the security of the Lisk blockchain and will ensure its immutability for a long time into the future. It also allows the removal of the block ID uniqueness check during block validation, as it is also now safe to assume that block ID collisions will not occur.

LIP0023: Introduce vote locking periods and new vote weight definition - banning mechanism

The main aspects of the new DPoS system were already added in Lisk SDK 4.0.0. For Lisk SDK 5.0.0 a banning mechanism for delegates due to low productivity has been introduced to complete our new DPoS system. A delegate can be banned due to low productivity if the delegate missed 50 blocks in a row, and at the same time did not forge for 260,000 blocks (around 1 month).

LIP0027: A generic serialization method

A generic, deterministic, size efficient, and deserializable serialization method is introduced. This enables the serialization of arbitrary data for hashing and signing, and also for storage and transmission. This is applied to (custom) transactions, blocks, and account states.

LIP0028: Define schema and use generic serialization for transactions

The generic serialization is applied to transactions and a new transaction serialization is introduced by specifying the appropriate JSON schema. This serialization method is applied in two steps. Firstly to the transaction asset, which contains the custom properties specific to the transaction, and then secondly to the transaction base object, which is the same for all transactions. This is used for signing, storing, and transmitting transactions in the Lisk protocol, and is now the default serialization method for custom transactions created via the Lisk SDK.

LIP0029: Define schema and use generic serialization for blocks

The generic serialization is applied to blocks and a new serialization for blocks is introduced by specifying the appropriate JSON schema. The block header is restructured and a new block header asset property is introduced. This block header asset allows storing properties specific to the corresponding chain.

LIP0030: Define schema and use generic serialization for account state

This improvement defines a standard and unique method for serializing accounts. The serialization improves storage efficiency, and developers using the SDK benefit from having an expandable and flexible serialization process.

LIP0031: Introduce Merkle trees and inclusion proofs

A generic Merkle tree structure is defined together with a format for inclusion proofs that can be used in different parts of the Lisk protocol.

LIP0032: Replace payload hash with Merkle tree root in block header

The payload hash value stored in each block header has been removed and replaced with a transaction root, this is the root of a Merkle tree built from the transactions included in the payload.

LIP0034: Define new block schema and processing for genesis block

We introduce a new block schema that enables the possibility to specify an initial state of accounts, and an initial delegate list valid for a certain bootstrapping period. This makes it easier to create a genesis block with a desired initial state using the Lisk SDK. The block schema is general enough to also define a more complex state, computed as a snapshot of an existing blockchain.

SDK Improvements

Migrate Lisk Framework to TypeScript

The entire Lisk SDK code base has been migrated to TypeScript, and the Lisk Framework was the last package left to be migrated, hence all Lisk Framework applications and related tests have now been updated to TypeScript.

Change to embedded database system

The PostgreSQL database has been replaced with an embedded database which reduces the preparation complexity for the Lisk SDK developer, optimizes the database storage size, provides open potential for cross-platform support, and also an increase in the read/write performance.

Remove undo steps

The need for explicit undo steps in transactions and blocks was removed. This enabled us to implement a "fail safe" delegate banning mechanism, which bans un-productive delegates from forging. The state difference is calculated automatically per block and stored in the database. When reverting blocks, for example in case of a fork, we use those stored state differences to revert back to the previous state.

Support IPC and WS communication to the Lisk Framework

We have improved the IPC and WS communication to Lisk Framework, and now the possibility exists for applications to communicate with the running node without using a plugin. Hence, all actions and events that are exposed from the application are now usable, including module to module communication. It is now possible for developers to invoke exposed actions (RPC) from an application. For this purpose, we have upgraded our APIClient (@liskhq/lisk-api-client) to be able to communicate through the “ws” and “ipc” port.

Update Lisk Commander to be network agnostic

The Lisk Commander specific functionalities related to the Lisk Core have now been migrated to Lisk Core and are network agnostic. Features for developers to build their own blockchain application will be introduced to Lisk Commander in the coming minor releases.

Simplify HTTP API plugin and separate from the Lisk Framework

The dependencies on the database indexes have now been removed where it reduces the blockchain performance. So now the HTTP API only uses the Lisk Framework actions and events to provide its endpoints and is also separated so that it can operate outside of Lisk Framework, and can also be installed separately if required.

Create forger plugin

A new forging helper plugin has now been introduced that provides and notifies forgers with more detailed information. This is designed to track the forging activity and also collect all the voters information, so that the delegate could use these actions to get the information about forging and voters.

Introduce on-chain architecture

The introduction of on-chain architecture has now been implemented which offers the developers the ability to make block level state changes, define per module account schemas, and encapsulate the application domain properly. The interface for registering custom transactions has been changed from registerTransaction to registerModule. By default, the Lisk SDK will now provide the following four modules: Tokens, Sequences, Keys, and Dpos.

Create monitor plugin

A plugin has now been created and introduced which provides utility endpoints and diagnostic information about the attached node. This has proved to be beneficial as additional measurements to reveal the actual status of the attached node or network are often required. The monitor plugin is mainly responsible for capturing application metrics and exposing statistics such as transactions, blocks, network, and forks. In addition, it also exposes an endpoint for the prometheus metrics.

Resources for Betanet v5

The Lisk Faucet was updated to 3.0.0-beta.2 to support the Betanet changes. Furthermore, the Lisk SDK documentation and Lisk Core documentation have been updated to reflect all new changes that have been implemented. Due to Lisk Desktop not being ready yet, creating transactions and monitoring applications for blocks and transactions is only possible with the new CLI interface of Lisk Core.

Known Issues

Currently there are no existing issues, as they have been rectified in Lisk SDK 5.0.0. Therefore, we ask everyone to test Lisk Core 3.0.0-beta.2 thoroughly and make use of our Bug Bounty program.

The Road Ahead with Lisk Core 3.0.0

The development and internal alpha testing for Lisk SDK 5.0.0 and Lisk Core 3.0.0-beta.2 have now been successfully completed. At the moment, the external security audit for Lisk SDK 5.0.0 and Lisk Core 3.0.0-beta.2 is in progress. After thorough testing of the Betanet v5 and the resolution of the external security audit, we will start to plan for both the Testnet and the Mainnet releases, including the migration testing next year.

More information about our release procedure can be found on the Lisk protocol roadmap and in the Lisk Core development schedule blog post. For further questions and comments regarding the new Lisk Betanet v5, we will host an AMA on Lisk.chat with Shusetsu Toda (Lead Backend Developer) on Friday 4th December at 3pm CET.