Sharder

Details on the Sharder, the block persistence manager

Sharder docker-compose-based deployment breakdown

Sharders are the providers responsible for persisting the blocks of the blockchain in the their local filesystem as well as the data model, they also provide APIs to access some data from the network data model.

Core responsibilities of the Sharder

• Store the blocks of the blockchain persistently.

• Store and update the network data model in a relational database (currently Postgres).

• Generation and persistence of historical data of some relations of the network data model, which is used in graphing the change of some data with time.

• Expose an HTTP API for external clients to get information about the network, especially statuses of the transactions.

Logical components of the Sharder Process

The Sharder Process consists of multiple workers (technically, lightweight execution threads or goroutines in the Go literature). Following are the most important workers running the miner operations

Block Finalization Workers

Workers responsible for finalizing blocks by executing the transactions, and generating events that will be used to update EventDB. They also run those events and update the network data model in EventDB.

Communication Workers (handlers)

Those are handlers of Node-to-Node (N2N) communication endpoint among miners and sharders. Those are responsible for sending/receiving different kinds of messages to and from the sharder among the Active Set.

Health check worker

A worker responsible for the heartbeats of the sharder which are signs of the sharder liveness. It works by generating a health check transaction periodically and communicate it with the other Active Set providers.

The Sharder HTTP API

The sharder exposes an API with endpoints used to view data from the network data model. This API is very useful for many clients that use the network. It will be discussed in more details in the Sharder API page.

Data stored by the Sharder

1. Network Data Model: These are all the data related to and changed by executing users' transactions against the smart contracts. It's persisted in a relational database often referred to as "EventDB", as its data depends on the events generated while executing the smart contracts.

2. Blocks of the blockchain: It's the sharders' main task to store and persist all the blocks generated by the network since the genesis. They're stored efficiently in the local filesystem of the sharders, and distributed among the different sharders of the system.

3. Some data is cached for fast access using rocksdb.

Requirements libraries for the Sharder process environment

The Sharder process requires the following libraries to exist in any environment it runs on:

• Rocksdb: It's the embedded in-memory key-value store used by the sharder to store some cached data for fast access.
• BLS and MCL: Two libraries used for cryptographic tasks.